Jul 18 01:54:59.329882 ip-10-0-143-8 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 18 01:54:59.329896 ip-10-0-143-8 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 18 01:54:59.329906 ip-10-0-143-8 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 18 01:54:59.330222 ip-10-0-143-8 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 18 01:55:09.571499 ip-10-0-143-8 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 18 01:55:09.571517 ip-10-0-143-8 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot e5f5666a73c745e095da2e1ec4e9a3e1 --
Jul 18 01:56:44.209237 ip-10-0-143-8 systemd[1]: Starting Kubernetes Kubelet...
Jul 18 01:56:44.952490 ip-10-0-143-8 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 18 01:56:44.952490 ip-10-0-143-8 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 18 01:56:44.952490 ip-10-0-143-8 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 18 01:56:44.952490 ip-10-0-143-8 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 18 01:56:44.952490 ip-10-0-143-8 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 18 01:56:44.952490 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.656208    2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 18 01:56:44.952490 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.661935    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 01:56:44.952490 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.661952    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 01:56:44.952490 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.661957    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 01:56:44.952490 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.661962    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 01:56:44.952490 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.661966    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 01:56:44.952490 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.661970    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 01:56:44.952490 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.661978    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 01:56:44.952490 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.661983    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 01:56:44.738169 ip-10-0-143-8 systemd[1]: Started Kubernetes Kubelet.
Jul 18 01:56:44.964640 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.661989    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 01:56:44.964640 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.661993    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 01:56:44.964640 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.661998    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 01:56:44.964640 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662001    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 01:56:44.964640 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662006    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 01:56:44.964640 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662010    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 01:56:44.964640 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662014    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 01:56:44.964640 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662020    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 01:56:44.964640 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662024    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 01:56:44.964640 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662029    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 01:56:44.964640 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662033    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 01:56:44.964640 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662037    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 01:56:44.964640 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662043    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 01:56:44.964640 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662048    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 01:56:44.964640 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662055    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 01:56:44.964640 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662060    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 01:56:44.964640 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662064    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 01:56:44.964640 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662068    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 01:56:44.964640 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662073    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 01:56:44.964640 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662077    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 01:56:44.967054 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662081    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 01:56:44.967054 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662085    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 01:56:44.967054 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662089    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 01:56:44.967054 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662093    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 01:56:44.967054 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662097    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 01:56:44.967054 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662101    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 01:56:44.967054 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662105    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 01:56:44.967054 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662109    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 01:56:44.967054 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662114    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 01:56:44.967054 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662118    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 01:56:44.967054 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662121    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 01:56:44.967054 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662126    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 01:56:44.967054 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662130    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 01:56:44.967054 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662134    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 01:56:44.967054 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662137    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 01:56:44.967054 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662140    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 01:56:44.967054 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662144    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 01:56:44.967054 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662147    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 01:56:44.967054 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662151    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 01:56:44.968822 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662154    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 01:56:44.968822 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662168    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 01:56:44.968822 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662172    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 01:56:44.968822 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662176    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 01:56:44.968822 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662180    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 01:56:44.968822 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662184    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 01:56:44.968822 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662188    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 01:56:44.968822 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662192    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 01:56:44.968822 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662196    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 01:56:44.968822 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662199    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 01:56:44.968822 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662203    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 01:56:44.968822 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662206    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 01:56:44.968822 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662209    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 01:56:44.968822 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662213    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 01:56:44.968822 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662216    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 01:56:44.968822 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662219    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 01:56:44.968822 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662223    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 01:56:44.968822 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662227    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 01:56:44.968822 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662230    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 01:56:44.968822 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662234    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 01:56:44.972739 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662238    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 01:56:44.972739 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662242    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 01:56:44.972739 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662245    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 01:56:44.972739 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662249    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 01:56:44.972739 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662252    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 01:56:44.972739 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662255    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 01:56:44.972739 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662259    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 01:56:44.972739 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662263    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 01:56:44.972739 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662266    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 01:56:44.972739 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662269    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 01:56:44.972739 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662274    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 01:56:44.972739 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662278    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 01:56:44.972739 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662281    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 01:56:44.972739 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662285    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 01:56:44.972739 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662289    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 01:56:44.972739 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662294    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 01:56:44.972739 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662298    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 01:56:44.972739 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662301    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 01:56:44.972739 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662304    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 01:56:44.972739 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662310    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 01:56:44.974960 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662314    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 01:56:44.974960 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662317    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 01:56:44.974960 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662320    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 01:56:44.974960 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662323    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 01:56:44.974960 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662326    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 01:56:44.974960 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662328    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 01:56:44.974960 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662331    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 01:56:44.974960 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662334    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 01:56:44.974960 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662337    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 01:56:44.974960 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662339    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 01:56:44.974960 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662342    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 01:56:44.974960 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.662345    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 01:56:44.974960 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.664812    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 01:56:44.974960 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.664825    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 01:56:44.974960 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.664830    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 01:56:44.974960 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.664835    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 01:56:44.974960 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.664839    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 01:56:44.974960 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.664843    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 01:56:44.974960 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.664846    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 01:56:44.974960 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.664850    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 01:56:44.978514 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.664854    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 01:56:44.978514 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.664858    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 01:56:44.978514 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.664862    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 01:56:44.978514 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.664867    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 01:56:44.978514 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.664871    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 01:56:44.978514 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.664875    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 01:56:44.978514 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.664879    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 01:56:44.978514 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.664884    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 01:56:44.978514 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.664888    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 01:56:44.978514 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.664892    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 01:56:44.978514 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.664914    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 01:56:44.978514 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.664919    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 01:56:44.978514 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.664924    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 01:56:44.978514 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.664928    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 01:56:44.978514 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.664932    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 01:56:44.978514 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.664937    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 01:56:44.978514 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.664941    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 01:56:44.978514 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.664945    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 01:56:44.978514 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.664949    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 01:56:44.978514 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.664953    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 01:56:44.980767 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.664959    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 01:56:44.980767 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.664964    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 01:56:44.980767 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.664968    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 01:56:44.980767 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.664973    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 01:56:44.980767 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.664976    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 01:56:44.980767 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.664983    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 01:56:44.980767 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.664987    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 01:56:44.980767 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.664991    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 01:56:44.980767 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.664995    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 01:56:44.980767 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.664999    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 01:56:44.980767 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665003    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 01:56:44.980767 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665006    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 01:56:44.980767 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665010    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 01:56:44.980767 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665014    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 01:56:44.980767 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665018    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 01:56:44.980767 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665022    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 01:56:44.980767 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665028    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 01:56:44.980767 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665034    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 01:56:44.980767 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665038    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 01:56:44.981872 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665042    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 01:56:44.981872 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665058    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 01:56:44.981872 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665062    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 01:56:44.981872 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665068    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 01:56:44.981872 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665072    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 01:56:44.981872 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665076    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 01:56:44.981872 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665079    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 01:56:44.981872 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665083    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 01:56:44.981872 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665087    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 01:56:44.981872 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665091    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 01:56:44.981872 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665097    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 01:56:44.981872 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665102    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 01:56:44.981872 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665107    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 01:56:44.981872 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665111    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 01:56:44.981872 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665114    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 01:56:44.981872 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665118    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 01:56:44.981872 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665122    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 01:56:44.981872 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665125    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 01:56:44.981872 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665131    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 01:56:44.981872 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665136    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 01:56:44.982951 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665140    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 01:56:44.982951 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665143    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 01:56:44.982951 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665147    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 01:56:44.982951 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665151    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 01:56:44.982951 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665154    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 01:56:44.982951 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665158    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 01:56:44.982951 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665164    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 01:56:44.982951 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665169    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 01:56:44.982951 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665174    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 01:56:44.982951 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665178    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 01:56:44.982951 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665182    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 01:56:44.982951 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665186    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 01:56:44.982951 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665190    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 01:56:44.982951 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665194    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 01:56:44.982951 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665199    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 01:56:44.982951 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665206    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 01:56:44.982951 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665209    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 01:56:44.982951 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665212    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 01:56:44.982951 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665215    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 01:56:44.982951 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665217    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 01:56:44.983934 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665220    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 01:56:44.983934 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665222    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 01:56:44.983934 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665225    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 01:56:44.983934 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665227    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 01:56:44.983934 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665230    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 01:56:44.983934 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665233    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 01:56:44.983934 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665235    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 01:56:44.983934 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665238    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 01:56:44.983934 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665240    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 01:56:44.983934 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665243    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 01:56:44.983934 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665247    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 01:56:44.983934 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.665249    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 01:56:44.983934 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666276    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 01:56:44.983934 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666283    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 01:56:44.983934 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666287    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 01:56:44.983934 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666290    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 01:56:44.983934 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666293    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 01:56:44.983934 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666295    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 01:56:44.983934 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666298    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 01:56:44.983934 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666300    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 01:56:44.984941 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666303    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 01:56:44.984941 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666306    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 01:56:44.984941 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666309    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 01:56:44.984941 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666312    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 01:56:44.984941 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666314    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 01:56:44.984941 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666317    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 01:56:44.984941 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666320    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 01:56:44.984941 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666324    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 01:56:44.984941 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666327    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 01:56:44.984941 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666330    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 01:56:44.984941 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666332    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 01:56:44.984941 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666335    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 01:56:44.984941 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666337    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 01:56:44.984941 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666340    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 01:56:44.984941 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666343    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 01:56:44.984941 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666346    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 01:56:44.984941 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666349    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 01:56:44.984941 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666351    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 01:56:44.984941 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666354    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 01:56:44.984941 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666357    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 01:56:44.985714 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666359    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 01:56:44.985714 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666362    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 01:56:44.985714 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666366    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 01:56:44.985714 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666369    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 01:56:44.985714 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666371    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 01:56:44.985714 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666376    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 01:56:44.985714 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666379    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 01:56:44.985714 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666382    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 01:56:44.985714 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666385    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 01:56:44.985714 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666388    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 01:56:44.985714 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666390    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 01:56:44.985714 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666393    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 01:56:44.985714 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666395    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 01:56:44.985714 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666398    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 01:56:44.985714 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666401    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 01:56:44.985714 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666403    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 01:56:44.985714 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666406    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 01:56:44.985714 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666408    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 01:56:44.985714 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666411    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 01:56:44.986467 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666415    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 01:56:44.986467 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666418    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 01:56:44.986467 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666421    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 01:56:44.986467 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666424    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 01:56:44.986467 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666427    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 01:56:44.986467 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666429    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 01:56:44.986467 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666432    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 01:56:44.986467 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666434    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 01:56:44.986467 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666436    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 01:56:44.986467 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666441    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 01:56:44.986467 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666443    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 01:56:44.986467 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666446    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 01:56:44.986467 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666448    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 01:56:44.986467 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666451    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 01:56:44.986467 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666454    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 01:56:44.986467 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666458    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 01:56:44.986467 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666460    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 01:56:44.986467 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666463    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 01:56:44.986467 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666465    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 01:56:44.986467 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666468    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 01:56:44.987017 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666471    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 01:56:44.987017 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666473    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 01:56:44.987017 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666475    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 01:56:44.987017 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666478    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 01:56:44.987017 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666481    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 01:56:44.987017 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666484    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 01:56:44.987017 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666486    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 01:56:44.987017 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666489    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 01:56:44.987017 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666491    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 01:56:44.987017 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666494    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 01:56:44.987017 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666497    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 01:56:44.987017 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666501    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 01:56:44.987017 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666506    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 01:56:44.987017 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666509    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 01:56:44.987017 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666512    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 01:56:44.987017 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666514    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 01:56:44.987017 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666518    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 01:56:44.987017 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666520    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 01:56:44.987017 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666524    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 01:56:44.987563 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666527    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 01:56:44.987563 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666530    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 01:56:44.987563 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666533    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 01:56:44.987563 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666536    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 01:56:44.987563 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666539    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 01:56:44.987563 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666542    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 01:56:44.987563 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666545    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 01:56:44.987563 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666547    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 01:56:44.987563 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666551    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 01:56:44.987563 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666553    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 01:56:44.987563 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666556    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 01:56:44.987563 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666558    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 01:56:44.987563 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.666561    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 01:56:44.987563 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666632    2576 flags.go:64] FLAG: --address="0.0.0.0"
Jul 18 01:56:44.987563 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666645    2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 18 01:56:44.987563 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666652    2576 flags.go:64] FLAG: --anonymous-auth="true"
Jul 18 01:56:44.987563 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666657    2576 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 18 01:56:44.987563 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666661    2576 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 18 01:56:44.987563 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666665    2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 18 01:56:44.987563 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666669    2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 18 01:56:44.987563 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666673    2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 18 01:56:44.988130 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666677    2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 18 01:56:44.988130 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666680    2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 18 01:56:44.988130 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666683    2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 18 01:56:44.988130 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666686    2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 18 01:56:44.988130 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666691    2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 18 01:56:44.988130 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666694    2576 flags.go:64] FLAG: --cgroup-root=""
Jul 18 01:56:44.988130 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666698    2576 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 18 01:56:44.988130 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666701    2576 flags.go:64] FLAG: --client-ca-file=""
Jul 18 01:56:44.988130 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666704    2576 flags.go:64] FLAG: --cloud-provider="external"
Jul 18 01:56:44.988130 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666707    2576 flags.go:64] FLAG: --cluster-dns="[]"
Jul 18 01:56:44.988130 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666711    2576 flags.go:64] FLAG: --cluster-domain=""
Jul 18 01:56:44.988130 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666714    2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 18 01:56:44.988130 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666717    2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 18 01:56:44.988130 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666720    2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 18 01:56:44.988130 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666724    2576 flags.go:64] FLAG: --container-log-max-files="5"
Jul 18 01:56:44.988130 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666727    2576 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 18 01:56:44.988130 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666730    2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 18 01:56:44.988130 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666733    2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 18 01:56:44.988130 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666737    2576 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 18 01:56:44.988130 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666741    2576 flags.go:64] FLAG: --contention-profiling="false"
Jul 18 01:56:44.988130 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666744    2576 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 18 01:56:44.988130 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666747    2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 18 01:56:44.988130 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666750    2576 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 18 01:56:44.988130 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666753    2576 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 18 01:56:44.988784 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666758    2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 18 01:56:44.988784 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666761    2576 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 18 01:56:44.988784 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666763    2576 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 18 01:56:44.988784 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666766    2576 flags.go:64] FLAG: --enable-load-reader="false"
Jul 18 01:56:44.988784 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666769    2576 flags.go:64] FLAG: --enable-server="true"
Jul 18 01:56:44.988784 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666772    2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 18 01:56:44.988784 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666778    2576 flags.go:64] FLAG: --event-burst="100"
Jul 18 01:56:44.988784 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666781    2576 flags.go:64] FLAG: --event-qps="50"
Jul 18 01:56:44.988784 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666784    2576 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 18 01:56:44.988784 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666787    2576 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 18 01:56:44.988784 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666790    2576 flags.go:64] FLAG: --eviction-hard=""
Jul 18 01:56:44.988784 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666794    2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 18 01:56:44.988784 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666798    2576 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 18 01:56:44.988784 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666802    2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 18 01:56:44.988784 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666805    2576 flags.go:64] FLAG: --eviction-soft=""
Jul 18 01:56:44.988784 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666808    2576 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 18 01:56:44.988784 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666811    2576 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 18 01:56:44.988784 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666814    2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 18 01:56:44.988784 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666824    2576 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 18 01:56:44.988784 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666827    2576 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 18 01:56:44.988784 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666830    2576 flags.go:64] FLAG: --fail-swap-on="true"
Jul 18 01:56:44.988784 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666833    2576 flags.go:64] FLAG: --feature-gates=""
Jul 18 01:56:44.988784 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666837    2576 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 18 01:56:44.988784 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666840    2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 18 01:56:44.988784 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666843    2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 18 01:56:44.989516 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666846    2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 18 01:56:44.989516 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666849    2576 flags.go:64] FLAG: --healthz-port="10248"
Jul 18 01:56:44.989516 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666854    2576 flags.go:64] FLAG: --help="false"
Jul 18 01:56:44.989516 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666857    2576 flags.go:64] FLAG: --hostname-override="ip-10-0-143-8.ec2.internal"
Jul 18 01:56:44.989516 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666860    2576 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 18 01:56:44.989516 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666863    2576 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 18 01:56:44.989516 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666866    2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 18 01:56:44.989516 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666870    2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 18 01:56:44.989516 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666873    2576 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 18 01:56:44.989516 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666876    2576 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 18 01:56:44.989516 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666879    2576 flags.go:64] FLAG: --image-service-endpoint=""
Jul 18 01:56:44.989516 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666882    2576 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 18 01:56:44.989516 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666885    2576 flags.go:64] FLAG: --kube-api-burst="100"
Jul 18 01:56:44.989516 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666888    2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 18 01:56:44.989516 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666891    2576 flags.go:64] FLAG: --kube-api-qps="50"
Jul 18 01:56:44.989516 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666894    2576 flags.go:64] FLAG: --kube-reserved=""
Jul 18 01:56:44.989516 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666916    2576 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 18 01:56:44.989516 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666919    2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 18 01:56:44.989516 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666923    2576 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 18 01:56:44.989516 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666927    2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 18 01:56:44.989516 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666930    2576 flags.go:64] FLAG: --lock-file=""
Jul 18 01:56:44.989516 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666933    2576 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 18 01:56:44.989516 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666936    2576 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 18 01:56:44.989516 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666940    2576 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 18 01:56:44.990198 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666946    2576 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 18 01:56:44.990198 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666949    2576 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 18 01:56:44.990198 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666952    2576 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 18 01:56:44.990198 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666955    2576 flags.go:64] FLAG: --logging-format="text"
Jul 18 01:56:44.990198 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666958    2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 18 01:56:44.990198 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666962    2576 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 18 01:56:44.990198 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666965    2576 flags.go:64] FLAG: --manifest-url=""
Jul 18 01:56:44.990198 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666968    2576 flags.go:64] FLAG: --manifest-url-header=""
Jul 18 01:56:44.990198 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666972    2576 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 18 01:56:44.990198 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666975    2576 flags.go:64] FLAG: --max-open-files="1000000"
Jul 18 01:56:44.990198 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666981    2576 flags.go:64] FLAG: --max-pods="110"
Jul 18 01:56:44.990198 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666985    2576 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 18 01:56:44.990198 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666988    2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 18 01:56:44.990198 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666991    2576 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 18 01:56:44.990198 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666993    2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 18 01:56:44.990198 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666996    2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 18 01:56:44.990198 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.666999    2576 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 18 01:56:44.990198 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667002    2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 18 01:56:44.990198 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667010    2576 flags.go:64] FLAG: --node-status-max-images="50"
Jul 18 01:56:44.990198 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667013    2576 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 18 01:56:44.990198 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667016    2576 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 18 01:56:44.990198 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667019    2576 flags.go:64] FLAG: --pod-cidr=""
Jul 18 01:56:44.990198 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667022    2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 18 01:56:44.991126 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667027    2576 flags.go:64] FLAG: --pod-manifest-path=""
Jul 18 01:56:44.991126 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667030    2576 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 18 01:56:44.991126 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667033    2576 flags.go:64] FLAG: --pods-per-core="0"
Jul 18 01:56:44.991126 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667036    2576 flags.go:64] FLAG: --port="10250"
Jul 18 01:56:44.991126 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667039    2576 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 18 01:56:44.991126 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667043    2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0dc90a8b2fe75ade2"
Jul 18 01:56:44.991126 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667046    2576 flags.go:64] FLAG: --qos-reserved=""
Jul 18 01:56:44.991126 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667049    2576 flags.go:64] FLAG: --read-only-port="10255"
Jul 18 01:56:44.991126 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667052    2576 flags.go:64] FLAG: --register-node="true"
Jul 18 01:56:44.991126 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667056    2576 flags.go:64] FLAG: --register-with-taints=""
Jul 18 01:56:44.991126 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667059    2576 flags.go:64] FLAG: --registry-burst="10"
Jul 18 01:56:44.991126 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667062    2576 flags.go:64] FLAG: --registry-qps="5"
Jul 18 01:56:44.991126 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667065    2576 flags.go:64] FLAG: --reserved-cpus=""
Jul 18 01:56:44.991126 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667068    2576 flags.go:64] FLAG: --reserved-memory=""
Jul 18 01:56:44.991126 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667072    2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 18 01:56:44.991126 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667075    2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 18 01:56:44.991126 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667078    2576 flags.go:64] FLAG: --rotate-certificates="false"
Jul 18 01:56:44.991126 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667081    2576 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 18 01:56:44.991126 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667083    2576 flags.go:64] FLAG: --runonce="false"
Jul 18 01:56:44.991126 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667088    2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 18 01:56:44.991126 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667091    2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 18 01:56:44.991126 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667093    2576 flags.go:64] FLAG: --seccomp-default="false"
Jul 18 01:56:44.991126 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667096    2576 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 18 01:56:44.991126 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667099    2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 18 01:56:44.991126 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667102    2576 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 18 01:56:44.991126 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667105    2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 18 01:56:44.991853 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667109    2576 flags.go:64] FLAG: --storage-driver-password="root"
Jul 18 01:56:44.991853 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667112    2576 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 18 01:56:44.991853 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667114    2576 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 18 01:56:44.991853 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667117    2576 flags.go:64] FLAG: --storage-driver-user="root"
Jul 18 01:56:44.991853 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667120    2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 18 01:56:44.991853 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667124    2576 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 18 01:56:44.991853 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667126    2576 flags.go:64] FLAG: --system-cgroups=""
Jul 18 01:56:44.991853 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667129    2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 18 01:56:44.991853 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667135    2576 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 18 01:56:44.991853 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667138    2576 flags.go:64] FLAG: --tls-cert-file=""
Jul 18 01:56:44.991853 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667141    2576 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 18 01:56:44.991853 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667146    2576 flags.go:64] FLAG: --tls-min-version=""
Jul 18 01:56:44.991853 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667149    2576 flags.go:64] FLAG: --tls-private-key-file=""
Jul 18 01:56:44.991853 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667152    2576 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 18 01:56:44.991853 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667155    2576 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 18 01:56:44.991853 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667158    2576 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 18 01:56:44.991853 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667161    2576 flags.go:64] FLAG: --v="2"
Jul 18 01:56:44.991853 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667165    2576 flags.go:64] FLAG: --version="false"
Jul 18 01:56:44.991853 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667169    2576 flags.go:64] FLAG: --vmodule=""
Jul 18 01:56:44.991853 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667173    2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 18 01:56:44.991853 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.667177    2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 18 01:56:44.991853 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667296    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 01:56:44.991853 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667300    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 01:56:44.991853 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667303    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 01:56:45.192593 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667306    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 01:56:45.192593 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667311    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 01:56:45.192593 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667314    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 01:56:45.192593 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667317    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 01:56:45.192593 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667319    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 01:56:45.192593 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667322    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 01:56:45.192593 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667325    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 01:56:45.192593 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667328    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 01:56:45.192593 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667330    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 01:56:45.192593 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667333    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 01:56:45.192593 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667335    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 01:56:45.192593 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667338    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 01:56:45.192593 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667340    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 01:56:45.192593 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667343    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 01:56:45.192593 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667345    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 01:56:45.192593 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667348    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 01:56:45.192593 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667350    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 01:56:45.192593 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667353    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 01:56:45.192593 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667355    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 01:56:45.192593 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667359    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 01:56:45.193311 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667362    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 01:56:45.193311 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667364    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 01:56:45.193311 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667367    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 01:56:45.193311 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667369    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 01:56:45.193311 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667372    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 01:56:45.193311 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667375    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 01:56:45.193311 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667378    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 01:56:45.193311 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667380    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 01:56:45.193311 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667383    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 01:56:45.193311 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667386    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 01:56:45.193311 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667388    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 01:56:45.193311 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667391    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 01:56:45.193311 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667393    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 01:56:45.193311 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667397    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 01:56:45.193311 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667399    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 01:56:45.193311 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667402    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 01:56:45.193311 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667405    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 01:56:45.193311 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667407    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 01:56:45.193311 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667410    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 01:56:45.193883 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667413    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 01:56:45.193883 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667415    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 01:56:45.193883 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667417    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 01:56:45.193883 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667420    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 01:56:45.193883 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667423    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 01:56:45.193883 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667425    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 01:56:45.193883 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667427    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 01:56:45.193883 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667430    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 01:56:45.193883 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667432    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 01:56:45.193883 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667435    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 01:56:45.193883 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667437    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 01:56:45.193883 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667440    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 01:56:45.193883 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667445    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 01:56:45.193883 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667448    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 01:56:45.193883 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667450    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 01:56:45.193883 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667453    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 01:56:45.193883 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667455    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 01:56:45.193883 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667459    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 01:56:45.193883 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667463    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 01:56:45.194446 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667468    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 01:56:45.194446 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667471    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 01:56:45.194446 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667474    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 01:56:45.194446 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667477    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 01:56:45.194446 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667479    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 01:56:45.194446 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667482    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 01:56:45.194446 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667485    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 01:56:45.194446 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667488    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 01:56:45.194446 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667491    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 01:56:45.194446 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667493    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 01:56:45.194446 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667496    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 01:56:45.194446 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667499    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 01:56:45.194446 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667501    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 01:56:45.194446 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667504    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 01:56:45.194446 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667506    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 01:56:45.194446 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667509    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 01:56:45.194446 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667511    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 01:56:45.194446 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667514    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 01:56:45.194446 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667516    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 01:56:45.194964 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667519    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 01:56:45.194964 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667522    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 01:56:45.194964 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667524    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 01:56:45.194964 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667527    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 01:56:45.194964 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667529    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 01:56:45.194964 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667531    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 01:56:45.194964 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667536    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 01:56:45.194964 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667538    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 01:56:45.194964 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667541    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 01:56:45.194964 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667545    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 01:56:45.194964 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667548    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 01:56:45.194964 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667551    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 01:56:45.194964 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667554    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 01:56:45.194964 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667557    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 01:56:45.194964 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667560    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 01:56:45.194964 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667562    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 01:56:45.194964 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667565    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 01:56:45.194964 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667567    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 01:56:45.194964 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.667570    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 01:56:45.195502 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.668328    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 18 01:56:45.195502 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.678005    2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 18 01:56:45.195502 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.678027    2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 18 01:56:45.195502 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.678044    2576 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 18 01:56:45.195502 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.678050    2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 18 01:56:45.195502 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678116    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 01:56:45.195502 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678124    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 01:56:45.195502 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678129    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 01:56:45.195502 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678134    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 01:56:45.195502 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678137    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 01:56:45.195502 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678140    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 01:56:45.195502 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678144    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 01:56:45.195502 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678147    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 01:56:45.195502 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678150    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 01:56:45.195909 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678154    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 01:56:45.195909 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678157    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 01:56:45.195909 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678160    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 01:56:45.195909 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678163    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 01:56:45.195909 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678167    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 01:56:45.195909 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678170    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 01:56:45.195909 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678173    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 01:56:45.195909 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678177    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 01:56:45.195909 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678180    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 01:56:45.195909 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678183    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 01:56:45.195909 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678186    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 01:56:45.195909 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678191    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 01:56:45.195909 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678194    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 01:56:45.195909 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678196    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 01:56:45.195909 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678200    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 01:56:45.195909 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678204    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 01:56:45.195909 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678206    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 01:56:45.195909 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678209    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 01:56:45.195909 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678213    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 01:56:45.195909 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678218    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 01:56:45.196495 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678224    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 01:56:45.196495 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678227    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 01:56:45.196495 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678230    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 01:56:45.196495 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678233    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 01:56:45.196495 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678237    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 01:56:45.196495 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678240    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 01:56:45.196495 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678245    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 01:56:45.196495 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678249    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 01:56:45.196495 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678253    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 01:56:45.196495 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678256    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 01:56:45.196495 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678259    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 01:56:45.196495 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678263    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 01:56:45.196495 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678266    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 01:56:45.196495 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678271    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 01:56:45.196495 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678273    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 01:56:45.196495 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678277    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 01:56:45.196495 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678281    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 01:56:45.196495 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678283    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 01:56:45.196495 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678288    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 01:56:45.197031 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678293    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 01:56:45.197031 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678296    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 01:56:45.197031 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678301    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 01:56:45.197031 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678304    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 01:56:45.197031 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678308    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 01:56:45.197031 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678312    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 01:56:45.197031 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678315    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 01:56:45.197031 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678318    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 01:56:45.197031 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678321    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 01:56:45.197031 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678325    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 01:56:45.197031 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678327    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 01:56:45.197031 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678330    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 01:56:45.197031 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678337    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 01:56:45.197031 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678340    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 01:56:45.197031 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678343    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 01:56:45.197031 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678347    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 01:56:45.197031 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678350    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 01:56:45.197031 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678353    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 01:56:45.197031 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678356    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 01:56:45.197031 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678360    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 01:56:45.197600 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678363    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 01:56:45.197600 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678366    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 01:56:45.197600 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678370    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 01:56:45.197600 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678373    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 01:56:45.197600 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678375    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 01:56:45.197600 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678378    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 01:56:45.197600 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678382    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 01:56:45.197600 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678384    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 01:56:45.197600 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678388    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 01:56:45.197600 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678391    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 01:56:45.197600 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678395    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 01:56:45.197600 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678399    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 01:56:45.197600 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678401    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 01:56:45.197600 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678405    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 01:56:45.197600 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678408    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 01:56:45.197600 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678410    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 01:56:45.197600 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678413    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 01:56:45.197600 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678417    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 01:56:45.197600 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678419    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 01:56:45.197600 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678422    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 01:56:45.198133 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678425    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 01:56:45.198133 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678429    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 01:56:45.198133 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678432    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 01:56:45.198133 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678435    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 01:56:45.198133 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678439    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 01:56:45.198133 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678442    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 01:56:45.198133 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678445    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 01:56:45.198133 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678450    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 01:56:45.198133 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678454    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 01:56:45.198133 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678457    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 01:56:45.198133 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678460    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 01:56:45.198133 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.678465    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 18 01:56:45.198133 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678616    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 01:56:45.198133 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678623    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 01:56:45.198133 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678627    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 01:56:45.198613 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678630    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 01:56:45.198613 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678634    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 01:56:45.198613 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678637    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 01:56:45.198613 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678640    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 01:56:45.198613 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678644    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 01:56:45.198613 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678647    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 01:56:45.198613 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678650    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 01:56:45.198613 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678654    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 01:56:45.198613 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678657    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 01:56:45.198613 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678661    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 01:56:45.198613 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678663    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 01:56:45.198613 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678666    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 01:56:45.198613 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678669    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 01:56:45.198613 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678672    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 01:56:45.198613 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678676    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 01:56:45.198613 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678679    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 01:56:45.198613 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678682    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 01:56:45.198613 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678686    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 01:56:45.198613 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678689    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 01:56:45.198613 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678692    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 01:56:45.199165 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678695    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 01:56:45.199165 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678700    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 01:56:45.199165 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678702    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 01:56:45.199165 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678707    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 01:56:45.199165 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678712    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 01:56:45.199165 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678716    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 01:56:45.199165 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678720    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 01:56:45.199165 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678724    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 01:56:45.199165 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678727    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 01:56:45.199165 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678730    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 01:56:45.199165 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678734    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 01:56:45.199165 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678737    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 01:56:45.199165 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678740    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 01:56:45.199165 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678744    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 01:56:45.199165 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678747    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 01:56:45.199165 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678749    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 01:56:45.199165 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678753    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 01:56:45.199165 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678757    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 01:56:45.199165 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678760    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 01:56:45.199700 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678762    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 01:56:45.199700 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678767    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 01:56:45.199700 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678771    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 01:56:45.199700 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678776    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 01:56:45.199700 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678780    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 01:56:45.199700 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678783    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 01:56:45.199700 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678786    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 01:56:45.199700 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678790    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 01:56:45.199700 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678792    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 01:56:45.199700 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678795    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 01:56:45.199700 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678798    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 01:56:45.199700 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678802    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 01:56:45.199700 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678805    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 01:56:45.199700 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678808    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 01:56:45.199700 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678813    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 01:56:45.199700 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678816    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 01:56:45.199700 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678818    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 01:56:45.199700 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678822    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 01:56:45.199700 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678827    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 01:56:45.200236 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678830    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 01:56:45.200236 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678834    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 01:56:45.200236 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678837    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 01:56:45.200236 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678840    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 01:56:45.200236 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678843    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 01:56:45.200236 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678846    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 01:56:45.200236 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678849    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 01:56:45.200236 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678852    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 01:56:45.200236 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678855    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 01:56:45.200236 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678859    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 01:56:45.200236 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678862    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 01:56:45.200236 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678865    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 01:56:45.200236 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678868    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 01:56:45.200236 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678870    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 01:56:45.200236 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678873    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 01:56:45.200236 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678876    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 01:56:45.200236 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678881    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 01:56:45.200236 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678886    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 01:56:45.200236 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678891    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 01:56:45.200797 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678894    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 01:56:45.200797 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678917    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 01:56:45.200797 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678921    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 01:56:45.200797 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678925    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 01:56:45.200797 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678928    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 01:56:45.200797 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678930    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 01:56:45.200797 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678933    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 01:56:45.200797 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678936    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 01:56:45.200797 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678940    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 01:56:45.200797 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678942    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 01:56:45.200797 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678945    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 01:56:45.200797 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678948    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 01:56:45.200797 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678952    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 01:56:45.200797 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678954    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 01:56:45.200797 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678957    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 01:56:45.200797 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678960    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 01:56:45.200797 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678963    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 01:56:45.200797 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678966    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 01:56:45.200797 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:44.678968    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 01:56:45.201341 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.678973    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 18 01:56:45.201341 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.679108    2576 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 18 01:56:45.201341 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.682361    2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 18 01:56:45.201341 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.684112    2576 server.go:1019] "Starting client certificate rotation"
Jul 18 01:56:45.201341 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.684218    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 18 01:56:45.201341 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.684255    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 18 01:56:45.201341 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.687158    2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 18 01:56:45.201341 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.688362    2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 18 01:56:45.201341 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.698283    2576 log.go:25] "Validated CRI v1 runtime API"
Jul 18 01:56:45.201341 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.700787    2576 log.go:25] "Validated CRI v1 image API"
Jul 18 01:56:45.201341 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.701612    2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 18 01:56:45.201341 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.704393    2576 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 97ca98c5-305a-4bda-941f-be26143d70a9:/dev/nvme0n1p3 b93eaa19-088d-4c29-af06-bad80e58e93e:/dev/nvme0n1p4]
Jul 18 01:56:45.201718 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.704408    2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 18 01:56:45.201788 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.712134    2576 manager.go:216] Machine: {Timestamp:2026-07-18 01:56:44.710529962 +0000 UTC m=+0.369095114 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103455 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2512ea16480477b78363ca7adbee6e SystemUUID:ec2512ea-1648-0477-b783-63ca7adbee6e BootID:e5f5666a-73c7-45e0-95da-2e1ec4e9a3e1 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:22:76:7a:73:f1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:22:76:7a:73:f1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:3a:7d:cf:98:49:eb Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 18 01:56:45.201788 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.712262    2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 18 01:56:45.201788 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.712373    2576 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 18 01:56:45.201788 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.716011    2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 18 01:56:45.201788 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.716048    2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-8.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 18 01:56:45.201788 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.716926    2576 topology_manager.go:138] "Creating topology manager with none policy"
Jul 18 01:56:45.201788 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.716938    2576 container_manager_linux.go:309] "Creating device plugin manager"
Jul 18 01:56:45.201788 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.716945    2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 18 01:56:45.202156 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.717798    2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 18 01:56:45.202156 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.718640    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 18 01:56:45.202156 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.719754    2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 18 01:56:45.202156 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.724524    2576 state_mem.go:36] "Initialized new in-memory state store"
Jul 18 01:56:45.202156 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.724681    2576 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 18 01:56:45.202156 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.724709    2576 kubelet.go:486] "Attempting to sync node with API server"
Jul 18 01:56:45.202156 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.724723    2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 18 01:56:45.202156 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.724739    2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 18 01:56:45.202156 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.724752    2576 kubelet.go:398] "Adding apiserver pod source"
Jul 18 01:56:45.202156 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.724773    2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 18 01:56:45.202156 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.725214    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 18 01:56:45.202156 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.725233    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 18 01:56:45.202156 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.728571    2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 18 01:56:45.202156 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.731218    2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 18 01:56:45.202156 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.731264    2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 18 01:56:45.202156 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.732658    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 18 01:56:45.202156 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.732671    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 18 01:56:45.202156 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.732678    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 18 01:56:45.202156 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.732683    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 18 01:56:45.202156 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.732695    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 18 01:56:45.202156 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.732703    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 18 01:56:45.202156 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.732712    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 18 01:56:45.202156 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.732717    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 18 01:56:45.202156 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.732725    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 18 01:56:45.202156 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.732730    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 18 01:56:45.202156 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.732739    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 18 01:56:45.202156 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.732745    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 18 01:56:45.202156 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.733740    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 18 01:56:45.202156 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.733750    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 18 01:56:45.202156 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:44.736539    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 18 01:56:45.202156 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:44.736541    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-8.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 18 01:56:45.202156 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.736582    2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-8.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 18 01:56:45.202156 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.737137    2576 server.go:1268] "Started kubelet"
Jul 18 01:56:45.202156 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.737210    2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 18 01:56:45.202156 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.737265    2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 18 01:56:45.202156 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.737330    2576 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 18 01:56:45.202156 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.739500    2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 18 01:56:45.202156 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.739762    2576 server.go:310] "Adding debug handlers to kubelet server"
Jul 18 01:56:45.202156 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:44.742423    2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 18 01:56:45.203630 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:44.745309    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-8.ec2.internal.18c33f993bf1ec76  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-8.ec2.internal,UID:ip-10-0-143-8.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-8.ec2.internal,},FirstTimestamp:2026-07-18 01:56:44.737105014 +0000 UTC m=+0.395670175,LastTimestamp:2026-07-18 01:56:44.737105014 +0000 UTC m=+0.395670175,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-8.ec2.internal,}"
Jul 18 01:56:45.203630 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.749608    2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 18 01:56:45.203630 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.749624    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 18 01:56:45.203630 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:44.749706    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-8.ec2.internal\" not found"
Jul 18 01:56:45.203630 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.750374    2576 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 18 01:56:45.203630 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.750389    2576 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 18 01:56:45.203630 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.750422    2576 factory.go:55] Registering systemd factory
Jul 18 01:56:45.203630 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.750436    2576 factory.go:223] Registration of the systemd container factory successfully
Jul 18 01:56:45.203630 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.750376    2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 18 01:56:45.203630 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.750502    2576 reconstruct.go:97] "Volume reconstruction finished"
Jul 18 01:56:45.203630 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.750510    2576 reconciler.go:29] "Reconciler: start to sync state"
Jul 18 01:56:45.203630 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.750630    2576 factory.go:153] Registering CRI-O factory
Jul 18 01:56:45.203630 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.750641    2576 factory.go:223] Registration of the crio container factory successfully
Jul 18 01:56:45.203630 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.750684    2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 18 01:56:45.203630 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.750720    2576 factory.go:103] Registering Raw factory
Jul 18 01:56:45.203630 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.750735    2576 manager.go:1196] Started watching for new ooms in manager
Jul 18 01:56:45.203630 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.751359    2576 manager.go:349] Starting recovery of all containers
Jul 18 01:56:45.203630 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:44.751744    2576 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-8.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 18 01:56:45.203630 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:44.751818    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 18 01:56:45.203630 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.762065    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gx9rc"
Jul 18 01:56:45.203630 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.764010    2576 manager.go:354] Recovery completed
Jul 18 01:56:45.203630 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.768963    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 01:56:45.203630 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.769373    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gx9rc"
Jul 18 01:56:45.203630 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.772313    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-8.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 01:56:45.203630 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.772342    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-8.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 01:56:45.203630 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.772367    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-8.ec2.internal" event="NodeHasSufficientPID"
Jul 18 01:56:45.203630 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.773590    2576 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 18 01:56:45.203630 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.773605    2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 18 01:56:45.203630 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.773611    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 01:56:45.203630 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.773623    2576 state_mem.go:36] "Initialized new in-memory state store"
Jul 18 01:56:45.203630 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.774592    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-8.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 01:56:45.203630 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.774621    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-8.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 01:56:45.203630 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.774634    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-8.ec2.internal" event="NodeHasSufficientPID"
Jul 18 01:56:45.398554 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.775862    2576 policy_none.go:49] "None policy: Start"
Jul 18 01:56:45.398554 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.775875    2576 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 18 01:56:45.398554 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.775893    2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 18 01:56:45.398554 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.778599    2576 policy_none.go:47] "Start"
Jul 18 01:56:45.398554 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.841602    2576 manager.go:337] "Starting Device Plugin manager"
Jul 18 01:56:45.398554 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:44.841634    2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 18 01:56:45.398554 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.841643    2576 server.go:87] "Starting device plugin registration server"
Jul 18 01:56:45.398554 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.841984    2576 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 18 01:56:45.398554 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.841999    2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 18 01:56:45.398554 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.842112    2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 18 01:56:45.398554 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.842215    2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 18 01:56:45.398554 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.842248    2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 18 01:56:45.398554 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:44.842750    2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 18 01:56:45.398554 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:44.842807    2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-8.ec2.internal\" not found"
Jul 18 01:56:45.398554 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.927656    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 18 01:56:45.398554 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.928894    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 18 01:56:45.398554 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.928967    2576 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 18 01:56:45.398554 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.929039    2576 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 18 01:56:45.398554 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:44.929083    2576 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 18 01:56:45.398554 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.931279    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 01:56:45.398554 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.943083    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 01:56:45.398554 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.943961    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-8.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 01:56:45.398554 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.943989    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-8.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 01:56:45.398554 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.943998    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-8.ec2.internal" event="NodeHasSufficientPID"
Jul 18 01:56:45.398554 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.944020    2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-8.ec2.internal"
Jul 18 01:56:45.398554 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:44.951061    2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-8.ec2.internal"
Jul 18 01:56:45.398554 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:44.951081    2576 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-8.ec2.internal\": node \"ip-10-0-143-8.ec2.internal\" not found"
Jul 18 01:56:45.398554 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:44.977931    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-8.ec2.internal\" not found"
Jul 18 01:56:45.398554 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:45.029673    2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-8.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-8.ec2.internal"]
Jul 18 01:56:45.398554 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:45.029788    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 01:56:45.398554 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:45.030932    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-8.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 01:56:45.398554 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:45.030954    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-8.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 01:56:45.398554 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:45.030972    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-8.ec2.internal" event="NodeHasSufficientPID"
Jul 18 01:56:45.398554 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:45.033345    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 01:56:45.398554 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:45.033854    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-8.ec2.internal"
Jul 18 01:56:45.398554 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:45.033890    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 01:56:45.398554 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:45.034039    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-8.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 01:56:45.398554 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:45.034061    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-8.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 01:56:45.401602 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:45.034074    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-8.ec2.internal" event="NodeHasSufficientPID"
Jul 18 01:56:45.401602 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:45.034561    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-8.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 01:56:45.401602 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:45.034583    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-8.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 01:56:45.401602 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:45.034592    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-8.ec2.internal" event="NodeHasSufficientPID"
Jul 18 01:56:45.401602 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:45.036178    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-8.ec2.internal"
Jul 18 01:56:45.401602 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:45.036199    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 01:56:45.401602 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:45.036797    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-8.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 01:56:45.401602 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:45.036818    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-8.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 01:56:45.401602 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:45.036831    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-8.ec2.internal" event="NodeHasSufficientPID"
Jul 18 01:56:45.401602 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:45.052340    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5bf26a2caa79e4ddbb4f56b4c1019bf4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-8.ec2.internal\" (UID: \"5bf26a2caa79e4ddbb4f56b4c1019bf4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-8.ec2.internal"
Jul 18 01:56:45.401602 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:45.052365    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5bf26a2caa79e4ddbb4f56b4c1019bf4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-8.ec2.internal\" (UID: \"5bf26a2caa79e4ddbb4f56b4c1019bf4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-8.ec2.internal"
Jul 18 01:56:45.401602 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:45.052384    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ef3a0a5df8ac39131447779dfceb447b-config\") pod \"kube-apiserver-proxy-ip-10-0-143-8.ec2.internal\" (UID: \"ef3a0a5df8ac39131447779dfceb447b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-8.ec2.internal"
Jul 18 01:56:45.401602 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:45.053472    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-8.ec2.internal\" not found" node="ip-10-0-143-8.ec2.internal"
Jul 18 01:56:45.401602 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:45.056851    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-8.ec2.internal\" not found" node="ip-10-0-143-8.ec2.internal"
Jul 18 01:56:45.401602 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:45.078938    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-8.ec2.internal\" not found"
Jul 18 01:56:45.401602 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:45.152889    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5bf26a2caa79e4ddbb4f56b4c1019bf4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-8.ec2.internal\" (UID: \"5bf26a2caa79e4ddbb4f56b4c1019bf4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-8.ec2.internal"
Jul 18 01:56:45.401602 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:45.152985    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5bf26a2caa79e4ddbb4f56b4c1019bf4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-8.ec2.internal\" (UID: \"5bf26a2caa79e4ddbb4f56b4c1019bf4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-8.ec2.internal"
Jul 18 01:56:45.401602 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:45.153068    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ef3a0a5df8ac39131447779dfceb447b-config\") pod \"kube-apiserver-proxy-ip-10-0-143-8.ec2.internal\" (UID: \"ef3a0a5df8ac39131447779dfceb447b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-8.ec2.internal"
Jul 18 01:56:45.401602 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:45.153081    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ef3a0a5df8ac39131447779dfceb447b-config\") pod \"kube-apiserver-proxy-ip-10-0-143-8.ec2.internal\" (UID: \"ef3a0a5df8ac39131447779dfceb447b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-8.ec2.internal"
Jul 18 01:56:45.401602 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:45.153116    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5bf26a2caa79e4ddbb4f56b4c1019bf4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-8.ec2.internal\" (UID: \"5bf26a2caa79e4ddbb4f56b4c1019bf4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-8.ec2.internal"
Jul 18 01:56:45.402411 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:45.153175    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5bf26a2caa79e4ddbb4f56b4c1019bf4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-8.ec2.internal\" (UID: \"5bf26a2caa79e4ddbb4f56b4c1019bf4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-8.ec2.internal"
Jul 18 01:56:45.402411 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:45.179086    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-8.ec2.internal\" not found"
Jul 18 01:56:45.402411 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:45.279615    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-8.ec2.internal\" not found"
Jul 18 01:56:45.402411 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:45.357125    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-8.ec2.internal"
Jul 18 01:56:45.402411 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:45.360268    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-8.ec2.internal"
Jul 18 01:56:45.402411 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:45.383224    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-8.ec2.internal\" not found"
Jul 18 01:56:45.611626 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:45.483487    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-8.ec2.internal\" not found"
Jul 18 01:56:45.611626 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:45.583795    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-8.ec2.internal\" not found"
Jul 18 01:56:46.861227 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:45.683545    2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 18 01:56:46.861227 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:45.683742    2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 18 01:56:46.861227 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:45.688477    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-8.ec2.internal\" not found"
Jul 18 01:56:46.861227 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:46.812157    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-17 01:51:45 +0000 UTC" deadline="2028-01-06 23:42:56.115464823 +0000 UTC"
Jul 18 01:56:46.861227 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:46.812205    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12909h46m9.303263486s"
Jul 18 01:56:46.861227 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:46.822287    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-8.ec2.internal\" not found"
Jul 18 01:56:46.861227 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:46.830753    2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 01:56:47.018871 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:46.883478    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-8.ec2.internal"
Jul 18 01:56:47.018871 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:46.894125    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 18 01:56:47.018871 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:46.897285    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-8.ec2.internal"
Jul 18 01:56:47.018871 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:46.931905    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 18 01:56:47.253622 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:47.047359    2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 01:56:47.253622 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:47.110292    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bf26a2caa79e4ddbb4f56b4c1019bf4.slice/crio-fffe3b0aff1eee4681e374ccf1740bd1c16ff53300efb27cad15419e7cf2e78d WatchSource:0}: Error finding container fffe3b0aff1eee4681e374ccf1740bd1c16ff53300efb27cad15419e7cf2e78d: Status 404 returned error can't find the container with id fffe3b0aff1eee4681e374ccf1740bd1c16ff53300efb27cad15419e7cf2e78d
Jul 18 01:56:47.253622 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:47.115850    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 01:56:47.385171 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:47.313660    2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 01:56:48.129920 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.071951    2576 apiserver.go:52] "Watching apiserver"
Jul 18 01:56:48.129920 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.072011    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 18 01:56:48.129920 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.074199    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-8.ec2.internal" event={"ID":"ef3a0a5df8ac39131447779dfceb447b","Type":"ContainerStarted","Data":"e7f6e32014175bebc76d2b4eb4d5dba0abb8b132ede8b8c0573a0c2f30f3b5f5"}
Jul 18 01:56:48.129920 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.075237    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-8.ec2.internal" event={"ID":"5bf26a2caa79e4ddbb4f56b4c1019bf4","Type":"ContainerStarted","Data":"fffe3b0aff1eee4681e374ccf1740bd1c16ff53300efb27cad15419e7cf2e78d"}
Jul 18 01:56:48.129920 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.079521    2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 18 01:56:48.129920 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.081085    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-grrdq","kube-system/konnectivity-agent-t7kmc","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-8.ec2.internal","openshift-multus/multus-additional-cni-plugins-sd77t","openshift-multus/network-metrics-daemon-qgrtk","openshift-network-diagnostics/network-check-target-h6pxz","openshift-network-operator/iptables-alerter-s42pb","openshift-ovn-kubernetes/ovnkube-node-jfzth","kube-system/global-pull-secret-syncer-jnffd","kube-system/kube-apiserver-proxy-ip-10-0-143-8.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k92lz","openshift-cluster-node-tuning-operator/tuned-xlrxq","openshift-dns/node-resolver-25924","openshift-image-registry/node-ca-f5kmg"]
Jul 18 01:56:48.129920 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.081651    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 18 01:56:48.129920 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.083780    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.129920 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.085833    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-t7kmc"
Jul 18 01:56:48.129920 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.086058    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 18 01:56:48.129920 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.086138    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 18 01:56:48.129920 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.086229    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-m8wvx\""
Jul 18 01:56:48.129920 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.086270    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 18 01:56:48.129920 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.086338    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 18 01:56:48.129920 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.087918    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 18 01:56:48.129920 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.088134    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-sd77t"
Jul 18 01:56:48.129920 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.088367    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-h7jnw\""
Jul 18 01:56:48.129920 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.088512    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 18 01:56:48.129920 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.090056    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-2l9g5\""
Jul 18 01:56:48.129920 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.090297    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 18 01:56:48.129920 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.090385    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qgrtk"
Jul 18 01:56:48.129920 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:48.090488    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qgrtk" podUID="48beaadf-7df1-497b-af20-2b57bb5a4250"
Jul 18 01:56:48.129920 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.090838    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 18 01:56:48.129920 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.094098    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6pxz"
Jul 18 01:56:48.133455 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:48.094165    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h6pxz" podUID="b212c851-6975-4dcc-bd43-9421f78f6c64"
Jul 18 01:56:48.133455 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.096284    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-s42pb"
Jul 18 01:56:48.133455 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.098625    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.133455 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.098858    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 18 01:56:48.133455 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.098941    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4jb2b\""
Jul 18 01:56:48.133455 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.099076    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 18 01:56:48.133455 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.099161    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 18 01:56:48.133455 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.100835    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 18 01:56:48.133455 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.100899    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jnffd"
Jul 18 01:56:48.133455 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.100932    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-t7grl\""
Jul 18 01:56:48.133455 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:48.100969    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jnffd" podUID="a370ba82-a51e-41aa-9ced-19bab3186de4"
Jul 18 01:56:48.133455 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.101023    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 18 01:56:48.133455 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.101288    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 18 01:56:48.133455 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.101298    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 18 01:56:48.133455 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.101837    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-j5t8b"
Jul 18 01:56:48.133455 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.101899    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 18 01:56:48.133455 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.101916    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 18 01:56:48.133455 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.103210    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k92lz"
Jul 18 01:56:48.133455 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.105052    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 18 01:56:48.133455 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.105136    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 18 01:56:48.133455 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.105140    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-hqh5h\""
Jul 18 01:56:48.133455 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.105420    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 18 01:56:48.133455 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.105630    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.133455 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.107848    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-25924"
Jul 18 01:56:48.133455 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.107879    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 18 01:56:48.133455 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.108044    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-5nr9r\""
Jul 18 01:56:48.133455 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.108072    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 18 01:56:48.133455 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.110028    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-l9q98\""
Jul 18 01:56:48.134538 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.110066    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 18 01:56:48.134538 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.110295    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-f5kmg"
Jul 18 01:56:48.134538 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.110406    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 18 01:56:48.134538 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.112402    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 18 01:56:48.134538 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.112537    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 18 01:56:48.134538 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.112556    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 18 01:56:48.134538 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.112657    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-jhg59\""
Jul 18 01:56:48.134538 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.114038    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-j5t8b"
Jul 18 01:56:48.230042 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.172363    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-host-var-lib-cni-multus\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.230042 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.172409    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-etc-kubernetes\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.230042 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.172444    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ad82bb0f-ce9c-416d-b049-6e6e8ddfaba8-konnectivity-ca\") pod \"konnectivity-agent-t7kmc\" (UID: \"ad82bb0f-ce9c-416d-b049-6e6e8ddfaba8\") " pod="kube-system/konnectivity-agent-t7kmc"
Jul 18 01:56:48.230042 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.172470    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-multus-cni-dir\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.230042 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.172493    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-os-release\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.230042 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.172511    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-multus-socket-dir-parent\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.230042 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.172531    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d01349d2-5a70-4df6-b978-a3cf84438584-multus-daemon-config\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.230042 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.172593    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ad82bb0f-ce9c-416d-b049-6e6e8ddfaba8-agent-certs\") pod \"konnectivity-agent-t7kmc\" (UID: \"ad82bb0f-ce9c-416d-b049-6e6e8ddfaba8\") " pod="kube-system/konnectivity-agent-t7kmc"
Jul 18 01:56:48.230042 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.172624    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-cnibin\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.230042 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.172651    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d01349d2-5a70-4df6-b978-a3cf84438584-cni-binary-copy\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.230042 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.172673    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-host-run-k8s-cni-cncf-io\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.230042 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.172696    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k95l2\" (UniqueName: \"kubernetes.io/projected/d01349d2-5a70-4df6-b978-a3cf84438584-kube-api-access-k95l2\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.230042 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.172718    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-system-cni-dir\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.230042 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.172739    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-host-run-netns\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.230988 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.172761    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-host-var-lib-cni-bin\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.230988 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.172796    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-host-var-lib-kubelet\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.230988 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.172822    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-hostroot\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.230988 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.172848    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-multus-conf-dir\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.230988 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.172875    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-host-run-multus-certs\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.230988 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.185894    2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 18 01:56:48.273588 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.273476    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cff575d5-ef70-4142-8450-b15c8038b6f8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-sd77t\" (UID: \"cff575d5-ef70-4142-8450-b15c8038b6f8\") " pod="openshift-multus/multus-additional-cni-plugins-sd77t"
Jul 18 01:56:48.273588 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.273523    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cff575d5-ef70-4142-8450-b15c8038b6f8-os-release\") pod \"multus-additional-cni-plugins-sd77t\" (UID: \"cff575d5-ef70-4142-8450-b15c8038b6f8\") " pod="openshift-multus/multus-additional-cni-plugins-sd77t"
Jul 18 01:56:48.273588 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.273547    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a370ba82-a51e-41aa-9ced-19bab3186de4-dbus\") pod \"global-pull-secret-syncer-jnffd\" (UID: \"a370ba82-a51e-41aa-9ced-19bab3186de4\") " pod="kube-system/global-pull-secret-syncer-jnffd"
Jul 18 01:56:48.273895 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.273621    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5zc7\" (UniqueName: \"kubernetes.io/projected/48beaadf-7df1-497b-af20-2b57bb5a4250-kube-api-access-p5zc7\") pod \"network-metrics-daemon-qgrtk\" (UID: \"48beaadf-7df1-497b-af20-2b57bb5a4250\") " pod="openshift-multus/network-metrics-daemon-qgrtk"
Jul 18 01:56:48.273895 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.273669    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-host-run-netns\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.273895 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.273717    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-host-var-lib-cni-bin\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.273895 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.273735    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-host-run-multus-certs\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.273895 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.273757    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/537a091c-f327-488e-85e8-600ce7edbfcf-iptables-alerter-script\") pod \"iptables-alerter-s42pb\" (UID: \"537a091c-f327-488e-85e8-600ce7edbfcf\") " pod="openshift-network-operator/iptables-alerter-s42pb"
Jul 18 01:56:48.273895 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.273784    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-host-kubelet\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.273895 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.273807    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-run-ovn\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.273895 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.273835    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-host-run-netns\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.273895 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.273824    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-host-var-lib-cni-bin\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.273895 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.273843    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-host-run-ovn-kubernetes\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.274481 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.273940    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/60cb632f-f03b-491f-8a38-d297305c9992-ovn-node-metrics-cert\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.274481 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.273954    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-host-run-multus-certs\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.274481 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.274000    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlhwq\" (UniqueName: \"kubernetes.io/projected/cff575d5-ef70-4142-8450-b15c8038b6f8-kube-api-access-tlhwq\") pod \"multus-additional-cni-plugins-sd77t\" (UID: \"cff575d5-ef70-4142-8450-b15c8038b6f8\") " pod="openshift-multus/multus-additional-cni-plugins-sd77t"
Jul 18 01:56:48.274481 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.274042    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0c5d4061-2764-4595-9bda-43de3b3cfe15-etc-sysconfig\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.274481 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.274068    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48beaadf-7df1-497b-af20-2b57bb5a4250-metrics-certs\") pod \"network-metrics-daemon-qgrtk\" (UID: \"48beaadf-7df1-497b-af20-2b57bb5a4250\") " pod="openshift-multus/network-metrics-daemon-qgrtk"
Jul 18 01:56:48.274481 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.274146    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-node-log\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.274481 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.274182    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-log-socket\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.274481 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.274217    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.274481 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.274248    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/60cb632f-f03b-491f-8a38-d297305c9992-ovnkube-script-lib\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.274481 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.274276    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k92lz\" (UID: \"d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k92lz"
Jul 18 01:56:48.274481 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.274320    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cff575d5-ef70-4142-8450-b15c8038b6f8-cni-binary-copy\") pod \"multus-additional-cni-plugins-sd77t\" (UID: \"cff575d5-ef70-4142-8450-b15c8038b6f8\") " pod="openshift-multus/multus-additional-cni-plugins-sd77t"
Jul 18 01:56:48.274481 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.274343    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a370ba82-a51e-41aa-9ced-19bab3186de4-original-pull-secret\") pod \"global-pull-secret-syncer-jnffd\" (UID: \"a370ba82-a51e-41aa-9ced-19bab3186de4\") " pod="kube-system/global-pull-secret-syncer-jnffd"
Jul 18 01:56:48.274481 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.274363    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0c5d4061-2764-4595-9bda-43de3b3cfe15-etc-sysctl-d\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.275172 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.274385    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58-device-dir\") pod \"aws-ebs-csi-driver-node-k92lz\" (UID: \"d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k92lz"
Jul 18 01:56:48.275172 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.274423    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/537a091c-f327-488e-85e8-600ce7edbfcf-host-slash\") pod \"iptables-alerter-s42pb\" (UID: \"537a091c-f327-488e-85e8-600ce7edbfcf\") " pod="openshift-network-operator/iptables-alerter-s42pb"
Jul 18 01:56:48.275172 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.274475    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-cnibin\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.275172 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.274501    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a370ba82-a51e-41aa-9ced-19bab3186de4-kubelet-config\") pod \"global-pull-secret-syncer-jnffd\" (UID: \"a370ba82-a51e-41aa-9ced-19bab3186de4\") " pod="kube-system/global-pull-secret-syncer-jnffd"
Jul 18 01:56:48.275172 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.274531    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0c5d4061-2764-4595-9bda-43de3b3cfe15-run\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.275172 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.274551    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0c5d4061-2764-4595-9bda-43de3b3cfe15-host\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.275172 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.274574    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3f34325-5076-4e41-a22d-5caece5dbece-tmp-dir\") pod \"node-resolver-25924\" (UID: \"a3f34325-5076-4e41-a22d-5caece5dbece\") " pod="openshift-dns/node-resolver-25924"
Jul 18 01:56:48.275172 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.274574    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-cnibin\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.275172 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.274600    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-run-systemd\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.275172 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.274675    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtjnd\" (UniqueName: \"kubernetes.io/projected/b212c851-6975-4dcc-bd43-9421f78f6c64-kube-api-access-wtjnd\") pod \"network-check-target-h6pxz\" (UID: \"b212c851-6975-4dcc-bd43-9421f78f6c64\") " pod="openshift-network-diagnostics/network-check-target-h6pxz"
Jul 18 01:56:48.275172 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.274711    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-system-cni-dir\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.275172 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.274762    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-system-cni-dir\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.275172 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.274791    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-host-var-lib-kubelet\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.275172 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.274815    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-hostroot\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.275172 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.274847    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-hostroot\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.275994 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.274850    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-host-var-lib-kubelet\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.275994 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.274876    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-host-cni-netd\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.275994 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.274897    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58-socket-dir\") pod \"aws-ebs-csi-driver-node-k92lz\" (UID: \"d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k92lz"
Jul 18 01:56:48.275994 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.274914    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58-sys-fs\") pod \"aws-ebs-csi-driver-node-k92lz\" (UID: \"d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k92lz"
Jul 18 01:56:48.275994 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.274928    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-run-openvswitch\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.275994 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.274948    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pntpj\" (UniqueName: \"kubernetes.io/projected/60cb632f-f03b-491f-8a38-d297305c9992-kube-api-access-pntpj\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.275994 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275001    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/33351d5c-d8c8-4d94-8209-28d15797b7a9-serviceca\") pod \"node-ca-f5kmg\" (UID: \"33351d5c-d8c8-4d94-8209-28d15797b7a9\") " pod="openshift-image-registry/node-ca-f5kmg"
Jul 18 01:56:48.275994 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275021    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-os-release\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.275994 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275042    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0c5d4061-2764-4595-9bda-43de3b3cfe15-etc-systemd\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.275994 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275064    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a3f34325-5076-4e41-a22d-5caece5dbece-hosts-file\") pod \"node-resolver-25924\" (UID: \"a3f34325-5076-4e41-a22d-5caece5dbece\") " pod="openshift-dns/node-resolver-25924"
Jul 18 01:56:48.275994 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275088    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-os-release\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.275994 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275095    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-var-lib-openvswitch\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.275994 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275126    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/33351d5c-d8c8-4d94-8209-28d15797b7a9-host\") pod \"node-ca-f5kmg\" (UID: \"33351d5c-d8c8-4d94-8209-28d15797b7a9\") " pod="openshift-image-registry/node-ca-f5kmg"
Jul 18 01:56:48.275994 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275155    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58-registration-dir\") pod \"aws-ebs-csi-driver-node-k92lz\" (UID: \"d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k92lz"
Jul 18 01:56:48.276655 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275181    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-etc-kubernetes\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.276655 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275203    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cff575d5-ef70-4142-8450-b15c8038b6f8-cnibin\") pod \"multus-additional-cni-plugins-sd77t\" (UID: \"cff575d5-ef70-4142-8450-b15c8038b6f8\") " pod="openshift-multus/multus-additional-cni-plugins-sd77t"
Jul 18 01:56:48.276655 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275225    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-multus-conf-dir\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.276655 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275246    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0c5d4061-2764-4595-9bda-43de3b3cfe15-etc-tuned\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.276655 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275265    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-systemd-units\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.276655 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275296    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqhkz\" (UniqueName: \"kubernetes.io/projected/0c5d4061-2764-4595-9bda-43de3b3cfe15-kube-api-access-jqhkz\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.276655 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275281    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-etc-kubernetes\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.276655 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275321    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74dzv\" (UniqueName: \"kubernetes.io/projected/537a091c-f327-488e-85e8-600ce7edbfcf-kube-api-access-74dzv\") pod \"iptables-alerter-s42pb\" (UID: \"537a091c-f327-488e-85e8-600ce7edbfcf\") " pod="openshift-network-operator/iptables-alerter-s42pb"
Jul 18 01:56:48.276655 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275316    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-multus-conf-dir\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.276655 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275344    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-host-slash\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.276655 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275367    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whn7v\" (UniqueName: \"kubernetes.io/projected/d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58-kube-api-access-whn7v\") pod \"aws-ebs-csi-driver-node-k92lz\" (UID: \"d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k92lz"
Jul 18 01:56:48.276655 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275391    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-multus-cni-dir\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.276655 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275414    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-multus-socket-dir-parent\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.276655 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275449    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d01349d2-5a70-4df6-b978-a3cf84438584-multus-daemon-config\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.277140 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275470    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/60cb632f-f03b-491f-8a38-d297305c9992-ovnkube-config\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.277140 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275495    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ad82bb0f-ce9c-416d-b049-6e6e8ddfaba8-agent-certs\") pod \"konnectivity-agent-t7kmc\" (UID: \"ad82bb0f-ce9c-416d-b049-6e6e8ddfaba8\") " pod="kube-system/konnectivity-agent-t7kmc"
Jul 18 01:56:48.277140 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275516    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d01349d2-5a70-4df6-b978-a3cf84438584-cni-binary-copy\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.277140 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275539    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-host-run-k8s-cni-cncf-io\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.277140 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275560    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k95l2\" (UniqueName: \"kubernetes.io/projected/d01349d2-5a70-4df6-b978-a3cf84438584-kube-api-access-k95l2\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.277140 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275568    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-multus-socket-dir-parent\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.277140 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275582    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0c5d4061-2764-4595-9bda-43de3b3cfe15-etc-modprobe-d\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.277140 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275586    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-multus-cni-dir\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.277140 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275612    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c5d4061-2764-4595-9bda-43de3b3cfe15-etc-kubernetes\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.277140 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275615    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-host-run-k8s-cni-cncf-io\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.277140 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275633    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-etc-openvswitch\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.277140 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275752    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-host-run-netns\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.277140 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275775    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-host-cni-bin\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.277140 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275805    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-host-var-lib-cni-multus\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.277642 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275833    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cff575d5-ef70-4142-8450-b15c8038b6f8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-sd77t\" (UID: \"cff575d5-ef70-4142-8450-b15c8038b6f8\") " pod="openshift-multus/multus-additional-cni-plugins-sd77t"
Jul 18 01:56:48.277642 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275864    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0c5d4061-2764-4595-9bda-43de3b3cfe15-etc-sysctl-conf\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.277642 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275878    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d01349d2-5a70-4df6-b978-a3cf84438584-host-var-lib-cni-multus\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.277642 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275888    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c5d4061-2764-4595-9bda-43de3b3cfe15-var-lib-kubelet\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.277642 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275924    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0c5d4061-2764-4595-9bda-43de3b3cfe15-tmp\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.277642 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275924    2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 18 01:56:48.277642 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275958    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92pxm\" (UniqueName: \"kubernetes.io/projected/33351d5c-d8c8-4d94-8209-28d15797b7a9-kube-api-access-92pxm\") pod \"node-ca-f5kmg\" (UID: \"33351d5c-d8c8-4d94-8209-28d15797b7a9\") " pod="openshift-image-registry/node-ca-f5kmg"
Jul 18 01:56:48.277642 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.275984    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ad82bb0f-ce9c-416d-b049-6e6e8ddfaba8-konnectivity-ca\") pod \"konnectivity-agent-t7kmc\" (UID: \"ad82bb0f-ce9c-416d-b049-6e6e8ddfaba8\") " pod="kube-system/konnectivity-agent-t7kmc"
Jul 18 01:56:48.277642 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.276007    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cff575d5-ef70-4142-8450-b15c8038b6f8-system-cni-dir\") pod \"multus-additional-cni-plugins-sd77t\" (UID: \"cff575d5-ef70-4142-8450-b15c8038b6f8\") " pod="openshift-multus/multus-additional-cni-plugins-sd77t"
Jul 18 01:56:48.277642 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.276029    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0c5d4061-2764-4595-9bda-43de3b3cfe15-sys\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.277642 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.276048    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0c5d4061-2764-4595-9bda-43de3b3cfe15-lib-modules\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.277642 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.276058    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d01349d2-5a70-4df6-b978-a3cf84438584-multus-daemon-config\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.277642 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.276069    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4frn\" (UniqueName: \"kubernetes.io/projected/a3f34325-5076-4e41-a22d-5caece5dbece-kube-api-access-m4frn\") pod \"node-resolver-25924\" (UID: \"a3f34325-5076-4e41-a22d-5caece5dbece\") " pod="openshift-dns/node-resolver-25924"
Jul 18 01:56:48.277642 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.276091    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/60cb632f-f03b-491f-8a38-d297305c9992-env-overrides\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.278115 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.276104    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d01349d2-5a70-4df6-b978-a3cf84438584-cni-binary-copy\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.278115 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.276116    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58-etc-selinux\") pod \"aws-ebs-csi-driver-node-k92lz\" (UID: \"d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k92lz"
Jul 18 01:56:48.278115 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.276140    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cff575d5-ef70-4142-8450-b15c8038b6f8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-sd77t\" (UID: \"cff575d5-ef70-4142-8450-b15c8038b6f8\") " pod="openshift-multus/multus-additional-cni-plugins-sd77t"
Jul 18 01:56:48.278115 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.276504    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ad82bb0f-ce9c-416d-b049-6e6e8ddfaba8-konnectivity-ca\") pod \"konnectivity-agent-t7kmc\" (UID: \"ad82bb0f-ce9c-416d-b049-6e6e8ddfaba8\") " pod="kube-system/konnectivity-agent-t7kmc"
Jul 18 01:56:48.279691 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.279673    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ad82bb0f-ce9c-416d-b049-6e6e8ddfaba8-agent-certs\") pod \"konnectivity-agent-t7kmc\" (UID: \"ad82bb0f-ce9c-416d-b049-6e6e8ddfaba8\") " pod="kube-system/konnectivity-agent-t7kmc"
Jul 18 01:56:48.283347 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.283323    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k95l2\" (UniqueName: \"kubernetes.io/projected/d01349d2-5a70-4df6-b978-a3cf84438584-kube-api-access-k95l2\") pod \"multus-grrdq\" (UID: \"d01349d2-5a70-4df6-b978-a3cf84438584\") " pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.295537 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.295508    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 01:56:48.376876 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.376842    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3f34325-5076-4e41-a22d-5caece5dbece-tmp-dir\") pod \"node-resolver-25924\" (UID: \"a3f34325-5076-4e41-a22d-5caece5dbece\") " pod="openshift-dns/node-resolver-25924"
Jul 18 01:56:48.376876 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.376876    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-run-systemd\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.377157 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.376899    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtjnd\" (UniqueName: \"kubernetes.io/projected/b212c851-6975-4dcc-bd43-9421f78f6c64-kube-api-access-wtjnd\") pod \"network-check-target-h6pxz\" (UID: \"b212c851-6975-4dcc-bd43-9421f78f6c64\") " pod="openshift-network-diagnostics/network-check-target-h6pxz"
Jul 18 01:56:48.377157 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.376924    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-host-cni-netd\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.377157 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.376944    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58-socket-dir\") pod \"aws-ebs-csi-driver-node-k92lz\" (UID: \"d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k92lz"
Jul 18 01:56:48.377157 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.376968    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58-sys-fs\") pod \"aws-ebs-csi-driver-node-k92lz\" (UID: \"d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k92lz"
Jul 18 01:56:48.377157 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.376972    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-run-systemd\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.377157 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.376988    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-run-openvswitch\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.377157 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377018    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pntpj\" (UniqueName: \"kubernetes.io/projected/60cb632f-f03b-491f-8a38-d297305c9992-kube-api-access-pntpj\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.377157 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377046    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/33351d5c-d8c8-4d94-8209-28d15797b7a9-serviceca\") pod \"node-ca-f5kmg\" (UID: \"33351d5c-d8c8-4d94-8209-28d15797b7a9\") " pod="openshift-image-registry/node-ca-f5kmg"
Jul 18 01:56:48.377157 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377054    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58-sys-fs\") pod \"aws-ebs-csi-driver-node-k92lz\" (UID: \"d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k92lz"
Jul 18 01:56:48.377157 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377063    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-host-cni-netd\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.377157 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377070    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0c5d4061-2764-4595-9bda-43de3b3cfe15-etc-systemd\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.377157 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377101    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-run-openvswitch\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.377157 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377117    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58-socket-dir\") pod \"aws-ebs-csi-driver-node-k92lz\" (UID: \"d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k92lz"
Jul 18 01:56:48.377645 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377174    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3f34325-5076-4e41-a22d-5caece5dbece-tmp-dir\") pod \"node-resolver-25924\" (UID: \"a3f34325-5076-4e41-a22d-5caece5dbece\") " pod="openshift-dns/node-resolver-25924"
Jul 18 01:56:48.377645 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377221    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a3f34325-5076-4e41-a22d-5caece5dbece-hosts-file\") pod \"node-resolver-25924\" (UID: \"a3f34325-5076-4e41-a22d-5caece5dbece\") " pod="openshift-dns/node-resolver-25924"
Jul 18 01:56:48.377645 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377258    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0c5d4061-2764-4595-9bda-43de3b3cfe15-etc-systemd\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.377645 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377258    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-var-lib-openvswitch\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.377645 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377316    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-var-lib-openvswitch\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.377645 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377320    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a3f34325-5076-4e41-a22d-5caece5dbece-hosts-file\") pod \"node-resolver-25924\" (UID: \"a3f34325-5076-4e41-a22d-5caece5dbece\") " pod="openshift-dns/node-resolver-25924"
Jul 18 01:56:48.377645 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377338    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/33351d5c-d8c8-4d94-8209-28d15797b7a9-host\") pod \"node-ca-f5kmg\" (UID: \"33351d5c-d8c8-4d94-8209-28d15797b7a9\") " pod="openshift-image-registry/node-ca-f5kmg"
Jul 18 01:56:48.377645 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377365    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58-registration-dir\") pod \"aws-ebs-csi-driver-node-k92lz\" (UID: \"d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k92lz"
Jul 18 01:56:48.377645 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377393    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cff575d5-ef70-4142-8450-b15c8038b6f8-cnibin\") pod \"multus-additional-cni-plugins-sd77t\" (UID: \"cff575d5-ef70-4142-8450-b15c8038b6f8\") " pod="openshift-multus/multus-additional-cni-plugins-sd77t"
Jul 18 01:56:48.377645 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377411    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0c5d4061-2764-4595-9bda-43de3b3cfe15-etc-tuned\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.377645 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377440    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/33351d5c-d8c8-4d94-8209-28d15797b7a9-host\") pod \"node-ca-f5kmg\" (UID: \"33351d5c-d8c8-4d94-8209-28d15797b7a9\") " pod="openshift-image-registry/node-ca-f5kmg"
Jul 18 01:56:48.377645 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377442    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-systemd-units\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.377645 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377457    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/33351d5c-d8c8-4d94-8209-28d15797b7a9-serviceca\") pod \"node-ca-f5kmg\" (UID: \"33351d5c-d8c8-4d94-8209-28d15797b7a9\") " pod="openshift-image-registry/node-ca-f5kmg"
Jul 18 01:56:48.377645 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377468    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cff575d5-ef70-4142-8450-b15c8038b6f8-cnibin\") pod \"multus-additional-cni-plugins-sd77t\" (UID: \"cff575d5-ef70-4142-8450-b15c8038b6f8\") " pod="openshift-multus/multus-additional-cni-plugins-sd77t"
Jul 18 01:56:48.377645 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377458    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58-registration-dir\") pod \"aws-ebs-csi-driver-node-k92lz\" (UID: \"d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k92lz"
Jul 18 01:56:48.378261 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377480    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-systemd-units\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.378261 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377480    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqhkz\" (UniqueName: \"kubernetes.io/projected/0c5d4061-2764-4595-9bda-43de3b3cfe15-kube-api-access-jqhkz\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.378261 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377564    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74dzv\" (UniqueName: \"kubernetes.io/projected/537a091c-f327-488e-85e8-600ce7edbfcf-kube-api-access-74dzv\") pod \"iptables-alerter-s42pb\" (UID: \"537a091c-f327-488e-85e8-600ce7edbfcf\") " pod="openshift-network-operator/iptables-alerter-s42pb"
Jul 18 01:56:48.378261 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377586    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-host-slash\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.378261 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377603    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whn7v\" (UniqueName: \"kubernetes.io/projected/d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58-kube-api-access-whn7v\") pod \"aws-ebs-csi-driver-node-k92lz\" (UID: \"d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k92lz"
Jul 18 01:56:48.378261 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377631    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/60cb632f-f03b-491f-8a38-d297305c9992-ovnkube-config\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.378261 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377662    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-host-slash\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.378261 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377749    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0c5d4061-2764-4595-9bda-43de3b3cfe15-etc-modprobe-d\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.378261 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377782    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c5d4061-2764-4595-9bda-43de3b3cfe15-etc-kubernetes\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.378261 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377809    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-etc-openvswitch\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.378261 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377835    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-host-run-netns\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.378261 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377859    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-host-cni-bin\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.378261 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377867    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c5d4061-2764-4595-9bda-43de3b3cfe15-etc-kubernetes\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.378261 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377885    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cff575d5-ef70-4142-8450-b15c8038b6f8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-sd77t\" (UID: \"cff575d5-ef70-4142-8450-b15c8038b6f8\") " pod="openshift-multus/multus-additional-cni-plugins-sd77t"
Jul 18 01:56:48.378907 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377890    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0c5d4061-2764-4595-9bda-43de3b3cfe15-etc-modprobe-d\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.378907 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377895    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-etc-openvswitch\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.378907 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377913    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0c5d4061-2764-4595-9bda-43de3b3cfe15-etc-sysctl-conf\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.378907 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377918    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-host-run-netns\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.378907 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377936    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-host-cni-bin\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.378907 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377974    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c5d4061-2764-4595-9bda-43de3b3cfe15-var-lib-kubelet\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.378907 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.377991    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0c5d4061-2764-4595-9bda-43de3b3cfe15-tmp\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.378907 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.378011    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92pxm\" (UniqueName: \"kubernetes.io/projected/33351d5c-d8c8-4d94-8209-28d15797b7a9-kube-api-access-92pxm\") pod \"node-ca-f5kmg\" (UID: \"33351d5c-d8c8-4d94-8209-28d15797b7a9\") " pod="openshift-image-registry/node-ca-f5kmg"
Jul 18 01:56:48.378907 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.378039    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cff575d5-ef70-4142-8450-b15c8038b6f8-system-cni-dir\") pod \"multus-additional-cni-plugins-sd77t\" (UID: \"cff575d5-ef70-4142-8450-b15c8038b6f8\") " pod="openshift-multus/multus-additional-cni-plugins-sd77t"
Jul 18 01:56:48.378907 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.378062    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0c5d4061-2764-4595-9bda-43de3b3cfe15-sys\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.378907 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.378113    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c5d4061-2764-4595-9bda-43de3b3cfe15-var-lib-kubelet\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.378907 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.378116    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0c5d4061-2764-4595-9bda-43de3b3cfe15-sys\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.378907 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.378130    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0c5d4061-2764-4595-9bda-43de3b3cfe15-etc-sysctl-conf\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.378907 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.378133    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0c5d4061-2764-4595-9bda-43de3b3cfe15-lib-modules\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.378907 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.378157    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cff575d5-ef70-4142-8450-b15c8038b6f8-system-cni-dir\") pod \"multus-additional-cni-plugins-sd77t\" (UID: \"cff575d5-ef70-4142-8450-b15c8038b6f8\") " pod="openshift-multus/multus-additional-cni-plugins-sd77t"
Jul 18 01:56:48.379940 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.378175    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/60cb632f-f03b-491f-8a38-d297305c9992-ovnkube-config\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.379940 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.378182    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4frn\" (UniqueName: \"kubernetes.io/projected/a3f34325-5076-4e41-a22d-5caece5dbece-kube-api-access-m4frn\") pod \"node-resolver-25924\" (UID: \"a3f34325-5076-4e41-a22d-5caece5dbece\") " pod="openshift-dns/node-resolver-25924"
Jul 18 01:56:48.379940 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.378223    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/60cb632f-f03b-491f-8a38-d297305c9992-env-overrides\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.379940 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.378246    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0c5d4061-2764-4595-9bda-43de3b3cfe15-lib-modules\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.379940 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.378398    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58-etc-selinux\") pod \"aws-ebs-csi-driver-node-k92lz\" (UID: \"d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k92lz"
Jul 18 01:56:48.379940 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.378422    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cff575d5-ef70-4142-8450-b15c8038b6f8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-sd77t\" (UID: \"cff575d5-ef70-4142-8450-b15c8038b6f8\") " pod="openshift-multus/multus-additional-cni-plugins-sd77t"
Jul 18 01:56:48.379940 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.378466    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cff575d5-ef70-4142-8450-b15c8038b6f8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-sd77t\" (UID: \"cff575d5-ef70-4142-8450-b15c8038b6f8\") " pod="openshift-multus/multus-additional-cni-plugins-sd77t"
Jul 18 01:56:48.379940 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.378485    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cff575d5-ef70-4142-8450-b15c8038b6f8-os-release\") pod \"multus-additional-cni-plugins-sd77t\" (UID: \"cff575d5-ef70-4142-8450-b15c8038b6f8\") " pod="openshift-multus/multus-additional-cni-plugins-sd77t"
Jul 18 01:56:48.379940 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.378507    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a370ba82-a51e-41aa-9ced-19bab3186de4-dbus\") pod \"global-pull-secret-syncer-jnffd\" (UID: \"a370ba82-a51e-41aa-9ced-19bab3186de4\") " pod="kube-system/global-pull-secret-syncer-jnffd"
Jul 18 01:56:48.379940 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.378532    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5zc7\" (UniqueName: \"kubernetes.io/projected/48beaadf-7df1-497b-af20-2b57bb5a4250-kube-api-access-p5zc7\") pod \"network-metrics-daemon-qgrtk\" (UID: \"48beaadf-7df1-497b-af20-2b57bb5a4250\") " pod="openshift-multus/network-metrics-daemon-qgrtk"
Jul 18 01:56:48.379940 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.378536    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58-etc-selinux\") pod \"aws-ebs-csi-driver-node-k92lz\" (UID: \"d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k92lz"
Jul 18 01:56:48.379940 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.378562    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/537a091c-f327-488e-85e8-600ce7edbfcf-iptables-alerter-script\") pod \"iptables-alerter-s42pb\" (UID: \"537a091c-f327-488e-85e8-600ce7edbfcf\") " pod="openshift-network-operator/iptables-alerter-s42pb"
Jul 18 01:56:48.379940 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.378640    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cff575d5-ef70-4142-8450-b15c8038b6f8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-sd77t\" (UID: \"cff575d5-ef70-4142-8450-b15c8038b6f8\") " pod="openshift-multus/multus-additional-cni-plugins-sd77t"
Jul 18 01:56:48.379940 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.378700    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a370ba82-a51e-41aa-9ced-19bab3186de4-dbus\") pod \"global-pull-secret-syncer-jnffd\" (UID: \"a370ba82-a51e-41aa-9ced-19bab3186de4\") " pod="kube-system/global-pull-secret-syncer-jnffd"
Jul 18 01:56:48.380633 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.378745    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/60cb632f-f03b-491f-8a38-d297305c9992-env-overrides\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.380633 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.378748    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cff575d5-ef70-4142-8450-b15c8038b6f8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-sd77t\" (UID: \"cff575d5-ef70-4142-8450-b15c8038b6f8\") " pod="openshift-multus/multus-additional-cni-plugins-sd77t"
Jul 18 01:56:48.380633 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.378753    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cff575d5-ef70-4142-8450-b15c8038b6f8-os-release\") pod \"multus-additional-cni-plugins-sd77t\" (UID: \"cff575d5-ef70-4142-8450-b15c8038b6f8\") " pod="openshift-multus/multus-additional-cni-plugins-sd77t"
Jul 18 01:56:48.380633 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.378791    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-host-kubelet\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.380633 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.378820    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-run-ovn\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.380633 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.378826    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-host-kubelet\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.380633 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.378848    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-host-run-ovn-kubernetes\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.380633 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.378874    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-run-ovn\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.380633 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.378875    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/60cb632f-f03b-491f-8a38-d297305c9992-ovn-node-metrics-cert\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.380633 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.378916    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlhwq\" (UniqueName: \"kubernetes.io/projected/cff575d5-ef70-4142-8450-b15c8038b6f8-kube-api-access-tlhwq\") pod \"multus-additional-cni-plugins-sd77t\" (UID: \"cff575d5-ef70-4142-8450-b15c8038b6f8\") " pod="openshift-multus/multus-additional-cni-plugins-sd77t"
Jul 18 01:56:48.380633 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.378941    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0c5d4061-2764-4595-9bda-43de3b3cfe15-etc-sysconfig\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.380633 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.378967    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48beaadf-7df1-497b-af20-2b57bb5a4250-metrics-certs\") pod \"network-metrics-daemon-qgrtk\" (UID: \"48beaadf-7df1-497b-af20-2b57bb5a4250\") " pod="openshift-multus/network-metrics-daemon-qgrtk"
Jul 18 01:56:48.380633 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.378972    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-host-run-ovn-kubernetes\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.380633 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.378994    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-node-log\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.381161 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.379026    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-node-log\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.381161 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.379029    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-log-socket\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.381161 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.379066    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cff575d5-ef70-4142-8450-b15c8038b6f8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-sd77t\" (UID: \"cff575d5-ef70-4142-8450-b15c8038b6f8\") " pod="openshift-multus/multus-additional-cni-plugins-sd77t"
Jul 18 01:56:48.381161 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.379069    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.381161 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.379102    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.381161 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.379107    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/537a091c-f327-488e-85e8-600ce7edbfcf-iptables-alerter-script\") pod \"iptables-alerter-s42pb\" (UID: \"537a091c-f327-488e-85e8-600ce7edbfcf\") " pod="openshift-network-operator/iptables-alerter-s42pb"
Jul 18 01:56:48.381161 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.379121    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/60cb632f-f03b-491f-8a38-d297305c9992-ovnkube-script-lib\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.381161 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.379164    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0c5d4061-2764-4595-9bda-43de3b3cfe15-etc-sysconfig\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.381161 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.379145    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/60cb632f-f03b-491f-8a38-d297305c9992-log-socket\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.381161 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:48.379223    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:56:48.381161 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:48.379301    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48beaadf-7df1-497b-af20-2b57bb5a4250-metrics-certs podName:48beaadf-7df1-497b-af20-2b57bb5a4250 nodeName:}" failed. No retries permitted until 2026-07-18 01:56:48.87926395 +0000 UTC m=+3.504300610 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/48beaadf-7df1-497b-af20-2b57bb5a4250-metrics-certs") pod "network-metrics-daemon-qgrtk" (UID: "48beaadf-7df1-497b-af20-2b57bb5a4250") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:56:48.381161 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.379320    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k92lz\" (UID: \"d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k92lz"
Jul 18 01:56:48.381161 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.379347    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cff575d5-ef70-4142-8450-b15c8038b6f8-cni-binary-copy\") pod \"multus-additional-cni-plugins-sd77t\" (UID: \"cff575d5-ef70-4142-8450-b15c8038b6f8\") " pod="openshift-multus/multus-additional-cni-plugins-sd77t"
Jul 18 01:56:48.381161 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.379374    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a370ba82-a51e-41aa-9ced-19bab3186de4-original-pull-secret\") pod \"global-pull-secret-syncer-jnffd\" (UID: \"a370ba82-a51e-41aa-9ced-19bab3186de4\") " pod="kube-system/global-pull-secret-syncer-jnffd"
Jul 18 01:56:48.381557 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.379400    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0c5d4061-2764-4595-9bda-43de3b3cfe15-etc-sysctl-d\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.381557 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.379422    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58-device-dir\") pod \"aws-ebs-csi-driver-node-k92lz\" (UID: \"d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k92lz"
Jul 18 01:56:48.381557 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.379471    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/537a091c-f327-488e-85e8-600ce7edbfcf-host-slash\") pod \"iptables-alerter-s42pb\" (UID: \"537a091c-f327-488e-85e8-600ce7edbfcf\") " pod="openshift-network-operator/iptables-alerter-s42pb"
Jul 18 01:56:48.381557 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.379501    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a370ba82-a51e-41aa-9ced-19bab3186de4-kubelet-config\") pod \"global-pull-secret-syncer-jnffd\" (UID: \"a370ba82-a51e-41aa-9ced-19bab3186de4\") " pod="kube-system/global-pull-secret-syncer-jnffd"
Jul 18 01:56:48.381557 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.379524    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0c5d4061-2764-4595-9bda-43de3b3cfe15-run\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.381557 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:48.379529    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 01:56:48.381557 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.379545    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0c5d4061-2764-4595-9bda-43de3b3cfe15-host\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.381557 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.379582    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k92lz\" (UID: \"d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k92lz"
Jul 18 01:56:48.381557 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:48.379607    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a370ba82-a51e-41aa-9ced-19bab3186de4-original-pull-secret podName:a370ba82-a51e-41aa-9ced-19bab3186de4 nodeName:}" failed. No retries permitted until 2026-07-18 01:56:48.879567754 +0000 UTC m=+3.504604411 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a370ba82-a51e-41aa-9ced-19bab3186de4-original-pull-secret") pod "global-pull-secret-syncer-jnffd" (UID: "a370ba82-a51e-41aa-9ced-19bab3186de4") : object "kube-system"/"original-pull-secret" not registered
Jul 18 01:56:48.381557 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.379614    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/60cb632f-f03b-491f-8a38-d297305c9992-ovnkube-script-lib\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.381557 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.379621    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0c5d4061-2764-4595-9bda-43de3b3cfe15-host\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.381557 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.379666    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0c5d4061-2764-4595-9bda-43de3b3cfe15-run\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.381557 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.379669    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/537a091c-f327-488e-85e8-600ce7edbfcf-host-slash\") pod \"iptables-alerter-s42pb\" (UID: \"537a091c-f327-488e-85e8-600ce7edbfcf\") " pod="openshift-network-operator/iptables-alerter-s42pb"
Jul 18 01:56:48.381557 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.379672    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58-device-dir\") pod \"aws-ebs-csi-driver-node-k92lz\" (UID: \"d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k92lz"
Jul 18 01:56:48.381947 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.379709    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a370ba82-a51e-41aa-9ced-19bab3186de4-kubelet-config\") pod \"global-pull-secret-syncer-jnffd\" (UID: \"a370ba82-a51e-41aa-9ced-19bab3186de4\") " pod="kube-system/global-pull-secret-syncer-jnffd"
Jul 18 01:56:48.381947 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.379760    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0c5d4061-2764-4595-9bda-43de3b3cfe15-etc-sysctl-d\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.381947 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.380029    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cff575d5-ef70-4142-8450-b15c8038b6f8-cni-binary-copy\") pod \"multus-additional-cni-plugins-sd77t\" (UID: \"cff575d5-ef70-4142-8450-b15c8038b6f8\") " pod="openshift-multus/multus-additional-cni-plugins-sd77t"
Jul 18 01:56:48.381947 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.380069    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0c5d4061-2764-4595-9bda-43de3b3cfe15-etc-tuned\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.381947 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.380259    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0c5d4061-2764-4595-9bda-43de3b3cfe15-tmp\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.381947 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.381158    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/60cb632f-f03b-491f-8a38-d297305c9992-ovn-node-metrics-cert\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.384533 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:48.384513    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 01:56:48.384533 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:48.384536    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 01:56:48.384648 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:48.384546    2576 projected.go:196] Error preparing data for projected volume kube-api-access-wtjnd for pod openshift-network-diagnostics/network-check-target-h6pxz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:56:48.384648 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:48.384603    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b212c851-6975-4dcc-bd43-9421f78f6c64-kube-api-access-wtjnd podName:b212c851-6975-4dcc-bd43-9421f78f6c64 nodeName:}" failed. No retries permitted until 2026-07-18 01:56:48.884591475 +0000 UTC m=+3.509628124 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wtjnd" (UniqueName: "kubernetes.io/projected/b212c851-6975-4dcc-bd43-9421f78f6c64-kube-api-access-wtjnd") pod "network-check-target-h6pxz" (UID: "b212c851-6975-4dcc-bd43-9421f78f6c64") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:56:48.387539 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.387514    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74dzv\" (UniqueName: \"kubernetes.io/projected/537a091c-f327-488e-85e8-600ce7edbfcf-kube-api-access-74dzv\") pod \"iptables-alerter-s42pb\" (UID: \"537a091c-f327-488e-85e8-600ce7edbfcf\") " pod="openshift-network-operator/iptables-alerter-s42pb"
Jul 18 01:56:48.387696 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.387661    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92pxm\" (UniqueName: \"kubernetes.io/projected/33351d5c-d8c8-4d94-8209-28d15797b7a9-kube-api-access-92pxm\") pod \"node-ca-f5kmg\" (UID: \"33351d5c-d8c8-4d94-8209-28d15797b7a9\") " pod="openshift-image-registry/node-ca-f5kmg"
Jul 18 01:56:48.388202 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.388181    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4frn\" (UniqueName: \"kubernetes.io/projected/a3f34325-5076-4e41-a22d-5caece5dbece-kube-api-access-m4frn\") pod \"node-resolver-25924\" (UID: \"a3f34325-5076-4e41-a22d-5caece5dbece\") " pod="openshift-dns/node-resolver-25924"
Jul 18 01:56:48.389053 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.389019    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pntpj\" (UniqueName: \"kubernetes.io/projected/60cb632f-f03b-491f-8a38-d297305c9992-kube-api-access-pntpj\") pod \"ovnkube-node-jfzth\" (UID: \"60cb632f-f03b-491f-8a38-d297305c9992\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.389390 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.389371    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whn7v\" (UniqueName: \"kubernetes.io/projected/d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58-kube-api-access-whn7v\") pod \"aws-ebs-csi-driver-node-k92lz\" (UID: \"d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k92lz"
Jul 18 01:56:48.389484 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.389389    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlhwq\" (UniqueName: \"kubernetes.io/projected/cff575d5-ef70-4142-8450-b15c8038b6f8-kube-api-access-tlhwq\") pod \"multus-additional-cni-plugins-sd77t\" (UID: \"cff575d5-ef70-4142-8450-b15c8038b6f8\") " pod="openshift-multus/multus-additional-cni-plugins-sd77t"
Jul 18 01:56:48.389599 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.389583    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5zc7\" (UniqueName: \"kubernetes.io/projected/48beaadf-7df1-497b-af20-2b57bb5a4250-kube-api-access-p5zc7\") pod \"network-metrics-daemon-qgrtk\" (UID: \"48beaadf-7df1-497b-af20-2b57bb5a4250\") " pod="openshift-multus/network-metrics-daemon-qgrtk"
Jul 18 01:56:48.389991 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.389976    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqhkz\" (UniqueName: \"kubernetes.io/projected/0c5d4061-2764-4595-9bda-43de3b3cfe15-kube-api-access-jqhkz\") pod \"tuned-xlrxq\" (UID: \"0c5d4061-2764-4595-9bda-43de3b3cfe15\") " pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.395481 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.395463    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-grrdq"
Jul 18 01:56:48.401949 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.401928    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-t7kmc"
Jul 18 01:56:48.405940 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:48.405902    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd01349d2_5a70_4df6_b978_a3cf84438584.slice/crio-deab896a77f0d585dc294d6c79b77c69b5cd64eec9a59ad7c66caa47ff3612f5 WatchSource:0}: Error finding container deab896a77f0d585dc294d6c79b77c69b5cd64eec9a59ad7c66caa47ff3612f5: Status 404 returned error can't find the container with id deab896a77f0d585dc294d6c79b77c69b5cd64eec9a59ad7c66caa47ff3612f5
Jul 18 01:56:48.409503 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.409481    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-sd77t"
Jul 18 01:56:48.411336 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:48.411311    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad82bb0f_ce9c_416d_b049_6e6e8ddfaba8.slice/crio-61fb24f33f7125982dddfc1a4aa7243b6ae7917d89e42180407cc1418971946e WatchSource:0}: Error finding container 61fb24f33f7125982dddfc1a4aa7243b6ae7917d89e42180407cc1418971946e: Status 404 returned error can't find the container with id 61fb24f33f7125982dddfc1a4aa7243b6ae7917d89e42180407cc1418971946e
Jul 18 01:56:48.419593 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:48.419568    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcff575d5_ef70_4142_8450_b15c8038b6f8.slice/crio-7ca29f008260b29b4ad5c921c951a0108237403969f6694c78aa0310499bee91 WatchSource:0}: Error finding container 7ca29f008260b29b4ad5c921c951a0108237403969f6694c78aa0310499bee91: Status 404 returned error can't find the container with id 7ca29f008260b29b4ad5c921c951a0108237403969f6694c78aa0310499bee91
Jul 18 01:56:48.605081 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.604990    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-s42pb"
Jul 18 01:56:48.613256 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.613167    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:56:48.619311 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.619165    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k92lz"
Jul 18 01:56:48.625738 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.625536    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xlrxq"
Jul 18 01:56:48.630886 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.630711    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-25924"
Jul 18 01:56:48.634095 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.634043    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-f5kmg"
Jul 18 01:56:48.656581 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:56:48.656550    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3f34325_5076_4e41_a22d_5caece5dbece.slice/crio-db9ce62f1c65ac89d08e7951af409f5ef06cc9de20a0656baca6d76fae20238a WatchSource:0}: Error finding container db9ce62f1c65ac89d08e7951af409f5ef06cc9de20a0656baca6d76fae20238a: Status 404 returned error can't find the container with id db9ce62f1c65ac89d08e7951af409f5ef06cc9de20a0656baca6d76fae20238a
Jul 18 01:56:48.884266 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.884182    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48beaadf-7df1-497b-af20-2b57bb5a4250-metrics-certs\") pod \"network-metrics-daemon-qgrtk\" (UID: \"48beaadf-7df1-497b-af20-2b57bb5a4250\") " pod="openshift-multus/network-metrics-daemon-qgrtk"
Jul 18 01:56:48.884266 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.884231    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a370ba82-a51e-41aa-9ced-19bab3186de4-original-pull-secret\") pod \"global-pull-secret-syncer-jnffd\" (UID: \"a370ba82-a51e-41aa-9ced-19bab3186de4\") " pod="kube-system/global-pull-secret-syncer-jnffd"
Jul 18 01:56:48.884525 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:48.884319    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:56:48.884525 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:48.884365    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 01:56:48.884525 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:48.884389    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48beaadf-7df1-497b-af20-2b57bb5a4250-metrics-certs podName:48beaadf-7df1-497b-af20-2b57bb5a4250 nodeName:}" failed. No retries permitted until 2026-07-18 01:56:49.884370034 +0000 UTC m=+4.509406697 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/48beaadf-7df1-497b-af20-2b57bb5a4250-metrics-certs") pod "network-metrics-daemon-qgrtk" (UID: "48beaadf-7df1-497b-af20-2b57bb5a4250") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:56:48.884525 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:48.884413    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a370ba82-a51e-41aa-9ced-19bab3186de4-original-pull-secret podName:a370ba82-a51e-41aa-9ced-19bab3186de4 nodeName:}" failed. No retries permitted until 2026-07-18 01:56:49.88439874 +0000 UTC m=+4.509435387 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a370ba82-a51e-41aa-9ced-19bab3186de4-original-pull-secret") pod "global-pull-secret-syncer-jnffd" (UID: "a370ba82-a51e-41aa-9ced-19bab3186de4") : object "kube-system"/"original-pull-secret" not registered
Jul 18 01:56:48.985788 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:48.985071    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtjnd\" (UniqueName: \"kubernetes.io/projected/b212c851-6975-4dcc-bd43-9421f78f6c64-kube-api-access-wtjnd\") pod \"network-check-target-h6pxz\" (UID: \"b212c851-6975-4dcc-bd43-9421f78f6c64\") " pod="openshift-network-diagnostics/network-check-target-h6pxz"
Jul 18 01:56:48.985788 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:48.985337    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 01:56:48.985788 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:48.985360    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 01:56:48.985788 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:48.985373    2576 projected.go:196] Error preparing data for projected volume kube-api-access-wtjnd for pod openshift-network-diagnostics/network-check-target-h6pxz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:56:48.985788 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:48.985450    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b212c851-6975-4dcc-bd43-9421f78f6c64-kube-api-access-wtjnd podName:b212c851-6975-4dcc-bd43-9421f78f6c64 nodeName:}" failed. No retries permitted until 2026-07-18 01:56:49.985414787 +0000 UTC m=+4.610451437 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wtjnd" (UniqueName: "kubernetes.io/projected/b212c851-6975-4dcc-bd43-9421f78f6c64-kube-api-access-wtjnd") pod "network-check-target-h6pxz" (UID: "b212c851-6975-4dcc-bd43-9421f78f6c64") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:56:49.079782 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:49.079740    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-25924" event={"ID":"a3f34325-5076-4e41-a22d-5caece5dbece","Type":"ContainerStarted","Data":"db9ce62f1c65ac89d08e7951af409f5ef06cc9de20a0656baca6d76fae20238a"}
Jul 18 01:56:49.081781 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:49.081600    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xlrxq" event={"ID":"0c5d4061-2764-4595-9bda-43de3b3cfe15","Type":"ContainerStarted","Data":"4bf0b42c43b4beaa98d35148973e2ad0dd6fccc7f2a36f874eeccf1130c47070"}
Jul 18 01:56:49.084331 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:49.084267    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k92lz" event={"ID":"d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58","Type":"ContainerStarted","Data":"b82fc675e7806af11d53d1641f39a64dcbbed38708b02c55a49cb029c8fb4bad"}
Jul 18 01:56:49.088496 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:49.088467    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jfzth" event={"ID":"60cb632f-f03b-491f-8a38-d297305c9992","Type":"ContainerStarted","Data":"a02b10f35a7ad7d17d3fff6b21f30d85b8040d31dfbd3c481a1df8ce1f2675e0"}
Jul 18 01:56:49.090167 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:49.090142    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-s42pb" event={"ID":"537a091c-f327-488e-85e8-600ce7edbfcf","Type":"ContainerStarted","Data":"105f53442a3d3aaf0e1a49e2742c51159575ef36267090ac941254b7be3f02d0"}
Jul 18 01:56:49.093251 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:49.093203    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sd77t" event={"ID":"cff575d5-ef70-4142-8450-b15c8038b6f8","Type":"ContainerStarted","Data":"7ca29f008260b29b4ad5c921c951a0108237403969f6694c78aa0310499bee91"}
Jul 18 01:56:49.094646 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:49.094622    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-f5kmg" event={"ID":"33351d5c-d8c8-4d94-8209-28d15797b7a9","Type":"ContainerStarted","Data":"7768de2eabe4f50b7b63dc0bca849b85795436f4464cf8d75988aa7f74ab40ab"}
Jul 18 01:56:49.096391 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:49.096368    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-t7kmc" event={"ID":"ad82bb0f-ce9c-416d-b049-6e6e8ddfaba8","Type":"ContainerStarted","Data":"61fb24f33f7125982dddfc1a4aa7243b6ae7917d89e42180407cc1418971946e"}
Jul 18 01:56:49.101322 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:49.101265    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-grrdq" event={"ID":"d01349d2-5a70-4df6-b978-a3cf84438584","Type":"ContainerStarted","Data":"deab896a77f0d585dc294d6c79b77c69b5cd64eec9a59ad7c66caa47ff3612f5"}
Jul 18 01:56:49.116813 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:49.116660    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-17 01:51:48 +0000 UTC" deadline="2028-02-14 00:58:55.436505424 +0000 UTC"
Jul 18 01:56:49.116813 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:49.116719    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13823h2m6.319790811s"
Jul 18 01:56:49.893739 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:49.893703    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a370ba82-a51e-41aa-9ced-19bab3186de4-original-pull-secret\") pod \"global-pull-secret-syncer-jnffd\" (UID: \"a370ba82-a51e-41aa-9ced-19bab3186de4\") " pod="kube-system/global-pull-secret-syncer-jnffd"
Jul 18 01:56:49.894729 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:49.893806    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48beaadf-7df1-497b-af20-2b57bb5a4250-metrics-certs\") pod \"network-metrics-daemon-qgrtk\" (UID: \"48beaadf-7df1-497b-af20-2b57bb5a4250\") " pod="openshift-multus/network-metrics-daemon-qgrtk"
Jul 18 01:56:49.894729 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:49.893936    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:56:49.894729 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:49.894000    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48beaadf-7df1-497b-af20-2b57bb5a4250-metrics-certs podName:48beaadf-7df1-497b-af20-2b57bb5a4250 nodeName:}" failed. No retries permitted until 2026-07-18 01:56:51.893981175 +0000 UTC m=+6.519017820 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/48beaadf-7df1-497b-af20-2b57bb5a4250-metrics-certs") pod "network-metrics-daemon-qgrtk" (UID: "48beaadf-7df1-497b-af20-2b57bb5a4250") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:56:49.894729 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:49.894375    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 01:56:49.894729 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:49.894454    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a370ba82-a51e-41aa-9ced-19bab3186de4-original-pull-secret podName:a370ba82-a51e-41aa-9ced-19bab3186de4 nodeName:}" failed. No retries permitted until 2026-07-18 01:56:51.894422882 +0000 UTC m=+6.519459530 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a370ba82-a51e-41aa-9ced-19bab3186de4-original-pull-secret") pod "global-pull-secret-syncer-jnffd" (UID: "a370ba82-a51e-41aa-9ced-19bab3186de4") : object "kube-system"/"original-pull-secret" not registered
Jul 18 01:56:49.963073 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:49.963042    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jnffd"
Jul 18 01:56:49.963210 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:49.963177    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jnffd" podUID="a370ba82-a51e-41aa-9ced-19bab3186de4"
Jul 18 01:56:49.963620 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:49.963597    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qgrtk"
Jul 18 01:56:49.963734 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:49.963713    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qgrtk" podUID="48beaadf-7df1-497b-af20-2b57bb5a4250"
Jul 18 01:56:49.963988 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:49.963869    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6pxz"
Jul 18 01:56:49.963988 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:49.963954    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h6pxz" podUID="b212c851-6975-4dcc-bd43-9421f78f6c64"
Jul 18 01:56:49.994980 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:49.994400    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtjnd\" (UniqueName: \"kubernetes.io/projected/b212c851-6975-4dcc-bd43-9421f78f6c64-kube-api-access-wtjnd\") pod \"network-check-target-h6pxz\" (UID: \"b212c851-6975-4dcc-bd43-9421f78f6c64\") " pod="openshift-network-diagnostics/network-check-target-h6pxz"
Jul 18 01:56:49.994980 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:49.994575    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 01:56:49.994980 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:49.994595    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 01:56:49.994980 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:49.994607    2576 projected.go:196] Error preparing data for projected volume kube-api-access-wtjnd for pod openshift-network-diagnostics/network-check-target-h6pxz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:56:49.994980 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:49.994661    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b212c851-6975-4dcc-bd43-9421f78f6c64-kube-api-access-wtjnd podName:b212c851-6975-4dcc-bd43-9421f78f6c64 nodeName:}" failed. No retries permitted until 2026-07-18 01:56:51.994644117 +0000 UTC m=+6.619680831 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wtjnd" (UniqueName: "kubernetes.io/projected/b212c851-6975-4dcc-bd43-9421f78f6c64-kube-api-access-wtjnd") pod "network-check-target-h6pxz" (UID: "b212c851-6975-4dcc-bd43-9421f78f6c64") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:56:50.117541 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:50.117438    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-17 01:51:48 +0000 UTC" deadline="2028-01-30 14:56:17.258727568 +0000 UTC"
Jul 18 01:56:50.117541 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:50.117474    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13476h59m27.141257991s"
Jul 18 01:56:51.912410 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:51.911745    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48beaadf-7df1-497b-af20-2b57bb5a4250-metrics-certs\") pod \"network-metrics-daemon-qgrtk\" (UID: \"48beaadf-7df1-497b-af20-2b57bb5a4250\") " pod="openshift-multus/network-metrics-daemon-qgrtk"
Jul 18 01:56:51.912410 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:51.911800    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a370ba82-a51e-41aa-9ced-19bab3186de4-original-pull-secret\") pod \"global-pull-secret-syncer-jnffd\" (UID: \"a370ba82-a51e-41aa-9ced-19bab3186de4\") " pod="kube-system/global-pull-secret-syncer-jnffd"
Jul 18 01:56:51.912410 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:51.911957    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 01:56:51.912410 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:51.912070    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a370ba82-a51e-41aa-9ced-19bab3186de4-original-pull-secret podName:a370ba82-a51e-41aa-9ced-19bab3186de4 nodeName:}" failed. No retries permitted until 2026-07-18 01:56:55.912023924 +0000 UTC m=+10.537060588 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a370ba82-a51e-41aa-9ced-19bab3186de4-original-pull-secret") pod "global-pull-secret-syncer-jnffd" (UID: "a370ba82-a51e-41aa-9ced-19bab3186de4") : object "kube-system"/"original-pull-secret" not registered
Jul 18 01:56:51.912410 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:51.912105    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:56:51.912410 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:51.912167    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48beaadf-7df1-497b-af20-2b57bb5a4250-metrics-certs podName:48beaadf-7df1-497b-af20-2b57bb5a4250 nodeName:}" failed. No retries permitted until 2026-07-18 01:56:55.912148186 +0000 UTC m=+10.537184844 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/48beaadf-7df1-497b-af20-2b57bb5a4250-metrics-certs") pod "network-metrics-daemon-qgrtk" (UID: "48beaadf-7df1-497b-af20-2b57bb5a4250") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:56:51.963502 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:51.962868    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qgrtk"
Jul 18 01:56:51.963502 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:51.962872    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jnffd"
Jul 18 01:56:51.963502 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:51.963009    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qgrtk" podUID="48beaadf-7df1-497b-af20-2b57bb5a4250"
Jul 18 01:56:51.963502 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:51.963093    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6pxz"
Jul 18 01:56:51.963502 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:51.963088    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jnffd" podUID="a370ba82-a51e-41aa-9ced-19bab3186de4"
Jul 18 01:56:51.963502 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:51.963189    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h6pxz" podUID="b212c851-6975-4dcc-bd43-9421f78f6c64"
Jul 18 01:56:52.013167 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:52.013128    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtjnd\" (UniqueName: \"kubernetes.io/projected/b212c851-6975-4dcc-bd43-9421f78f6c64-kube-api-access-wtjnd\") pod \"network-check-target-h6pxz\" (UID: \"b212c851-6975-4dcc-bd43-9421f78f6c64\") " pod="openshift-network-diagnostics/network-check-target-h6pxz"
Jul 18 01:56:52.013332 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:52.013305    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 01:56:52.013391 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:52.013356    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 01:56:52.013391 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:52.013370    2576 projected.go:196] Error preparing data for projected volume kube-api-access-wtjnd for pod openshift-network-diagnostics/network-check-target-h6pxz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:56:52.013531 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:52.013441    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b212c851-6975-4dcc-bd43-9421f78f6c64-kube-api-access-wtjnd podName:b212c851-6975-4dcc-bd43-9421f78f6c64 nodeName:}" failed. No retries permitted until 2026-07-18 01:56:56.01340899 +0000 UTC m=+10.638445638 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wtjnd" (UniqueName: "kubernetes.io/projected/b212c851-6975-4dcc-bd43-9421f78f6c64-kube-api-access-wtjnd") pod "network-check-target-h6pxz" (UID: "b212c851-6975-4dcc-bd43-9421f78f6c64") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:56:53.963035 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:53.962943    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jnffd"
Jul 18 01:56:53.963035 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:53.962967    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qgrtk"
Jul 18 01:56:53.963563 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:53.963082    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jnffd" podUID="a370ba82-a51e-41aa-9ced-19bab3186de4"
Jul 18 01:56:53.963563 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:53.963252    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6pxz"
Jul 18 01:56:53.963563 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:53.963331    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h6pxz" podUID="b212c851-6975-4dcc-bd43-9421f78f6c64"
Jul 18 01:56:53.963563 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:53.963243    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qgrtk" podUID="48beaadf-7df1-497b-af20-2b57bb5a4250"
Jul 18 01:56:55.947795 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:55.947759    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48beaadf-7df1-497b-af20-2b57bb5a4250-metrics-certs\") pod \"network-metrics-daemon-qgrtk\" (UID: \"48beaadf-7df1-497b-af20-2b57bb5a4250\") " pod="openshift-multus/network-metrics-daemon-qgrtk"
Jul 18 01:56:55.948376 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:55.947812    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a370ba82-a51e-41aa-9ced-19bab3186de4-original-pull-secret\") pod \"global-pull-secret-syncer-jnffd\" (UID: \"a370ba82-a51e-41aa-9ced-19bab3186de4\") " pod="kube-system/global-pull-secret-syncer-jnffd"
Jul 18 01:56:55.948376 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:55.947956    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 01:56:55.948376 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:55.948022    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a370ba82-a51e-41aa-9ced-19bab3186de4-original-pull-secret podName:a370ba82-a51e-41aa-9ced-19bab3186de4 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:03.948002332 +0000 UTC m=+18.573038982 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a370ba82-a51e-41aa-9ced-19bab3186de4-original-pull-secret") pod "global-pull-secret-syncer-jnffd" (UID: "a370ba82-a51e-41aa-9ced-19bab3186de4") : object "kube-system"/"original-pull-secret" not registered
Jul 18 01:56:55.948576 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:55.948410    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:56:55.948576 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:55.948472    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48beaadf-7df1-497b-af20-2b57bb5a4250-metrics-certs podName:48beaadf-7df1-497b-af20-2b57bb5a4250 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:03.948455157 +0000 UTC m=+18.573491808 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/48beaadf-7df1-497b-af20-2b57bb5a4250-metrics-certs") pod "network-metrics-daemon-qgrtk" (UID: "48beaadf-7df1-497b-af20-2b57bb5a4250") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:56:55.964617 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:55.963911    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jnffd"
Jul 18 01:56:55.964617 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:55.964027    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jnffd" podUID="a370ba82-a51e-41aa-9ced-19bab3186de4"
Jul 18 01:56:55.964617 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:55.964353    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qgrtk"
Jul 18 01:56:55.964617 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:55.964475    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qgrtk" podUID="48beaadf-7df1-497b-af20-2b57bb5a4250"
Jul 18 01:56:55.964617 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:55.964516    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6pxz"
Jul 18 01:56:55.964617 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:55.964580    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h6pxz" podUID="b212c851-6975-4dcc-bd43-9421f78f6c64"
Jul 18 01:56:56.049166 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:56.049131    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtjnd\" (UniqueName: \"kubernetes.io/projected/b212c851-6975-4dcc-bd43-9421f78f6c64-kube-api-access-wtjnd\") pod \"network-check-target-h6pxz\" (UID: \"b212c851-6975-4dcc-bd43-9421f78f6c64\") " pod="openshift-network-diagnostics/network-check-target-h6pxz"
Jul 18 01:56:56.049369 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:56.049347    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 01:56:56.049472 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:56.049379    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 01:56:56.049472 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:56.049395    2576 projected.go:196] Error preparing data for projected volume kube-api-access-wtjnd for pod openshift-network-diagnostics/network-check-target-h6pxz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:56:56.049573 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:56.049475    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b212c851-6975-4dcc-bd43-9421f78f6c64-kube-api-access-wtjnd podName:b212c851-6975-4dcc-bd43-9421f78f6c64 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:04.049454526 +0000 UTC m=+18.674491185 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-wtjnd" (UniqueName: "kubernetes.io/projected/b212c851-6975-4dcc-bd43-9421f78f6c64-kube-api-access-wtjnd") pod "network-check-target-h6pxz" (UID: "b212c851-6975-4dcc-bd43-9421f78f6c64") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:56:57.966820 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:57.966778    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jnffd"
Jul 18 01:56:57.967244 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:57.966779    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qgrtk"
Jul 18 01:56:57.967244 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:57.966902    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jnffd" podUID="a370ba82-a51e-41aa-9ced-19bab3186de4"
Jul 18 01:56:57.967244 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:57.967007    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qgrtk" podUID="48beaadf-7df1-497b-af20-2b57bb5a4250"
Jul 18 01:56:57.967244 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:57.966779    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6pxz"
Jul 18 01:56:57.967244 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:57.967100    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h6pxz" podUID="b212c851-6975-4dcc-bd43-9421f78f6c64"
Jul 18 01:56:59.966128 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:59.966098    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6pxz"
Jul 18 01:56:59.966571 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:59.966201    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jnffd"
Jul 18 01:56:59.966571 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:59.966213    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h6pxz" podUID="b212c851-6975-4dcc-bd43-9421f78f6c64"
Jul 18 01:56:59.966571 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:59.966308    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jnffd" podUID="a370ba82-a51e-41aa-9ced-19bab3186de4"
Jul 18 01:56:59.966571 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:56:59.966379    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qgrtk"
Jul 18 01:56:59.966571 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:56:59.966486    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qgrtk" podUID="48beaadf-7df1-497b-af20-2b57bb5a4250"
Jul 18 01:57:01.963602 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:01.963568    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jnffd"
Jul 18 01:57:01.964448 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:01.963568    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qgrtk"
Jul 18 01:57:01.964448 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:01.963698    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jnffd" podUID="a370ba82-a51e-41aa-9ced-19bab3186de4"
Jul 18 01:57:01.964448 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:01.963570    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6pxz"
Jul 18 01:57:01.964448 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:01.963800    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qgrtk" podUID="48beaadf-7df1-497b-af20-2b57bb5a4250"
Jul 18 01:57:01.964448 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:01.963911    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h6pxz" podUID="b212c851-6975-4dcc-bd43-9421f78f6c64"
Jul 18 01:57:03.963109 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:03.963071    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qgrtk"
Jul 18 01:57:03.963616 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:03.963071    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jnffd"
Jul 18 01:57:03.963616 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:03.963208    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qgrtk" podUID="48beaadf-7df1-497b-af20-2b57bb5a4250"
Jul 18 01:57:03.963616 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:03.963071    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6pxz"
Jul 18 01:57:03.963616 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:03.963305    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jnffd" podUID="a370ba82-a51e-41aa-9ced-19bab3186de4"
Jul 18 01:57:03.963616 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:03.963346    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h6pxz" podUID="b212c851-6975-4dcc-bd43-9421f78f6c64"
Jul 18 01:57:04.006875 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:04.006844    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48beaadf-7df1-497b-af20-2b57bb5a4250-metrics-certs\") pod \"network-metrics-daemon-qgrtk\" (UID: \"48beaadf-7df1-497b-af20-2b57bb5a4250\") " pod="openshift-multus/network-metrics-daemon-qgrtk"
Jul 18 01:57:04.007045 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:04.006884    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a370ba82-a51e-41aa-9ced-19bab3186de4-original-pull-secret\") pod \"global-pull-secret-syncer-jnffd\" (UID: \"a370ba82-a51e-41aa-9ced-19bab3186de4\") " pod="kube-system/global-pull-secret-syncer-jnffd"
Jul 18 01:57:04.007045 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:04.006996    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 01:57:04.007126 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:04.007048    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a370ba82-a51e-41aa-9ced-19bab3186de4-original-pull-secret podName:a370ba82-a51e-41aa-9ced-19bab3186de4 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:20.00703477 +0000 UTC m=+34.632071415 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a370ba82-a51e-41aa-9ced-19bab3186de4-original-pull-secret") pod "global-pull-secret-syncer-jnffd" (UID: "a370ba82-a51e-41aa-9ced-19bab3186de4") : object "kube-system"/"original-pull-secret" not registered
Jul 18 01:57:04.007126 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:04.006997    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:57:04.007126 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:04.007116    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48beaadf-7df1-497b-af20-2b57bb5a4250-metrics-certs podName:48beaadf-7df1-497b-af20-2b57bb5a4250 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:20.007103332 +0000 UTC m=+34.632139993 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/48beaadf-7df1-497b-af20-2b57bb5a4250-metrics-certs") pod "network-metrics-daemon-qgrtk" (UID: "48beaadf-7df1-497b-af20-2b57bb5a4250") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:57:04.107745 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:04.107701    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtjnd\" (UniqueName: \"kubernetes.io/projected/b212c851-6975-4dcc-bd43-9421f78f6c64-kube-api-access-wtjnd\") pod \"network-check-target-h6pxz\" (UID: \"b212c851-6975-4dcc-bd43-9421f78f6c64\") " pod="openshift-network-diagnostics/network-check-target-h6pxz"
Jul 18 01:57:04.107905 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:04.107878    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 01:57:04.107905 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:04.107905    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 01:57:04.108004 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:04.107920    2576 projected.go:196] Error preparing data for projected volume kube-api-access-wtjnd for pod openshift-network-diagnostics/network-check-target-h6pxz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:57:04.108004 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:04.107992    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b212c851-6975-4dcc-bd43-9421f78f6c64-kube-api-access-wtjnd podName:b212c851-6975-4dcc-bd43-9421f78f6c64 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:20.107970738 +0000 UTC m=+34.733007384 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-wtjnd" (UniqueName: "kubernetes.io/projected/b212c851-6975-4dcc-bd43-9421f78f6c64-kube-api-access-wtjnd") pod "network-check-target-h6pxz" (UID: "b212c851-6975-4dcc-bd43-9421f78f6c64") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:57:05.964226 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:05.964187    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qgrtk"
Jul 18 01:57:05.964226 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:05.964215    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6pxz"
Jul 18 01:57:05.964754 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:05.964316    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qgrtk" podUID="48beaadf-7df1-497b-af20-2b57bb5a4250"
Jul 18 01:57:05.964754 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:05.964386    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jnffd"
Jul 18 01:57:05.964754 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:05.964379    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h6pxz" podUID="b212c851-6975-4dcc-bd43-9421f78f6c64"
Jul 18 01:57:05.964754 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:05.964493    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jnffd" podUID="a370ba82-a51e-41aa-9ced-19bab3186de4"
Jul 18 01:57:07.963093 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:07.962937    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jnffd"
Jul 18 01:57:07.963443 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:07.962947    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qgrtk"
Jul 18 01:57:07.963443 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:07.963171    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jnffd" podUID="a370ba82-a51e-41aa-9ced-19bab3186de4"
Jul 18 01:57:07.963443 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:07.963244    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qgrtk" podUID="48beaadf-7df1-497b-af20-2b57bb5a4250"
Jul 18 01:57:07.963443 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:07.962960    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6pxz"
Jul 18 01:57:07.963443 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:07.963305    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h6pxz" podUID="b212c851-6975-4dcc-bd43-9421f78f6c64"
Jul 18 01:57:08.147512 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:08.147222    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-8.ec2.internal" event={"ID":"ef3a0a5df8ac39131447779dfceb447b","Type":"ContainerStarted","Data":"492590c6a45436d0e77aca27ee6cebf14a6a6aaa9d1b8b3597dce5e2e6c19c7c"}
Jul 18 01:57:08.148887 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:08.148853    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xlrxq" event={"ID":"0c5d4061-2764-4595-9bda-43de3b3cfe15","Type":"ContainerStarted","Data":"74120c94a118689c5328102ae2284fb11b3468e6ef2aeaf81a58fa2598eff2d4"}
Jul 18 01:57:08.156180 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:08.156155    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-grrdq" event={"ID":"d01349d2-5a70-4df6-b978-a3cf84438584","Type":"ContainerStarted","Data":"d053a78d4f18e433299913ac37b3e0794991e8ea69329d8f280bbea0d1e789ce"}
Jul 18 01:57:08.162611 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:08.162568    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-8.ec2.internal" podStartSLOduration=22.162551816 podStartE2EDuration="22.162551816s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:56:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:08.161919374 +0000 UTC m=+22.786956042" watchObservedRunningTime="2026-07-18 01:57:08.162551816 +0000 UTC m=+22.787588477"
Jul 18 01:57:08.178046 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:08.177985    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-grrdq" podStartSLOduration=2.810047268 podStartE2EDuration="22.1779674s" totalImagesPullingTime="19.367920132s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:56:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:08.177390429 +0000 UTC m=+22.802427097" watchObservedRunningTime="2026-07-18 01:57:08.1779674 +0000 UTC m=+22.803004066"
Jul 18 01:57:08.194359 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:08.194017    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xlrxq" podStartSLOduration=3.103524765 podStartE2EDuration="22.193993467s" totalImagesPullingTime="19.090468702s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:56:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:08.192848205 +0000 UTC m=+22.817884886" watchObservedRunningTime="2026-07-18 01:57:08.193993467 +0000 UTC m=+22.819030142"
Jul 18 01:57:09.160662 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:09.160309    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k92lz" event={"ID":"d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58","Type":"ContainerStarted","Data":"6b5cd2f7433d3a8345d578dadacfe22c11c308aab6393c1cdcf50d29f545124b"}
Jul 18 01:57:09.163710 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:09.163678    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jfzth" event={"ID":"60cb632f-f03b-491f-8a38-d297305c9992","Type":"ContainerStarted","Data":"6b62c03a96c9f1aa8e69fabf3b111143594a3f216df1174420399268cb0a6215"}
Jul 18 01:57:09.163839 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:09.163760    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jfzth" event={"ID":"60cb632f-f03b-491f-8a38-d297305c9992","Type":"ContainerStarted","Data":"ce7da5f4abf58f3cee9e835aaf059abdbce0828c6779d753c403a41c5a45609c"}
Jul 18 01:57:09.163839 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:09.163811    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jfzth" event={"ID":"60cb632f-f03b-491f-8a38-d297305c9992","Type":"ContainerStarted","Data":"32992d57a063d658f98b971e07cdef831eca4fae0658b99de3b633d3187229ce"}
Jul 18 01:57:09.163958 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:09.163859    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jfzth" event={"ID":"60cb632f-f03b-491f-8a38-d297305c9992","Type":"ContainerStarted","Data":"fd9ce2cd006bc2b07b7f0f971417bc08c2a7d33c87ccd57f1d6b6fdcbd07b486"}
Jul 18 01:57:09.163958 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:09.163905    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jfzth" event={"ID":"60cb632f-f03b-491f-8a38-d297305c9992","Type":"ContainerStarted","Data":"1edc6abe55e407f9b3c487276e8a496353f4ef459434c3c606795b4956cc8849"}
Jul 18 01:57:09.163958 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:09.163951    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jfzth" event={"ID":"60cb632f-f03b-491f-8a38-d297305c9992","Type":"ContainerStarted","Data":"acf025bcdddebcd2a4f705e8482a3c2f0405443714224c2740196c435a473ef2"}
Jul 18 01:57:09.165336 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:09.165310    2576 generic.go:356] "Generic (PLEG): container finished" podID="cff575d5-ef70-4142-8450-b15c8038b6f8" containerID="fe91ddac64206f0d8880f531bc10e1ccd115d1ac1c9e5edff96be578641e2dfb" exitCode=0
Jul 18 01:57:09.165467 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:09.165384    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sd77t" event={"ID":"cff575d5-ef70-4142-8450-b15c8038b6f8","Type":"ContainerDied","Data":"fe91ddac64206f0d8880f531bc10e1ccd115d1ac1c9e5edff96be578641e2dfb"}
Jul 18 01:57:09.166941 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:09.166892    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-f5kmg" event={"ID":"33351d5c-d8c8-4d94-8209-28d15797b7a9","Type":"ContainerStarted","Data":"1ea5322d665b412f2014f603dae31d362a002c1b83d98e40f0dff0f66715ddba"}
Jul 18 01:57:09.169367 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:09.168510    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-t7kmc" event={"ID":"ad82bb0f-ce9c-416d-b049-6e6e8ddfaba8","Type":"ContainerStarted","Data":"be21788117f4fa703ac121fb0d1bfa911a81dd05aec31b105165f6212005c48a"}
Jul 18 01:57:09.170243 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:09.170205    2576 generic.go:356] "Generic (PLEG): container finished" podID="5bf26a2caa79e4ddbb4f56b4c1019bf4" containerID="7191328fd235157cb3a9f457dcd803ab2743c412e6181fc977cf234ec8a070f2" exitCode=0
Jul 18 01:57:09.170468 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:09.170444    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-8.ec2.internal" event={"ID":"5bf26a2caa79e4ddbb4f56b4c1019bf4","Type":"ContainerDied","Data":"7191328fd235157cb3a9f457dcd803ab2743c412e6181fc977cf234ec8a070f2"}
Jul 18 01:57:09.172461 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:09.172384    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-25924" event={"ID":"a3f34325-5076-4e41-a22d-5caece5dbece","Type":"ContainerStarted","Data":"2cfec3f4e1fb28d91056d067ea24ae40130f244143d80be956eea5091c1a7b4f"}
Jul 18 01:57:09.201921 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:09.201865    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-t7kmc" podStartSLOduration=13.475683174 podStartE2EDuration="23.201844183s" totalImagesPullingTime="9.726161009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:56:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:09.201338181 +0000 UTC m=+23.826374848" watchObservedRunningTime="2026-07-18 01:57:09.201844183 +0000 UTC m=+23.826880851"
Jul 18 01:57:09.219728 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:09.219656    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-25924" podStartSLOduration=4.140085963 podStartE2EDuration="23.219641905s" totalImagesPullingTime="19.079555942s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:56:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:09.219171825 +0000 UTC m=+23.844208491" watchObservedRunningTime="2026-07-18 01:57:09.219641905 +0000 UTC m=+23.844678572"
Jul 18 01:57:09.233676 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:09.233623    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-f5kmg" podStartSLOduration=4.154077523 podStartE2EDuration="23.233593083s" totalImagesPullingTime="19.07951556s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:56:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:09.233569955 +0000 UTC m=+23.858606621" watchObservedRunningTime="2026-07-18 01:57:09.233593083 +0000 UTC m=+23.858629751"
Jul 18 01:57:09.647704 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:09.647530    2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 18 01:57:09.963276 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:09.963239    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jnffd"
Jul 18 01:57:09.963570 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:09.963239    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6pxz"
Jul 18 01:57:09.963570 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:09.963360    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jnffd" podUID="a370ba82-a51e-41aa-9ced-19bab3186de4"
Jul 18 01:57:09.963570 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:09.963407    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h6pxz" podUID="b212c851-6975-4dcc-bd43-9421f78f6c64"
Jul 18 01:57:09.963570 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:09.963239    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qgrtk"
Jul 18 01:57:09.963570 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:09.963558    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qgrtk" podUID="48beaadf-7df1-497b-af20-2b57bb5a4250"
Jul 18 01:57:10.085189 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:10.085105    2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-18T01:57:09.647699598Z","UUID":"1f048e4f-59fe-4241-a9f3-e307c1966713","Handler":null,"Name":"","Endpoint":""}
Jul 18 01:57:10.086904 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:10.086876    2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 18 01:57:10.087026 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:10.086915    2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 18 01:57:10.176012 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:10.175977    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-8.ec2.internal" event={"ID":"5bf26a2caa79e4ddbb4f56b4c1019bf4","Type":"ContainerStarted","Data":"432994dd8e4e0bc75d5111e1249bced6d709b52fd72c8ad97255f1924e1cb051"}
Jul 18 01:57:10.177655 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:10.177628    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k92lz" event={"ID":"d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58","Type":"ContainerStarted","Data":"2a082ea5de192d0d89c6f445a3d682058f63a9fb648ba6e5cd52891e657d228c"}
Jul 18 01:57:10.178837 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:10.178813    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-s42pb" event={"ID":"537a091c-f327-488e-85e8-600ce7edbfcf","Type":"ContainerStarted","Data":"bfeac1e0f466737fb0a93273802005e5dc965c4a01ef2443443c43b1e3bfbd7b"}
Jul 18 01:57:10.194249 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:10.194209    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-8.ec2.internal" podStartSLOduration=24.19419616 podStartE2EDuration="24.19419616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:56:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:10.193743782 +0000 UTC m=+24.818780449" watchObservedRunningTime="2026-07-18 01:57:10.19419616 +0000 UTC m=+24.819232825"
Jul 18 01:57:10.210927 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:10.210872    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-s42pb" podStartSLOduration=5.116707352 podStartE2EDuration="24.210850567s" totalImagesPullingTime="19.094143215s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:56:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:10.210507252 +0000 UTC m=+24.835543919" watchObservedRunningTime="2026-07-18 01:57:10.210850567 +0000 UTC m=+24.835887237"
Jul 18 01:57:11.183497 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:11.183465    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k92lz" event={"ID":"d7d8a55e-33c8-4795-b0fe-6d0f6bae9d58","Type":"ContainerStarted","Data":"29b126ec0e93bd59fd24b61877886a6ca65765b1be16e601a145cd420c411339"}
Jul 18 01:57:11.187153 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:11.187122    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jfzth" event={"ID":"60cb632f-f03b-491f-8a38-d297305c9992","Type":"ContainerStarted","Data":"be1a79dd8eebebae406e848bfbf2268398fdbeff533ce58533d8fdd2c224fe49"}
Jul 18 01:57:11.217096 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:11.216991    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k92lz" podStartSLOduration=8.290810107 podStartE2EDuration="25.216975009s" totalImagesPullingTime="16.926164902s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:56:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:11.216859714 +0000 UTC m=+25.841896383" watchObservedRunningTime="2026-07-18 01:57:11.216975009 +0000 UTC m=+25.842011677"
Jul 18 01:57:11.963853 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:11.963816    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jnffd"
Jul 18 01:57:11.964075 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:11.963816    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6pxz"
Jul 18 01:57:11.964075 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:11.963981    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jnffd" podUID="a370ba82-a51e-41aa-9ced-19bab3186de4"
Jul 18 01:57:11.964075 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:11.963816    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qgrtk"
Jul 18 01:57:11.964075 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:11.964031    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h6pxz" podUID="b212c851-6975-4dcc-bd43-9421f78f6c64"
Jul 18 01:57:11.964291 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:11.964099    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qgrtk" podUID="48beaadf-7df1-497b-af20-2b57bb5a4250"
Jul 18 01:57:13.456812 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:13.456727    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-t7kmc"
Jul 18 01:57:13.457350 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:13.457329    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-t7kmc"
Jul 18 01:57:13.966758 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:13.966732    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jnffd"
Jul 18 01:57:13.966939 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:13.966836    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6pxz"
Jul 18 01:57:13.966939 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:13.966870    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qgrtk"
Jul 18 01:57:13.966939 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:13.966866    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jnffd" podUID="a370ba82-a51e-41aa-9ced-19bab3186de4"
Jul 18 01:57:13.967087 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:13.966945    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h6pxz" podUID="b212c851-6975-4dcc-bd43-9421f78f6c64"
Jul 18 01:57:13.967087 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:13.967031    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qgrtk" podUID="48beaadf-7df1-497b-af20-2b57bb5a4250"
Jul 18 01:57:15.200768 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:15.200559    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jfzth" event={"ID":"60cb632f-f03b-491f-8a38-d297305c9992","Type":"ContainerStarted","Data":"3c173eb83e98ea14b5f54895a1153f0b4350c448e795187966fa9b2717e6f965"}
Jul 18 01:57:15.201530 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:15.200831    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:57:15.201530 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:15.200876    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:57:15.202831 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:15.202801    2576 generic.go:356] "Generic (PLEG): container finished" podID="cff575d5-ef70-4142-8450-b15c8038b6f8" containerID="c4b329b2d042401afaefe704c56d8f16630dd0e8638846d8cac841352fe8196b" exitCode=0
Jul 18 01:57:15.202947 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:15.202844    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sd77t" event={"ID":"cff575d5-ef70-4142-8450-b15c8038b6f8","Type":"ContainerDied","Data":"c4b329b2d042401afaefe704c56d8f16630dd0e8638846d8cac841352fe8196b"}
Jul 18 01:57:15.217536 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:15.217515    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:57:15.229114 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:15.229061    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-jfzth" podStartSLOduration=9.775649649 podStartE2EDuration="29.229016618s" totalImagesPullingTime="19.453366969s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:56:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:15.226907042 +0000 UTC m=+29.851943709" watchObservedRunningTime="2026-07-18 01:57:15.229016618 +0000 UTC m=+29.854053287"
Jul 18 01:57:15.966851 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:15.966820    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-t7kmc"
Jul 18 01:57:15.967027 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:15.966935    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-t7kmc"
Jul 18 01:57:15.967079 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:15.967027    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6pxz"
Jul 18 01:57:15.967079 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:15.967032    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jnffd"
Jul 18 01:57:15.967243 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:15.967216    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h6pxz" podUID="b212c851-6975-4dcc-bd43-9421f78f6c64"
Jul 18 01:57:15.967339 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:15.967318    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jnffd" podUID="a370ba82-a51e-41aa-9ced-19bab3186de4"
Jul 18 01:57:15.967399 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:15.967369    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qgrtk"
Jul 18 01:57:15.967492 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:15.967455    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qgrtk" podUID="48beaadf-7df1-497b-af20-2b57bb5a4250"
Jul 18 01:57:16.185158 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:16.185128    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jnffd"]
Jul 18 01:57:16.188042 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:16.188016    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qgrtk"]
Jul 18 01:57:16.202736 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:16.201372    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-h6pxz"]
Jul 18 01:57:16.205012 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:16.204988    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6pxz"
Jul 18 01:57:16.205149 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:16.205020    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qgrtk"
Jul 18 01:57:16.205149 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:16.204989    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jnffd"
Jul 18 01:57:16.205149 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:16.205091    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h6pxz" podUID="b212c851-6975-4dcc-bd43-9421f78f6c64"
Jul 18 01:57:16.205312 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:16.205194    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jnffd" podUID="a370ba82-a51e-41aa-9ced-19bab3186de4"
Jul 18 01:57:16.205649 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:16.205617    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qgrtk" podUID="48beaadf-7df1-497b-af20-2b57bb5a4250"
Jul 18 01:57:16.205649 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:16.205631    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:57:16.221173 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:16.221110    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:57:17.209200 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:17.208979    2576 generic.go:356] "Generic (PLEG): container finished" podID="cff575d5-ef70-4142-8450-b15c8038b6f8" containerID="4aae3e4e681ae9a8872f1059caaa7fbcc7acd87addd1e7bb22e048542438ce25" exitCode=0
Jul 18 01:57:17.209200 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:17.209037    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sd77t" event={"ID":"cff575d5-ef70-4142-8450-b15c8038b6f8","Type":"ContainerDied","Data":"4aae3e4e681ae9a8872f1059caaa7fbcc7acd87addd1e7bb22e048542438ce25"}
Jul 18 01:57:17.966440 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:17.966392    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6pxz"
Jul 18 01:57:17.966599 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:17.966504    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jnffd"
Jul 18 01:57:17.966599 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:17.966516    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h6pxz" podUID="b212c851-6975-4dcc-bd43-9421f78f6c64"
Jul 18 01:57:17.966599 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:17.966534    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qgrtk"
Jul 18 01:57:17.966742 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:17.966609    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jnffd" podUID="a370ba82-a51e-41aa-9ced-19bab3186de4"
Jul 18 01:57:17.966742 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:17.966685    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qgrtk" podUID="48beaadf-7df1-497b-af20-2b57bb5a4250"
Jul 18 01:57:19.216342 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:19.216312    2576 generic.go:356] "Generic (PLEG): container finished" podID="cff575d5-ef70-4142-8450-b15c8038b6f8" containerID="2e6c2879ace716c55a150d72c06a5d9546b5a3b8eabe01a653d9e3e0b51634fe" exitCode=0
Jul 18 01:57:19.216726 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:19.216359    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sd77t" event={"ID":"cff575d5-ef70-4142-8450-b15c8038b6f8","Type":"ContainerDied","Data":"2e6c2879ace716c55a150d72c06a5d9546b5a3b8eabe01a653d9e3e0b51634fe"}
Jul 18 01:57:19.966973 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:19.966943    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jnffd"
Jul 18 01:57:19.966973 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:19.966965    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qgrtk"
Jul 18 01:57:19.967208 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:19.966943    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6pxz"
Jul 18 01:57:19.967208 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:19.967066    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jnffd" podUID="a370ba82-a51e-41aa-9ced-19bab3186de4"
Jul 18 01:57:19.967208 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:19.967138    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qgrtk" podUID="48beaadf-7df1-497b-af20-2b57bb5a4250"
Jul 18 01:57:19.967356 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:19.967226    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h6pxz" podUID="b212c851-6975-4dcc-bd43-9421f78f6c64"
Jul 18 01:57:20.025068 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:20.025018    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48beaadf-7df1-497b-af20-2b57bb5a4250-metrics-certs\") pod \"network-metrics-daemon-qgrtk\" (UID: \"48beaadf-7df1-497b-af20-2b57bb5a4250\") " pod="openshift-multus/network-metrics-daemon-qgrtk"
Jul 18 01:57:20.025258 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:20.025147    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:57:20.025258 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:20.025159    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a370ba82-a51e-41aa-9ced-19bab3186de4-original-pull-secret\") pod \"global-pull-secret-syncer-jnffd\" (UID: \"a370ba82-a51e-41aa-9ced-19bab3186de4\") " pod="kube-system/global-pull-secret-syncer-jnffd"
Jul 18 01:57:20.025258 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:20.025222    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48beaadf-7df1-497b-af20-2b57bb5a4250-metrics-certs podName:48beaadf-7df1-497b-af20-2b57bb5a4250 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:52.025203442 +0000 UTC m=+66.650240143 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/48beaadf-7df1-497b-af20-2b57bb5a4250-metrics-certs") pod "network-metrics-daemon-qgrtk" (UID: "48beaadf-7df1-497b-af20-2b57bb5a4250") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:57:20.025258 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:20.025253    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 01:57:20.025547 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:20.025311    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a370ba82-a51e-41aa-9ced-19bab3186de4-original-pull-secret podName:a370ba82-a51e-41aa-9ced-19bab3186de4 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:52.025290349 +0000 UTC m=+66.650326994 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a370ba82-a51e-41aa-9ced-19bab3186de4-original-pull-secret") pod "global-pull-secret-syncer-jnffd" (UID: "a370ba82-a51e-41aa-9ced-19bab3186de4") : object "kube-system"/"original-pull-secret" not registered
Jul 18 01:57:20.125808 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:20.125712    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtjnd\" (UniqueName: \"kubernetes.io/projected/b212c851-6975-4dcc-bd43-9421f78f6c64-kube-api-access-wtjnd\") pod \"network-check-target-h6pxz\" (UID: \"b212c851-6975-4dcc-bd43-9421f78f6c64\") " pod="openshift-network-diagnostics/network-check-target-h6pxz"
Jul 18 01:57:20.125972 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:20.125866    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 01:57:20.125972 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:20.125885    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 01:57:20.125972 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:20.125897    2576 projected.go:196] Error preparing data for projected volume kube-api-access-wtjnd for pod openshift-network-diagnostics/network-check-target-h6pxz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:57:20.125972 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:20.125963    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b212c851-6975-4dcc-bd43-9421f78f6c64-kube-api-access-wtjnd podName:b212c851-6975-4dcc-bd43-9421f78f6c64 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:52.125946123 +0000 UTC m=+66.750982781 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-wtjnd" (UniqueName: "kubernetes.io/projected/b212c851-6975-4dcc-bd43-9421f78f6c64-kube-api-access-wtjnd") pod "network-check-target-h6pxz" (UID: "b212c851-6975-4dcc-bd43-9421f78f6c64") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:57:21.650834 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.650800    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-8.ec2.internal" event="NodeReady"
Jul 18 01:57:21.651290 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.650944    2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 18 01:57:21.715766 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.715730    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-76cf94c9f7-87dhh"]
Jul 18 01:57:21.719407 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.719379    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh"
Jul 18 01:57:21.722093 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.722069    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-q7jhh\""
Jul 18 01:57:21.722093 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.722069    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 18 01:57:21.722275 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.722187    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 18 01:57:21.722577 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.722417    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 18 01:57:21.728126 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.728078    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 18 01:57:21.730040 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.730017    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-kj28b"]
Jul 18 01:57:21.734699 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.734665    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76cf94c9f7-87dhh"]
Jul 18 01:57:21.734797 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.734773    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-tw7tf"]
Jul 18 01:57:21.734838 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.734797    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kj28b"
Jul 18 01:57:21.737228 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.737205    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-96mg4\""
Jul 18 01:57:21.737339 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.737274    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 18 01:57:21.737509 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.737492    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 18 01:57:21.740481 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.739816    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tw7tf"
Jul 18 01:57:21.741962 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.741943    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 18 01:57:21.742595 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.742576    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kcv7c\""
Jul 18 01:57:21.742849 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.742635    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 18 01:57:21.742849 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.742585    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 18 01:57:21.746536 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.746511    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-kj28b"]
Jul 18 01:57:21.749403 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.749383    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tw7tf"]
Jul 18 01:57:21.841052 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.840978    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-trusted-ca\") pod \"image-registry-76cf94c9f7-87dhh\" (UID: \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\") " pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh"
Jul 18 01:57:21.841052 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.841024    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/66623dd6-501f-42b2-b12a-188cf5520290-cert\") pod \"ingress-canary-tw7tf\" (UID: \"66623dd6-501f-42b2-b12a-188cf5520290\") " pod="openshift-ingress-canary/ingress-canary-tw7tf"
Jul 18 01:57:21.841223 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.841100    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-installation-pull-secrets\") pod \"image-registry-76cf94c9f7-87dhh\" (UID: \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\") " pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh"
Jul 18 01:57:21.841223 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.841118    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-bound-sa-token\") pod \"image-registry-76cf94c9f7-87dhh\" (UID: \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\") " pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh"
Jul 18 01:57:21.841223 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.841134    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlcm8\" (UniqueName: \"kubernetes.io/projected/c9fa15c4-1db0-4b36-999d-5507f4cb2d19-kube-api-access-dlcm8\") pod \"dns-default-kj28b\" (UID: \"c9fa15c4-1db0-4b36-999d-5507f4cb2d19\") " pod="openshift-dns/dns-default-kj28b"
Jul 18 01:57:21.841223 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.841164    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c9fa15c4-1db0-4b36-999d-5507f4cb2d19-config-volume\") pod \"dns-default-kj28b\" (UID: \"c9fa15c4-1db0-4b36-999d-5507f4cb2d19\") " pod="openshift-dns/dns-default-kj28b"
Jul 18 01:57:21.841223 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.841182    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-registry-certificates\") pod \"image-registry-76cf94c9f7-87dhh\" (UID: \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\") " pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh"
Jul 18 01:57:21.841415 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.841221    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c9fa15c4-1db0-4b36-999d-5507f4cb2d19-metrics-tls\") pod \"dns-default-kj28b\" (UID: \"c9fa15c4-1db0-4b36-999d-5507f4cb2d19\") " pod="openshift-dns/dns-default-kj28b"
Jul 18 01:57:21.841415 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.841254    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-registry-tls\") pod \"image-registry-76cf94c9f7-87dhh\" (UID: \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\") " pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh"
Jul 18 01:57:21.841415 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.841280    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82t5j\" (UniqueName: \"kubernetes.io/projected/66623dd6-501f-42b2-b12a-188cf5520290-kube-api-access-82t5j\") pod \"ingress-canary-tw7tf\" (UID: \"66623dd6-501f-42b2-b12a-188cf5520290\") " pod="openshift-ingress-canary/ingress-canary-tw7tf"
Jul 18 01:57:21.841415 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.841342    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbt55\" (UniqueName: \"kubernetes.io/projected/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-kube-api-access-vbt55\") pod \"image-registry-76cf94c9f7-87dhh\" (UID: \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\") " pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh"
Jul 18 01:57:21.841415 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.841378    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-ca-trust-extracted\") pod \"image-registry-76cf94c9f7-87dhh\" (UID: \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\") " pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh"
Jul 18 01:57:21.841415 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.841408    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-image-registry-private-configuration\") pod \"image-registry-76cf94c9f7-87dhh\" (UID: \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\") " pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh"
Jul 18 01:57:21.841624 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.841437    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9fa15c4-1db0-4b36-999d-5507f4cb2d19-tmp-dir\") pod \"dns-default-kj28b\" (UID: \"c9fa15c4-1db0-4b36-999d-5507f4cb2d19\") " pod="openshift-dns/dns-default-kj28b"
Jul 18 01:57:21.942381 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.942339    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-registry-tls\") pod \"image-registry-76cf94c9f7-87dhh\" (UID: \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\") " pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh"
Jul 18 01:57:21.942593 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.942401    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82t5j\" (UniqueName: \"kubernetes.io/projected/66623dd6-501f-42b2-b12a-188cf5520290-kube-api-access-82t5j\") pod \"ingress-canary-tw7tf\" (UID: \"66623dd6-501f-42b2-b12a-188cf5520290\") " pod="openshift-ingress-canary/ingress-canary-tw7tf"
Jul 18 01:57:21.942593 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.942447    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbt55\" (UniqueName: \"kubernetes.io/projected/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-kube-api-access-vbt55\") pod \"image-registry-76cf94c9f7-87dhh\" (UID: \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\") " pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh"
Jul 18 01:57:21.942593 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.942472    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-ca-trust-extracted\") pod \"image-registry-76cf94c9f7-87dhh\" (UID: \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\") " pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh"
Jul 18 01:57:21.942593 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:21.942501    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 01:57:21.942593 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:21.942527    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76cf94c9f7-87dhh: secret "image-registry-tls" not found
Jul 18 01:57:21.942593 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:21.942593    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-registry-tls podName:f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:22.442570577 +0000 UTC m=+37.067607287 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-registry-tls") pod "image-registry-76cf94c9f7-87dhh" (UID: "f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09") : secret "image-registry-tls" not found
Jul 18 01:57:21.942902 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.942505    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-image-registry-private-configuration\") pod \"image-registry-76cf94c9f7-87dhh\" (UID: \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\") " pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh"
Jul 18 01:57:21.942902 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.942756    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9fa15c4-1db0-4b36-999d-5507f4cb2d19-tmp-dir\") pod \"dns-default-kj28b\" (UID: \"c9fa15c4-1db0-4b36-999d-5507f4cb2d19\") " pod="openshift-dns/dns-default-kj28b"
Jul 18 01:57:21.942902 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.942797    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-trusted-ca\") pod \"image-registry-76cf94c9f7-87dhh\" (UID: \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\") " pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh"
Jul 18 01:57:21.942902 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.942830    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/66623dd6-501f-42b2-b12a-188cf5520290-cert\") pod \"ingress-canary-tw7tf\" (UID: \"66623dd6-501f-42b2-b12a-188cf5520290\") " pod="openshift-ingress-canary/ingress-canary-tw7tf"
Jul 18 01:57:21.942902 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.942899    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-installation-pull-secrets\") pod \"image-registry-76cf94c9f7-87dhh\" (UID: \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\") " pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh"
Jul 18 01:57:21.943160 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.942904    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-ca-trust-extracted\") pod \"image-registry-76cf94c9f7-87dhh\" (UID: \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\") " pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh"
Jul 18 01:57:21.943160 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.942929    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-bound-sa-token\") pod \"image-registry-76cf94c9f7-87dhh\" (UID: \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\") " pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh"
Jul 18 01:57:21.943160 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.942954    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlcm8\" (UniqueName: \"kubernetes.io/projected/c9fa15c4-1db0-4b36-999d-5507f4cb2d19-kube-api-access-dlcm8\") pod \"dns-default-kj28b\" (UID: \"c9fa15c4-1db0-4b36-999d-5507f4cb2d19\") " pod="openshift-dns/dns-default-kj28b"
Jul 18 01:57:21.943160 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.943009    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c9fa15c4-1db0-4b36-999d-5507f4cb2d19-config-volume\") pod \"dns-default-kj28b\" (UID: \"c9fa15c4-1db0-4b36-999d-5507f4cb2d19\") " pod="openshift-dns/dns-default-kj28b"
Jul 18 01:57:21.943160 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.943038    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-registry-certificates\") pod \"image-registry-76cf94c9f7-87dhh\" (UID: \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\") " pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh"
Jul 18 01:57:21.943160 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.943063    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c9fa15c4-1db0-4b36-999d-5507f4cb2d19-metrics-tls\") pod \"dns-default-kj28b\" (UID: \"c9fa15c4-1db0-4b36-999d-5507f4cb2d19\") " pod="openshift-dns/dns-default-kj28b"
Jul 18 01:57:21.943160 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.943121    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9fa15c4-1db0-4b36-999d-5507f4cb2d19-tmp-dir\") pod \"dns-default-kj28b\" (UID: \"c9fa15c4-1db0-4b36-999d-5507f4cb2d19\") " pod="openshift-dns/dns-default-kj28b"
Jul 18 01:57:21.943513 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:21.943175    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 01:57:21.943513 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:21.943219    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9fa15c4-1db0-4b36-999d-5507f4cb2d19-metrics-tls podName:c9fa15c4-1db0-4b36-999d-5507f4cb2d19 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:22.443204208 +0000 UTC m=+37.068240857 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c9fa15c4-1db0-4b36-999d-5507f4cb2d19-metrics-tls") pod "dns-default-kj28b" (UID: "c9fa15c4-1db0-4b36-999d-5507f4cb2d19") : secret "dns-default-metrics-tls" not found
Jul 18 01:57:21.943876 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.943853    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-trusted-ca\") pod \"image-registry-76cf94c9f7-87dhh\" (UID: \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\") " pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh"
Jul 18 01:57:21.943975 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.943862    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c9fa15c4-1db0-4b36-999d-5507f4cb2d19-config-volume\") pod \"dns-default-kj28b\" (UID: \"c9fa15c4-1db0-4b36-999d-5507f4cb2d19\") " pod="openshift-dns/dns-default-kj28b"
Jul 18 01:57:21.944027 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.944013    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-registry-certificates\") pod \"image-registry-76cf94c9f7-87dhh\" (UID: \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\") " pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh"
Jul 18 01:57:21.947639 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.947618    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-image-registry-private-configuration\") pod \"image-registry-76cf94c9f7-87dhh\" (UID: \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\") " pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh"
Jul 18 01:57:21.947743 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.947615    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-installation-pull-secrets\") pod \"image-registry-76cf94c9f7-87dhh\" (UID: \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\") " pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh"
Jul 18 01:57:21.947743 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.947628    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/66623dd6-501f-42b2-b12a-188cf5520290-cert\") pod \"ingress-canary-tw7tf\" (UID: \"66623dd6-501f-42b2-b12a-188cf5520290\") " pod="openshift-ingress-canary/ingress-canary-tw7tf"
Jul 18 01:57:21.951899 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.951878    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlcm8\" (UniqueName: \"kubernetes.io/projected/c9fa15c4-1db0-4b36-999d-5507f4cb2d19-kube-api-access-dlcm8\") pod \"dns-default-kj28b\" (UID: \"c9fa15c4-1db0-4b36-999d-5507f4cb2d19\") " pod="openshift-dns/dns-default-kj28b"
Jul 18 01:57:21.952095 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.952073    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82t5j\" (UniqueName: \"kubernetes.io/projected/66623dd6-501f-42b2-b12a-188cf5520290-kube-api-access-82t5j\") pod \"ingress-canary-tw7tf\" (UID: \"66623dd6-501f-42b2-b12a-188cf5520290\") " pod="openshift-ingress-canary/ingress-canary-tw7tf"
Jul 18 01:57:21.952205 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.952184    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbt55\" (UniqueName: \"kubernetes.io/projected/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-kube-api-access-vbt55\") pod \"image-registry-76cf94c9f7-87dhh\" (UID: \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\") " pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh"
Jul 18 01:57:21.952585 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.952570    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-bound-sa-token\") pod \"image-registry-76cf94c9f7-87dhh\" (UID: \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\") " pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh"
Jul 18 01:57:21.963185 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.963160    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qgrtk"
Jul 18 01:57:21.963285 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.963163    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jnffd"
Jul 18 01:57:21.963285 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.963166    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6pxz"
Jul 18 01:57:21.965865 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.965709    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-k5xbv\""
Jul 18 01:57:21.965865 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.965720    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 18 01:57:21.965865 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.965730    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zr2dl\""
Jul 18 01:57:21.965865 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.965751    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 18 01:57:21.965865 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.965870    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 18 01:57:21.966166 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:21.966045    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 18 01:57:22.056729 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:22.056532    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tw7tf"
Jul 18 01:57:22.446597 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:22.446560    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c9fa15c4-1db0-4b36-999d-5507f4cb2d19-metrics-tls\") pod \"dns-default-kj28b\" (UID: \"c9fa15c4-1db0-4b36-999d-5507f4cb2d19\") " pod="openshift-dns/dns-default-kj28b"
Jul 18 01:57:22.446771 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:22.446604    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-registry-tls\") pod \"image-registry-76cf94c9f7-87dhh\" (UID: \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\") " pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh"
Jul 18 01:57:22.446771 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:22.446696    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 01:57:22.446771 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:22.446726    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 01:57:22.446771 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:22.446740    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76cf94c9f7-87dhh: secret "image-registry-tls" not found
Jul 18 01:57:22.446945 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:22.446778    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9fa15c4-1db0-4b36-999d-5507f4cb2d19-metrics-tls podName:c9fa15c4-1db0-4b36-999d-5507f4cb2d19 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:23.44675706 +0000 UTC m=+38.071793720 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c9fa15c4-1db0-4b36-999d-5507f4cb2d19-metrics-tls") pod "dns-default-kj28b" (UID: "c9fa15c4-1db0-4b36-999d-5507f4cb2d19") : secret "dns-default-metrics-tls" not found
Jul 18 01:57:22.446945 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:22.446794    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-registry-tls podName:f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:23.44678875 +0000 UTC m=+38.071825397 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-registry-tls") pod "image-registry-76cf94c9f7-87dhh" (UID: "f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09") : secret "image-registry-tls" not found
Jul 18 01:57:23.445024 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:23.444999    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tw7tf"]
Jul 18 01:57:23.448452 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:57:23.448414    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66623dd6_501f_42b2_b12a_188cf5520290.slice/crio-e75598f510f109d9d1100eba11de536df801ce00b173ca9a4dcede0a75816ac6 WatchSource:0}: Error finding container e75598f510f109d9d1100eba11de536df801ce00b173ca9a4dcede0a75816ac6: Status 404 returned error can't find the container with id e75598f510f109d9d1100eba11de536df801ce00b173ca9a4dcede0a75816ac6
Jul 18 01:57:23.453958 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:23.453935    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-registry-tls\") pod \"image-registry-76cf94c9f7-87dhh\" (UID: \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\") " pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh"
Jul 18 01:57:23.454036 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:23.454021    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c9fa15c4-1db0-4b36-999d-5507f4cb2d19-metrics-tls\") pod \"dns-default-kj28b\" (UID: \"c9fa15c4-1db0-4b36-999d-5507f4cb2d19\") " pod="openshift-dns/dns-default-kj28b"
Jul 18 01:57:23.454134 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:23.454099    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 01:57:23.454134 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:23.454109    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 01:57:23.454134 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:23.454117    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76cf94c9f7-87dhh: secret "image-registry-tls" not found
Jul 18 01:57:23.454239 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:23.454152    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9fa15c4-1db0-4b36-999d-5507f4cb2d19-metrics-tls podName:c9fa15c4-1db0-4b36-999d-5507f4cb2d19 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:25.454139458 +0000 UTC m=+40.079176106 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c9fa15c4-1db0-4b36-999d-5507f4cb2d19-metrics-tls") pod "dns-default-kj28b" (UID: "c9fa15c4-1db0-4b36-999d-5507f4cb2d19") : secret "dns-default-metrics-tls" not found
Jul 18 01:57:23.454239 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:23.454164    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-registry-tls podName:f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:25.454158329 +0000 UTC m=+40.079194974 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-registry-tls") pod "image-registry-76cf94c9f7-87dhh" (UID: "f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09") : secret "image-registry-tls" not found
Jul 18 01:57:24.232368 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.232323    2576 generic.go:356] "Generic (PLEG): container finished" podID="cff575d5-ef70-4142-8450-b15c8038b6f8" containerID="2f47e8d2458f33019124989e4866f467467b88126e29be2913d483603c27df21" exitCode=0
Jul 18 01:57:24.232544 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.232397    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sd77t" event={"ID":"cff575d5-ef70-4142-8450-b15c8038b6f8","Type":"ContainerDied","Data":"2f47e8d2458f33019124989e4866f467467b88126e29be2913d483603c27df21"}
Jul 18 01:57:24.233788 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.233758    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tw7tf" event={"ID":"66623dd6-501f-42b2-b12a-188cf5520290","Type":"ContainerStarted","Data":"e75598f510f109d9d1100eba11de536df801ce00b173ca9a4dcede0a75816ac6"}
Jul 18 01:57:24.699247 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.699211    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-smbjg"]
Jul 18 01:57:24.702489 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.702460    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-smbjg"
Jul 18 01:57:24.705632 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.705589    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-g6ghg"]
Jul 18 01:57:24.706312 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.706088    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 18 01:57:24.706312 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.706139    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 18 01:57:24.706312 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.706095    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 18 01:57:24.706606 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.706374    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-qzmdb\""
Jul 18 01:57:24.706606 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.706474    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 18 01:57:24.708739 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.708718    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qgdh7"]
Jul 18 01:57:24.708855 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.708836    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-g6ghg"
Jul 18 01:57:24.710919 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.710899    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 18 01:57:24.711031 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.710935    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 18 01:57:24.711031 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.710937    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-jkbcn\""
Jul 18 01:57:24.711031 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.710949    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 18 01:57:24.711698 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.711678    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-zhjgp"]
Jul 18 01:57:24.711822 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.711808    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qgdh7"
Jul 18 01:57:24.714666 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.714641    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-xk5l2"]
Jul 18 01:57:24.714794 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.714777    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-zhjgp"
Jul 18 01:57:24.714938 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.714909    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 18 01:57:24.715449 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.715321    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-9kdm6\""
Jul 18 01:57:24.715541 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.715507    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 18 01:57:24.717739 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.717718    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 18 01:57:24.717834 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.717817    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 18 01:57:24.717887 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.717850    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 18 01:57:24.718350 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.718333    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-g6fw2\""
Jul 18 01:57:24.718786 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.718766    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-smbjg"]
Jul 18 01:57:24.718881 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.718868    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-xk5l2"
Jul 18 01:57:24.720800 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.720572    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 18 01:57:24.720970 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.720950    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-g6ghg"]
Jul 18 01:57:24.722359 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.721838    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-bscfr\""
Jul 18 01:57:24.723487 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.723468    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-zhjgp"]
Jul 18 01:57:24.724588 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.724565    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-xk5l2"]
Jul 18 01:57:24.734874 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.734856    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qgdh7"]
Jul 18 01:57:24.764631 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.764601    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k96vc\" (UniqueName: \"kubernetes.io/projected/73fe74bc-8fc3-4907-874e-5b83af50d2a1-kube-api-access-k96vc\") pod \"service-ca-operator-5bcbc94c78-smbjg\" (UID: \"73fe74bc-8fc3-4907-874e-5b83af50d2a1\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-smbjg"
Jul 18 01:57:24.764758 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.764659    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whhdq\" (UniqueName: \"kubernetes.io/projected/a40d1613-67f5-4603-a488-4895ddbe78f0-kube-api-access-whhdq\") pod \"volume-data-source-validator-78cf4c6f6-qgdh7\" (UID: \"a40d1613-67f5-4603-a488-4895ddbe78f0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qgdh7"
Jul 18 01:57:24.764758 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.764700    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvrnh\" (UniqueName: \"kubernetes.io/projected/eaae66f5-7295-483b-a1d3-9077f4e9c82b-kube-api-access-mvrnh\") pod \"cluster-monitoring-operator-76cf8966f8-zhjgp\" (UID: \"eaae66f5-7295-483b-a1d3-9077f4e9c82b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-zhjgp"
Jul 18 01:57:24.764758 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.764746    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/73fe74bc-8fc3-4907-874e-5b83af50d2a1-serving-cert\") pod \"service-ca-operator-5bcbc94c78-smbjg\" (UID: \"73fe74bc-8fc3-4907-874e-5b83af50d2a1\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-smbjg"
Jul 18 01:57:24.764857 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.764782    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/eaae66f5-7295-483b-a1d3-9077f4e9c82b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-zhjgp\" (UID: \"eaae66f5-7295-483b-a1d3-9077f4e9c82b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-zhjgp"
Jul 18 01:57:24.764857 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.764804    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/73fe74bc-8fc3-4907-874e-5b83af50d2a1-config\") pod \"service-ca-operator-5bcbc94c78-smbjg\" (UID: \"73fe74bc-8fc3-4907-874e-5b83af50d2a1\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-smbjg"
Jul 18 01:57:24.764857 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.764823    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbcz8\" (UniqueName: \"kubernetes.io/projected/9f46f015-6114-42c0-8bf2-0fa0d0e2e778-kube-api-access-bbcz8\") pod \"cluster-samples-operator-79b7c55f87-g6ghg\" (UID: \"9f46f015-6114-42c0-8bf2-0fa0d0e2e778\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-g6ghg"
Jul 18 01:57:24.764857 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.764852    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/eaae66f5-7295-483b-a1d3-9077f4e9c82b-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-zhjgp\" (UID: \"eaae66f5-7295-483b-a1d3-9077f4e9c82b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-zhjgp"
Jul 18 01:57:24.764972 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.764873    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f46f015-6114-42c0-8bf2-0fa0d0e2e778-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-g6ghg\" (UID: \"9f46f015-6114-42c0-8bf2-0fa0d0e2e778\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-g6ghg"
Jul 18 01:57:24.799667 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.799635    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-69sww"]
Jul 18 01:57:24.804498 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.804469    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-69sww"
Jul 18 01:57:24.805141 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.805118    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5db757d4dd-mt2r5"]
Jul 18 01:57:24.806883 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.806861    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 18 01:57:24.807125 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.807104    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 18 01:57:24.807549 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.807529    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 18 01:57:24.807645 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.807573    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-xqc4f\""
Jul 18 01:57:24.807645 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.807573    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 18 01:57:24.808368 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.808347    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5db757d4dd-mt2r5"
Jul 18 01:57:24.810223 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.810201    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 18 01:57:24.810817 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.810793    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 18 01:57:24.811217 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.810910    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 18 01:57:24.811217 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.810952    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-qvh4j\""
Jul 18 01:57:24.811217 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.810954    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 18 01:57:24.811217 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.811024    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 18 01:57:24.811610 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.811469    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 18 01:57:24.812645 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.812623    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-69sww"]
Jul 18 01:57:24.812987 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.812967    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 18 01:57:24.827509 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.827487    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5db757d4dd-mt2r5"]
Jul 18 01:57:24.865199 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.865165    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/eaae66f5-7295-483b-a1d3-9077f4e9c82b-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-zhjgp\" (UID: \"eaae66f5-7295-483b-a1d3-9077f4e9c82b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-zhjgp"
Jul 18 01:57:24.865199 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.865199    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f46f015-6114-42c0-8bf2-0fa0d0e2e778-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-g6ghg\" (UID: \"9f46f015-6114-42c0-8bf2-0fa0d0e2e778\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-g6ghg"
Jul 18 01:57:24.865459 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.865297    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k96vc\" (UniqueName: \"kubernetes.io/projected/73fe74bc-8fc3-4907-874e-5b83af50d2a1-kube-api-access-k96vc\") pod \"service-ca-operator-5bcbc94c78-smbjg\" (UID: \"73fe74bc-8fc3-4907-874e-5b83af50d2a1\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-smbjg"
Jul 18 01:57:24.865459 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:24.865320    2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 18 01:57:24.865459 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.865354    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-metrics-certs\") pod \"router-default-5db757d4dd-mt2r5\" (UID: \"dc92e8cf-d868-4c6f-963b-04ff7caf51aa\") " pod="openshift-ingress/router-default-5db757d4dd-mt2r5"
Jul 18 01:57:24.865459 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.865386    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/dcf31fe7-21f9-4a41-9b37-2f8440a7c439-snapshots\") pod \"insights-operator-555d9d7885-69sww\" (UID: \"dcf31fe7-21f9-4a41-9b37-2f8440a7c439\") " pod="openshift-insights/insights-operator-555d9d7885-69sww"
Jul 18 01:57:24.865459 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.865417    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dcf31fe7-21f9-4a41-9b37-2f8440a7c439-tmp\") pod \"insights-operator-555d9d7885-69sww\" (UID: \"dcf31fe7-21f9-4a41-9b37-2f8440a7c439\") " pod="openshift-insights/insights-operator-555d9d7885-69sww"
Jul 18 01:57:24.865459 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:24.865449    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f46f015-6114-42c0-8bf2-0fa0d0e2e778-samples-operator-tls podName:9f46f015-6114-42c0-8bf2-0fa0d0e2e778 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:25.365416252 +0000 UTC m=+39.990452914 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9f46f015-6114-42c0-8bf2-0fa0d0e2e778-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-g6ghg" (UID: "9f46f015-6114-42c0-8bf2-0fa0d0e2e778") : secret "samples-operator-tls" not found
Jul 18 01:57:24.865780 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.865524    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvrnh\" (UniqueName: \"kubernetes.io/projected/eaae66f5-7295-483b-a1d3-9077f4e9c82b-kube-api-access-mvrnh\") pod \"cluster-monitoring-operator-76cf8966f8-zhjgp\" (UID: \"eaae66f5-7295-483b-a1d3-9077f4e9c82b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-zhjgp"
Jul 18 01:57:24.865780 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.865562    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-default-certificate\") pod \"router-default-5db757d4dd-mt2r5\" (UID: \"dc92e8cf-d868-4c6f-963b-04ff7caf51aa\") " pod="openshift-ingress/router-default-5db757d4dd-mt2r5"
Jul 18 01:57:24.865780 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.865581    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlhg9\" (UniqueName: \"kubernetes.io/projected/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-kube-api-access-zlhg9\") pod \"router-default-5db757d4dd-mt2r5\" (UID: \"dc92e8cf-d868-4c6f-963b-04ff7caf51aa\") " pod="openshift-ingress/router-default-5db757d4dd-mt2r5"
Jul 18 01:57:24.865780 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.865743    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-stats-auth\") pod \"router-default-5db757d4dd-mt2r5\" (UID: \"dc92e8cf-d868-4c6f-963b-04ff7caf51aa\") " pod="openshift-ingress/router-default-5db757d4dd-mt2r5"
Jul 18 01:57:24.865980 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.865792    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbcz8\" (UniqueName: \"kubernetes.io/projected/9f46f015-6114-42c0-8bf2-0fa0d0e2e778-kube-api-access-bbcz8\") pod \"cluster-samples-operator-79b7c55f87-g6ghg\" (UID: \"9f46f015-6114-42c0-8bf2-0fa0d0e2e778\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-g6ghg"
Jul 18 01:57:24.865980 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.865828    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whhdq\" (UniqueName: \"kubernetes.io/projected/a40d1613-67f5-4603-a488-4895ddbe78f0-kube-api-access-whhdq\") pod \"volume-data-source-validator-78cf4c6f6-qgdh7\" (UID: \"a40d1613-67f5-4603-a488-4895ddbe78f0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qgdh7"
Jul 18 01:57:24.866078 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.866023    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/73fe74bc-8fc3-4907-874e-5b83af50d2a1-serving-cert\") pod \"service-ca-operator-5bcbc94c78-smbjg\" (UID: \"73fe74bc-8fc3-4907-874e-5b83af50d2a1\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-smbjg"
Jul 18 01:57:24.866127 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.866080    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dcf31fe7-21f9-4a41-9b37-2f8440a7c439-serving-cert\") pod \"insights-operator-555d9d7885-69sww\" (UID: \"dcf31fe7-21f9-4a41-9b37-2f8440a7c439\") " pod="openshift-insights/insights-operator-555d9d7885-69sww"
Jul 18 01:57:24.866231 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.866133    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dcf31fe7-21f9-4a41-9b37-2f8440a7c439-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-69sww\" (UID: \"dcf31fe7-21f9-4a41-9b37-2f8440a7c439\") " pod="openshift-insights/insights-operator-555d9d7885-69sww"
Jul 18 01:57:24.866231 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.866198    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jh5cf\" (UniqueName: \"kubernetes.io/projected/e2ae0fc9-8dc6-4440-9b8e-1cbb96ab4c98-kube-api-access-jh5cf\") pod \"network-check-source-7bc5b6bf85-xk5l2\" (UID: \"e2ae0fc9-8dc6-4440-9b8e-1cbb96ab4c98\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-xk5l2"
Jul 18 01:57:24.866331 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.866239    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnk59\" (UniqueName: \"kubernetes.io/projected/dcf31fe7-21f9-4a41-9b37-2f8440a7c439-kube-api-access-qnk59\") pod \"insights-operator-555d9d7885-69sww\" (UID: \"dcf31fe7-21f9-4a41-9b37-2f8440a7c439\") " pod="openshift-insights/insights-operator-555d9d7885-69sww"
Jul 18 01:57:24.866331 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.866276    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/eaae66f5-7295-483b-a1d3-9077f4e9c82b-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-zhjgp\" (UID: \"eaae66f5-7295-483b-a1d3-9077f4e9c82b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-zhjgp"
Jul 18 01:57:24.866442 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.866359    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dcf31fe7-21f9-4a41-9b37-2f8440a7c439-service-ca-bundle\") pod \"insights-operator-555d9d7885-69sww\" (UID: \"dcf31fe7-21f9-4a41-9b37-2f8440a7c439\") " pod="openshift-insights/insights-operator-555d9d7885-69sww"
Jul 18 01:57:24.866505 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.866453    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/eaae66f5-7295-483b-a1d3-9077f4e9c82b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-zhjgp\" (UID: \"eaae66f5-7295-483b-a1d3-9077f4e9c82b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-zhjgp"
Jul 18 01:57:24.866505 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.866492    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/73fe74bc-8fc3-4907-874e-5b83af50d2a1-config\") pod \"service-ca-operator-5bcbc94c78-smbjg\" (UID: \"73fe74bc-8fc3-4907-874e-5b83af50d2a1\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-smbjg"
Jul 18 01:57:24.866604 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.866518    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-service-ca-bundle\") pod \"router-default-5db757d4dd-mt2r5\" (UID: \"dc92e8cf-d868-4c6f-963b-04ff7caf51aa\") " pod="openshift-ingress/router-default-5db757d4dd-mt2r5"
Jul 18 01:57:24.866604 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:24.866585    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 18 01:57:24.866679 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:24.866643    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eaae66f5-7295-483b-a1d3-9077f4e9c82b-cluster-monitoring-operator-tls podName:eaae66f5-7295-483b-a1d3-9077f4e9c82b nodeName:}" failed. No retries permitted until 2026-07-18 01:57:25.366628108 +0000 UTC m=+39.991664764 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/eaae66f5-7295-483b-a1d3-9077f4e9c82b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-zhjgp" (UID: "eaae66f5-7295-483b-a1d3-9077f4e9c82b") : secret "cluster-monitoring-operator-tls" not found
Jul 18 01:57:24.866971 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.866948    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/73fe74bc-8fc3-4907-874e-5b83af50d2a1-config\") pod \"service-ca-operator-5bcbc94c78-smbjg\" (UID: \"73fe74bc-8fc3-4907-874e-5b83af50d2a1\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-smbjg"
Jul 18 01:57:24.869555 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.869531    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/73fe74bc-8fc3-4907-874e-5b83af50d2a1-serving-cert\") pod \"service-ca-operator-5bcbc94c78-smbjg\" (UID: \"73fe74bc-8fc3-4907-874e-5b83af50d2a1\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-smbjg"
Jul 18 01:57:24.874279 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.874257    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k96vc\" (UniqueName: \"kubernetes.io/projected/73fe74bc-8fc3-4907-874e-5b83af50d2a1-kube-api-access-k96vc\") pod \"service-ca-operator-5bcbc94c78-smbjg\" (UID: \"73fe74bc-8fc3-4907-874e-5b83af50d2a1\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-smbjg"
Jul 18 01:57:24.874753 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.874711    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvrnh\" (UniqueName: \"kubernetes.io/projected/eaae66f5-7295-483b-a1d3-9077f4e9c82b-kube-api-access-mvrnh\") pod \"cluster-monitoring-operator-76cf8966f8-zhjgp\" (UID: \"eaae66f5-7295-483b-a1d3-9077f4e9c82b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-zhjgp"
Jul 18 01:57:24.875107 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.875087    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whhdq\" (UniqueName: \"kubernetes.io/projected/a40d1613-67f5-4603-a488-4895ddbe78f0-kube-api-access-whhdq\") pod \"volume-data-source-validator-78cf4c6f6-qgdh7\" (UID: \"a40d1613-67f5-4603-a488-4895ddbe78f0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qgdh7"
Jul 18 01:57:24.875385 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.875360    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbcz8\" (UniqueName: \"kubernetes.io/projected/9f46f015-6114-42c0-8bf2-0fa0d0e2e778-kube-api-access-bbcz8\") pod \"cluster-samples-operator-79b7c55f87-g6ghg\" (UID: \"9f46f015-6114-42c0-8bf2-0fa0d0e2e778\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-g6ghg"
Jul 18 01:57:24.900255 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.900228    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-rxgl4"]
Jul 18 01:57:24.904940 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.904920    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rxgl4"
Jul 18 01:57:24.906342 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.906323    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-bfrdg"]
Jul 18 01:57:24.907332 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.907314    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-hfwkq\""
Jul 18 01:57:24.907571 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.907558    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 18 01:57:24.907892 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.907876    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 18 01:57:24.909253 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.909236    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-tbs7d"]
Jul 18 01:57:24.909366 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.909352    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-bfrdg"
Jul 18 01:57:24.912061 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.912042    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-tc9wq\""
Jul 18 01:57:24.913982 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.913892    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 18 01:57:24.914137 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.914098    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 18 01:57:24.914398 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.914373    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 18 01:57:24.914606 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.914496    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 18 01:57:24.914773 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.914656    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-tbs7d"
Jul 18 01:57:24.919604 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.919566    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-rxgl4"]
Jul 18 01:57:24.920959 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.920940    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 18 01:57:24.921600 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.920987    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 18 01:57:24.921812 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.921125    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-jbg6n\""
Jul 18 01:57:24.922047 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.921149    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 18 01:57:24.922105 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.921340    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 18 01:57:24.924332 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.924310    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 18 01:57:24.924891 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.924872    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-bfrdg"]
Jul 18 01:57:24.928228 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.928208    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-tbs7d"]
Jul 18 01:57:24.967959 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.967881    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8j84q\" (UniqueName: \"kubernetes.io/projected/28a9ce44-0900-4265-a0ee-67acdcfde240-kube-api-access-8j84q\") pod \"kube-storage-version-migrator-operator-7658d9b749-tbs7d\" (UID: \"28a9ce44-0900-4265-a0ee-67acdcfde240\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-tbs7d"
Jul 18 01:57:24.967959 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.967930    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dcf31fe7-21f9-4a41-9b37-2f8440a7c439-serving-cert\") pod \"insights-operator-555d9d7885-69sww\" (UID: \"dcf31fe7-21f9-4a41-9b37-2f8440a7c439\") " pod="openshift-insights/insights-operator-555d9d7885-69sww"
Jul 18 01:57:24.967959 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.967949    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/28a9ce44-0900-4265-a0ee-67acdcfde240-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-tbs7d\" (UID: \"28a9ce44-0900-4265-a0ee-67acdcfde240\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-tbs7d"
Jul 18 01:57:24.968227 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.967978    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dcf31fe7-21f9-4a41-9b37-2f8440a7c439-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-69sww\" (UID: \"dcf31fe7-21f9-4a41-9b37-2f8440a7c439\") " pod="openshift-insights/insights-operator-555d9d7885-69sww"
Jul 18 01:57:24.968227 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.967997    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dv9h8\" (UniqueName: \"kubernetes.io/projected/9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777-kube-api-access-dv9h8\") pod \"console-operator-565b7b569d-bfrdg\" (UID: \"9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777\") " pod="openshift-console-operator/console-operator-565b7b569d-bfrdg"
Jul 18 01:57:24.968227 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.968015    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jh5cf\" (UniqueName: \"kubernetes.io/projected/e2ae0fc9-8dc6-4440-9b8e-1cbb96ab4c98-kube-api-access-jh5cf\") pod \"network-check-source-7bc5b6bf85-xk5l2\" (UID: \"e2ae0fc9-8dc6-4440-9b8e-1cbb96ab4c98\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-xk5l2"
Jul 18 01:57:24.968227 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.968031    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnk59\" (UniqueName: \"kubernetes.io/projected/dcf31fe7-21f9-4a41-9b37-2f8440a7c439-kube-api-access-qnk59\") pod \"insights-operator-555d9d7885-69sww\" (UID: \"dcf31fe7-21f9-4a41-9b37-2f8440a7c439\") " pod="openshift-insights/insights-operator-555d9d7885-69sww"
Jul 18 01:57:24.968227 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.968054    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dcf31fe7-21f9-4a41-9b37-2f8440a7c439-service-ca-bundle\") pod \"insights-operator-555d9d7885-69sww\" (UID: \"dcf31fe7-21f9-4a41-9b37-2f8440a7c439\") " pod="openshift-insights/insights-operator-555d9d7885-69sww"
Jul 18 01:57:24.968227 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.968083    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777-serving-cert\") pod \"console-operator-565b7b569d-bfrdg\" (UID: \"9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777\") " pod="openshift-console-operator/console-operator-565b7b569d-bfrdg"
Jul 18 01:57:24.968227 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.968222    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-service-ca-bundle\") pod \"router-default-5db757d4dd-mt2r5\" (UID: \"dc92e8cf-d868-4c6f-963b-04ff7caf51aa\") " pod="openshift-ingress/router-default-5db757d4dd-mt2r5"
Jul 18 01:57:24.968617 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.968285    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/aa61e8ac-7c4a-4625-a8c8-97854f4fe565-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-rxgl4\" (UID: \"aa61e8ac-7c4a-4625-a8c8-97854f4fe565\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rxgl4"
Jul 18 01:57:24.968617 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.968317    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/28a9ce44-0900-4265-a0ee-67acdcfde240-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-tbs7d\" (UID: \"28a9ce44-0900-4265-a0ee-67acdcfde240\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-tbs7d"
Jul 18 01:57:24.968617 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.968381    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777-config\") pod \"console-operator-565b7b569d-bfrdg\" (UID: \"9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777\") " pod="openshift-console-operator/console-operator-565b7b569d-bfrdg"
Jul 18 01:57:24.968617 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.968418    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-metrics-certs\") pod \"router-default-5db757d4dd-mt2r5\" (UID: \"dc92e8cf-d868-4c6f-963b-04ff7caf51aa\") " pod="openshift-ingress/router-default-5db757d4dd-mt2r5"
Jul 18 01:57:24.968617 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:24.968455    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-service-ca-bundle podName:dc92e8cf-d868-4c6f-963b-04ff7caf51aa nodeName:}" failed. No retries permitted until 2026-07-18 01:57:25.468416103 +0000 UTC m=+40.093452767 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-service-ca-bundle") pod "router-default-5db757d4dd-mt2r5" (UID: "dc92e8cf-d868-4c6f-963b-04ff7caf51aa") : configmap references non-existent config key: service-ca.crt
Jul 18 01:57:24.968617 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.968495    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/dcf31fe7-21f9-4a41-9b37-2f8440a7c439-snapshots\") pod \"insights-operator-555d9d7885-69sww\" (UID: \"dcf31fe7-21f9-4a41-9b37-2f8440a7c439\") " pod="openshift-insights/insights-operator-555d9d7885-69sww"
Jul 18 01:57:24.968617 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.968527    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dcf31fe7-21f9-4a41-9b37-2f8440a7c439-tmp\") pod \"insights-operator-555d9d7885-69sww\" (UID: \"dcf31fe7-21f9-4a41-9b37-2f8440a7c439\") " pod="openshift-insights/insights-operator-555d9d7885-69sww"
Jul 18 01:57:24.968617 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:24.968538    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 18 01:57:24.968617 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.968573    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/aa61e8ac-7c4a-4625-a8c8-97854f4fe565-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-rxgl4\" (UID: \"aa61e8ac-7c4a-4625-a8c8-97854f4fe565\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rxgl4"
Jul 18 01:57:24.968617 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:24.968578    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-metrics-certs podName:dc92e8cf-d868-4c6f-963b-04ff7caf51aa nodeName:}" failed. No retries permitted until 2026-07-18 01:57:25.468566911 +0000 UTC m=+40.093603569 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-metrics-certs") pod "router-default-5db757d4dd-mt2r5" (UID: "dc92e8cf-d868-4c6f-963b-04ff7caf51aa") : secret "router-metrics-certs-default" not found
Jul 18 01:57:24.968617 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.968611    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-default-certificate\") pod \"router-default-5db757d4dd-mt2r5\" (UID: \"dc92e8cf-d868-4c6f-963b-04ff7caf51aa\") " pod="openshift-ingress/router-default-5db757d4dd-mt2r5"
Jul 18 01:57:24.969168 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.968640    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlhg9\" (UniqueName: \"kubernetes.io/projected/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-kube-api-access-zlhg9\") pod \"router-default-5db757d4dd-mt2r5\" (UID: \"dc92e8cf-d868-4c6f-963b-04ff7caf51aa\") " pod="openshift-ingress/router-default-5db757d4dd-mt2r5"
Jul 18 01:57:24.969168 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.968671    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-stats-auth\") pod \"router-default-5db757d4dd-mt2r5\" (UID: \"dc92e8cf-d868-4c6f-963b-04ff7caf51aa\") " pod="openshift-ingress/router-default-5db757d4dd-mt2r5"
Jul 18 01:57:24.969168 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.968698    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777-trusted-ca\") pod \"console-operator-565b7b569d-bfrdg\" (UID: \"9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777\") " pod="openshift-console-operator/console-operator-565b7b569d-bfrdg"
Jul 18 01:57:24.969168 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.968769    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dcf31fe7-21f9-4a41-9b37-2f8440a7c439-service-ca-bundle\") pod \"insights-operator-555d9d7885-69sww\" (UID: \"dcf31fe7-21f9-4a41-9b37-2f8440a7c439\") " pod="openshift-insights/insights-operator-555d9d7885-69sww"
Jul 18 01:57:24.969168 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.968848    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dcf31fe7-21f9-4a41-9b37-2f8440a7c439-tmp\") pod \"insights-operator-555d9d7885-69sww\" (UID: \"dcf31fe7-21f9-4a41-9b37-2f8440a7c439\") " pod="openshift-insights/insights-operator-555d9d7885-69sww"
Jul 18 01:57:24.969168 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.969097    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dcf31fe7-21f9-4a41-9b37-2f8440a7c439-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-69sww\" (UID: \"dcf31fe7-21f9-4a41-9b37-2f8440a7c439\") " pod="openshift-insights/insights-operator-555d9d7885-69sww"
Jul 18 01:57:24.969463 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.969310    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/dcf31fe7-21f9-4a41-9b37-2f8440a7c439-snapshots\") pod \"insights-operator-555d9d7885-69sww\" (UID: \"dcf31fe7-21f9-4a41-9b37-2f8440a7c439\") " pod="openshift-insights/insights-operator-555d9d7885-69sww"
Jul 18 01:57:24.970575 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.970551    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dcf31fe7-21f9-4a41-9b37-2f8440a7c439-serving-cert\") pod \"insights-operator-555d9d7885-69sww\" (UID: \"dcf31fe7-21f9-4a41-9b37-2f8440a7c439\") " pod="openshift-insights/insights-operator-555d9d7885-69sww"
Jul 18 01:57:24.970996 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.970977    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-default-certificate\") pod \"router-default-5db757d4dd-mt2r5\" (UID: \"dc92e8cf-d868-4c6f-963b-04ff7caf51aa\") " pod="openshift-ingress/router-default-5db757d4dd-mt2r5"
Jul 18 01:57:24.971169 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.971139    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-stats-auth\") pod \"router-default-5db757d4dd-mt2r5\" (UID: \"dc92e8cf-d868-4c6f-963b-04ff7caf51aa\") " pod="openshift-ingress/router-default-5db757d4dd-mt2r5"
Jul 18 01:57:24.980193 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.980164    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jh5cf\" (UniqueName: \"kubernetes.io/projected/e2ae0fc9-8dc6-4440-9b8e-1cbb96ab4c98-kube-api-access-jh5cf\") pod \"network-check-source-7bc5b6bf85-xk5l2\" (UID: \"e2ae0fc9-8dc6-4440-9b8e-1cbb96ab4c98\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-xk5l2"
Jul 18 01:57:24.980801 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.980772    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlhg9\" (UniqueName: \"kubernetes.io/projected/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-kube-api-access-zlhg9\") pod \"router-default-5db757d4dd-mt2r5\" (UID: \"dc92e8cf-d868-4c6f-963b-04ff7caf51aa\") " pod="openshift-ingress/router-default-5db757d4dd-mt2r5"
Jul 18 01:57:24.980928 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:24.980887    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnk59\" (UniqueName: \"kubernetes.io/projected/dcf31fe7-21f9-4a41-9b37-2f8440a7c439-kube-api-access-qnk59\") pod \"insights-operator-555d9d7885-69sww\" (UID: \"dcf31fe7-21f9-4a41-9b37-2f8440a7c439\") " pod="openshift-insights/insights-operator-555d9d7885-69sww"
Jul 18 01:57:25.016894 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:25.016856    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-smbjg"
Jul 18 01:57:25.032822 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:25.032787    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qgdh7"
Jul 18 01:57:25.046175 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:25.046147    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-xk5l2"
Jul 18 01:57:25.069763 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:25.069724    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/aa61e8ac-7c4a-4625-a8c8-97854f4fe565-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-rxgl4\" (UID: \"aa61e8ac-7c4a-4625-a8c8-97854f4fe565\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rxgl4"
Jul 18 01:57:25.069919 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:25.069770    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/28a9ce44-0900-4265-a0ee-67acdcfde240-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-tbs7d\" (UID: \"28a9ce44-0900-4265-a0ee-67acdcfde240\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-tbs7d"
Jul 18 01:57:25.069999 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:25.069922    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 18 01:57:25.069999 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:25.069938    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777-config\") pod \"console-operator-565b7b569d-bfrdg\" (UID: \"9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777\") " pod="openshift-console-operator/console-operator-565b7b569d-bfrdg"
Jul 18 01:57:25.070090 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:25.070014    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/aa61e8ac-7c4a-4625-a8c8-97854f4fe565-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-rxgl4\" (UID: \"aa61e8ac-7c4a-4625-a8c8-97854f4fe565\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rxgl4"
Jul 18 01:57:25.070090 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:25.070052    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777-trusted-ca\") pod \"console-operator-565b7b569d-bfrdg\" (UID: \"9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777\") " pod="openshift-console-operator/console-operator-565b7b569d-bfrdg"
Jul 18 01:57:25.070197 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:25.070113    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa61e8ac-7c4a-4625-a8c8-97854f4fe565-networking-console-plugin-cert podName:aa61e8ac-7c4a-4625-a8c8-97854f4fe565 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:25.570081777 +0000 UTC m=+40.195118435 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/aa61e8ac-7c4a-4625-a8c8-97854f4fe565-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-rxgl4" (UID: "aa61e8ac-7c4a-4625-a8c8-97854f4fe565") : secret "networking-console-plugin-cert" not found
Jul 18 01:57:25.070271 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:25.070192    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8j84q\" (UniqueName: \"kubernetes.io/projected/28a9ce44-0900-4265-a0ee-67acdcfde240-kube-api-access-8j84q\") pod \"kube-storage-version-migrator-operator-7658d9b749-tbs7d\" (UID: \"28a9ce44-0900-4265-a0ee-67acdcfde240\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-tbs7d"
Jul 18 01:57:25.070271 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:25.070244    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/28a9ce44-0900-4265-a0ee-67acdcfde240-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-tbs7d\" (UID: \"28a9ce44-0900-4265-a0ee-67acdcfde240\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-tbs7d"
Jul 18 01:57:25.070369 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:25.070291    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dv9h8\" (UniqueName: \"kubernetes.io/projected/9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777-kube-api-access-dv9h8\") pod \"console-operator-565b7b569d-bfrdg\" (UID: \"9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777\") " pod="openshift-console-operator/console-operator-565b7b569d-bfrdg"
Jul 18 01:57:25.070486 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:25.070464    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777-serving-cert\") pod \"console-operator-565b7b569d-bfrdg\" (UID: \"9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777\") " pod="openshift-console-operator/console-operator-565b7b569d-bfrdg"
Jul 18 01:57:25.070776 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:25.070721    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777-config\") pod \"console-operator-565b7b569d-bfrdg\" (UID: \"9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777\") " pod="openshift-console-operator/console-operator-565b7b569d-bfrdg"
Jul 18 01:57:25.070875 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:25.070813    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/aa61e8ac-7c4a-4625-a8c8-97854f4fe565-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-rxgl4\" (UID: \"aa61e8ac-7c4a-4625-a8c8-97854f4fe565\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rxgl4"
Jul 18 01:57:25.070933 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:25.070872    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/28a9ce44-0900-4265-a0ee-67acdcfde240-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-tbs7d\" (UID: \"28a9ce44-0900-4265-a0ee-67acdcfde240\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-tbs7d"
Jul 18 01:57:25.070990 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:25.070965    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777-trusted-ca\") pod \"console-operator-565b7b569d-bfrdg\" (UID: \"9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777\") " pod="openshift-console-operator/console-operator-565b7b569d-bfrdg"
Jul 18 01:57:25.072482 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:25.072464    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/28a9ce44-0900-4265-a0ee-67acdcfde240-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-tbs7d\" (UID: \"28a9ce44-0900-4265-a0ee-67acdcfde240\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-tbs7d"
Jul 18 01:57:25.073441 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:25.073409    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777-serving-cert\") pod \"console-operator-565b7b569d-bfrdg\" (UID: \"9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777\") " pod="openshift-console-operator/console-operator-565b7b569d-bfrdg"
Jul 18 01:57:25.082976 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:25.082955    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8j84q\" (UniqueName: \"kubernetes.io/projected/28a9ce44-0900-4265-a0ee-67acdcfde240-kube-api-access-8j84q\") pod \"kube-storage-version-migrator-operator-7658d9b749-tbs7d\" (UID: \"28a9ce44-0900-4265-a0ee-67acdcfde240\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-tbs7d"
Jul 18 01:57:25.083875 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:25.083848    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dv9h8\" (UniqueName: \"kubernetes.io/projected/9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777-kube-api-access-dv9h8\") pod \"console-operator-565b7b569d-bfrdg\" (UID: \"9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777\") " pod="openshift-console-operator/console-operator-565b7b569d-bfrdg"
Jul 18 01:57:25.117750 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:25.117707    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-69sww"
Jul 18 01:57:25.229206 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:25.228979    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-bfrdg"
Jul 18 01:57:25.235537 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:25.235510    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-tbs7d"
Jul 18 01:57:25.239643 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:25.239617    2576 generic.go:356] "Generic (PLEG): container finished" podID="cff575d5-ef70-4142-8450-b15c8038b6f8" containerID="bf1b22505426ebf5ba879379c6ddb120170fae2ec7087350282fe96446390322" exitCode=0
Jul 18 01:57:25.239765 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:25.239664    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sd77t" event={"ID":"cff575d5-ef70-4142-8450-b15c8038b6f8","Type":"ContainerDied","Data":"bf1b22505426ebf5ba879379c6ddb120170fae2ec7087350282fe96446390322"}
Jul 18 01:57:25.373416 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:25.373384    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/eaae66f5-7295-483b-a1d3-9077f4e9c82b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-zhjgp\" (UID: \"eaae66f5-7295-483b-a1d3-9077f4e9c82b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-zhjgp"
Jul 18 01:57:25.373616 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:25.373479    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f46f015-6114-42c0-8bf2-0fa0d0e2e778-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-g6ghg\" (UID: \"9f46f015-6114-42c0-8bf2-0fa0d0e2e778\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-g6ghg"
Jul 18 01:57:25.373616 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:25.373555    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 18 01:57:25.373728 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:25.373637    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eaae66f5-7295-483b-a1d3-9077f4e9c82b-cluster-monitoring-operator-tls podName:eaae66f5-7295-483b-a1d3-9077f4e9c82b nodeName:}" failed. No retries permitted until 2026-07-18 01:57:26.373616925 +0000 UTC m=+40.998653571 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/eaae66f5-7295-483b-a1d3-9077f4e9c82b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-zhjgp" (UID: "eaae66f5-7295-483b-a1d3-9077f4e9c82b") : secret "cluster-monitoring-operator-tls" not found
Jul 18 01:57:25.374040 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:25.374018    2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 18 01:57:25.374182 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:25.374070    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f46f015-6114-42c0-8bf2-0fa0d0e2e778-samples-operator-tls podName:9f46f015-6114-42c0-8bf2-0fa0d0e2e778 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:26.374055344 +0000 UTC m=+40.999092000 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9f46f015-6114-42c0-8bf2-0fa0d0e2e778-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-g6ghg" (UID: "9f46f015-6114-42c0-8bf2-0fa0d0e2e778") : secret "samples-operator-tls" not found
Jul 18 01:57:25.474821 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:25.474782    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c9fa15c4-1db0-4b36-999d-5507f4cb2d19-metrics-tls\") pod \"dns-default-kj28b\" (UID: \"c9fa15c4-1db0-4b36-999d-5507f4cb2d19\") " pod="openshift-dns/dns-default-kj28b"
Jul 18 01:57:25.475009 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:25.474860    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-service-ca-bundle\") pod \"router-default-5db757d4dd-mt2r5\" (UID: \"dc92e8cf-d868-4c6f-963b-04ff7caf51aa\") " pod="openshift-ingress/router-default-5db757d4dd-mt2r5"
Jul 18 01:57:25.475009 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:25.474945    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 01:57:25.475009 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:25.474967    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-metrics-certs\") pod \"router-default-5db757d4dd-mt2r5\" (UID: \"dc92e8cf-d868-4c6f-963b-04ff7caf51aa\") " pod="openshift-ingress/router-default-5db757d4dd-mt2r5"
Jul 18 01:57:25.475009 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:25.475005    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-registry-tls\") pod \"image-registry-76cf94c9f7-87dhh\" (UID: \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\") " pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh"
Jul 18 01:57:25.475148 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:25.475021    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9fa15c4-1db0-4b36-999d-5507f4cb2d19-metrics-tls podName:c9fa15c4-1db0-4b36-999d-5507f4cb2d19 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:29.475001826 +0000 UTC m=+44.100038491 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c9fa15c4-1db0-4b36-999d-5507f4cb2d19-metrics-tls") pod "dns-default-kj28b" (UID: "c9fa15c4-1db0-4b36-999d-5507f4cb2d19") : secret "dns-default-metrics-tls" not found
Jul 18 01:57:25.475148 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:25.475071    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-service-ca-bundle podName:dc92e8cf-d868-4c6f-963b-04ff7caf51aa nodeName:}" failed. No retries permitted until 2026-07-18 01:57:26.475042002 +0000 UTC m=+41.100078659 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-service-ca-bundle") pod "router-default-5db757d4dd-mt2r5" (UID: "dc92e8cf-d868-4c6f-963b-04ff7caf51aa") : configmap references non-existent config key: service-ca.crt
Jul 18 01:57:25.475148 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:25.475126    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 18 01:57:25.475148 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:25.475075    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 01:57:25.475295 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:25.475161    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76cf94c9f7-87dhh: secret "image-registry-tls" not found
Jul 18 01:57:25.475295 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:25.475162    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-metrics-certs podName:dc92e8cf-d868-4c6f-963b-04ff7caf51aa nodeName:}" failed. No retries permitted until 2026-07-18 01:57:26.475150656 +0000 UTC m=+41.100187302 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-metrics-certs") pod "router-default-5db757d4dd-mt2r5" (UID: "dc92e8cf-d868-4c6f-963b-04ff7caf51aa") : secret "router-metrics-certs-default" not found
Jul 18 01:57:25.475295 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:25.475213    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-registry-tls podName:f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:29.475201028 +0000 UTC m=+44.100237681 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-registry-tls") pod "image-registry-76cf94c9f7-87dhh" (UID: "f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09") : secret "image-registry-tls" not found
Jul 18 01:57:25.576132 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:25.575998    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/aa61e8ac-7c4a-4625-a8c8-97854f4fe565-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-rxgl4\" (UID: \"aa61e8ac-7c4a-4625-a8c8-97854f4fe565\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rxgl4"
Jul 18 01:57:25.576304 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:25.576158    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 18 01:57:25.576304 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:25.576238    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa61e8ac-7c4a-4625-a8c8-97854f4fe565-networking-console-plugin-cert podName:aa61e8ac-7c4a-4625-a8c8-97854f4fe565 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:26.576217768 +0000 UTC m=+41.201254413 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/aa61e8ac-7c4a-4625-a8c8-97854f4fe565-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-rxgl4" (UID: "aa61e8ac-7c4a-4625-a8c8-97854f4fe565") : secret "networking-console-plugin-cert" not found
Jul 18 01:57:25.972339 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:25.972312    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-69sww"]
Jul 18 01:57:26.093517 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:57:26.093483    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcf31fe7_21f9_4a41_9b37_2f8440a7c439.slice/crio-d7a4a05a5f9186dac5a9af1ca7b341f60a79f05c367091b7782e223c56c5cafc WatchSource:0}: Error finding container d7a4a05a5f9186dac5a9af1ca7b341f60a79f05c367091b7782e223c56c5cafc: Status 404 returned error can't find the container with id d7a4a05a5f9186dac5a9af1ca7b341f60a79f05c367091b7782e223c56c5cafc
Jul 18 01:57:26.179339 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:26.179225    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-smbjg"]
Jul 18 01:57:26.182935 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:26.182911    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-xk5l2"]
Jul 18 01:57:26.183476 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:57:26.183452    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73fe74bc_8fc3_4907_874e_5b83af50d2a1.slice/crio-fcee99bf42796048cca4c34c3b69a375b10c78c017582b8684c216ce142b1cc6 WatchSource:0}: Error finding container fcee99bf42796048cca4c34c3b69a375b10c78c017582b8684c216ce142b1cc6: Status 404 returned error can't find the container with id fcee99bf42796048cca4c34c3b69a375b10c78c017582b8684c216ce142b1cc6
Jul 18 01:57:26.185977 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:26.185949    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-bfrdg"]
Jul 18 01:57:26.186096 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:57:26.186065    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2ae0fc9_8dc6_4440_9b8e_1cbb96ab4c98.slice/crio-c0ebe2815c6f4a21e118d97b28876ed29208a12a7ecd094cae0dad7c1c42d8de WatchSource:0}: Error finding container c0ebe2815c6f4a21e118d97b28876ed29208a12a7ecd094cae0dad7c1c42d8de: Status 404 returned error can't find the container with id c0ebe2815c6f4a21e118d97b28876ed29208a12a7ecd094cae0dad7c1c42d8de
Jul 18 01:57:26.189168 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:57:26.189146    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ae8cf0a_bdff_4e54_b5b8_a9e64e2d0777.slice/crio-9f474f9e33e32ddc8ba0e42642cbb9033f24135f8a13a42599ee21fcff4cb52d WatchSource:0}: Error finding container 9f474f9e33e32ddc8ba0e42642cbb9033f24135f8a13a42599ee21fcff4cb52d: Status 404 returned error can't find the container with id 9f474f9e33e32ddc8ba0e42642cbb9033f24135f8a13a42599ee21fcff4cb52d
Jul 18 01:57:26.221398 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:26.221377    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qgdh7"]
Jul 18 01:57:26.223555 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:57:26.223520    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda40d1613_67f5_4603_a488_4895ddbe78f0.slice/crio-f041c1cacbd8e2c6a4be5a5d9078307d77f688b306a276d3b328bf646f176ce0 WatchSource:0}: Error finding container f041c1cacbd8e2c6a4be5a5d9078307d77f688b306a276d3b328bf646f176ce0: Status 404 returned error can't find the container with id f041c1cacbd8e2c6a4be5a5d9078307d77f688b306a276d3b328bf646f176ce0
Jul 18 01:57:26.224603 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:26.224576    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-tbs7d"]
Jul 18 01:57:26.227194 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:57:26.227175    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28a9ce44_0900_4265_a0ee_67acdcfde240.slice/crio-e6d11610cd37633b8189c794cc4cb88e90b4b01d1a7a7a361a2e17e2c2d6b0a7 WatchSource:0}: Error finding container e6d11610cd37633b8189c794cc4cb88e90b4b01d1a7a7a361a2e17e2c2d6b0a7: Status 404 returned error can't find the container with id e6d11610cd37633b8189c794cc4cb88e90b4b01d1a7a7a361a2e17e2c2d6b0a7
Jul 18 01:57:26.245942 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:26.245918    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sd77t" event={"ID":"cff575d5-ef70-4142-8450-b15c8038b6f8","Type":"ContainerStarted","Data":"f0495b6bccd11944c0389eede09a3381a2ba90901031c92e2bc836c1cf571c95"}
Jul 18 01:57:26.247090 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:26.247069    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qgdh7" event={"ID":"a40d1613-67f5-4603-a488-4895ddbe78f0","Type":"ContainerStarted","Data":"f041c1cacbd8e2c6a4be5a5d9078307d77f688b306a276d3b328bf646f176ce0"}
Jul 18 01:57:26.248251 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:26.248231    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-smbjg" event={"ID":"73fe74bc-8fc3-4907-874e-5b83af50d2a1","Type":"ContainerStarted","Data":"fcee99bf42796048cca4c34c3b69a375b10c78c017582b8684c216ce142b1cc6"}
Jul 18 01:57:26.249242 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:26.249221    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-69sww" event={"ID":"dcf31fe7-21f9-4a41-9b37-2f8440a7c439","Type":"ContainerStarted","Data":"d7a4a05a5f9186dac5a9af1ca7b341f60a79f05c367091b7782e223c56c5cafc"}
Jul 18 01:57:26.250393 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:26.250374    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-xk5l2" event={"ID":"e2ae0fc9-8dc6-4440-9b8e-1cbb96ab4c98","Type":"ContainerStarted","Data":"c0ebe2815c6f4a21e118d97b28876ed29208a12a7ecd094cae0dad7c1c42d8de"}
Jul 18 01:57:26.251385 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:26.251367    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-tbs7d" event={"ID":"28a9ce44-0900-4265-a0ee-67acdcfde240","Type":"ContainerStarted","Data":"e6d11610cd37633b8189c794cc4cb88e90b4b01d1a7a7a361a2e17e2c2d6b0a7"}
Jul 18 01:57:26.253693 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:26.253676    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-bfrdg" event={"ID":"9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777","Type":"ContainerStarted","Data":"9f474f9e33e32ddc8ba0e42642cbb9033f24135f8a13a42599ee21fcff4cb52d"}
Jul 18 01:57:26.268822 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:26.268771    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-sd77t" podStartSLOduration=9.441310953 podStartE2EDuration="40.268752685s" totalImagesPullingTime="30.827441732s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:56:46 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:26.267042196 +0000 UTC m=+40.892078863" watchObservedRunningTime="2026-07-18 01:57:26.268752685 +0000 UTC m=+40.893789352"
Jul 18 01:57:26.385537 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:26.385496    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/eaae66f5-7295-483b-a1d3-9077f4e9c82b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-zhjgp\" (UID: \"eaae66f5-7295-483b-a1d3-9077f4e9c82b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-zhjgp"
Jul 18 01:57:26.385726 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:26.385592    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f46f015-6114-42c0-8bf2-0fa0d0e2e778-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-g6ghg\" (UID: \"9f46f015-6114-42c0-8bf2-0fa0d0e2e778\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-g6ghg"
Jul 18 01:57:26.385726 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:26.385598    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 18 01:57:26.385726 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:26.385688    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eaae66f5-7295-483b-a1d3-9077f4e9c82b-cluster-monitoring-operator-tls podName:eaae66f5-7295-483b-a1d3-9077f4e9c82b nodeName:}" failed. No retries permitted until 2026-07-18 01:57:28.385666391 +0000 UTC m=+43.010703060 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/eaae66f5-7295-483b-a1d3-9077f4e9c82b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-zhjgp" (UID: "eaae66f5-7295-483b-a1d3-9077f4e9c82b") : secret "cluster-monitoring-operator-tls" not found
Jul 18 01:57:26.385726 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:26.385712    2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 18 01:57:26.385942 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:26.385768    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f46f015-6114-42c0-8bf2-0fa0d0e2e778-samples-operator-tls podName:9f46f015-6114-42c0-8bf2-0fa0d0e2e778 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:28.385750362 +0000 UTC m=+43.010787020 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9f46f015-6114-42c0-8bf2-0fa0d0e2e778-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-g6ghg" (UID: "9f46f015-6114-42c0-8bf2-0fa0d0e2e778") : secret "samples-operator-tls" not found
Jul 18 01:57:26.486239 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:26.486207    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-service-ca-bundle\") pod \"router-default-5db757d4dd-mt2r5\" (UID: \"dc92e8cf-d868-4c6f-963b-04ff7caf51aa\") " pod="openshift-ingress/router-default-5db757d4dd-mt2r5"
Jul 18 01:57:26.486416 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:26.486368    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-metrics-certs\") pod \"router-default-5db757d4dd-mt2r5\" (UID: \"dc92e8cf-d868-4c6f-963b-04ff7caf51aa\") " pod="openshift-ingress/router-default-5db757d4dd-mt2r5"
Jul 18 01:57:26.486526 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:26.486456    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-service-ca-bundle podName:dc92e8cf-d868-4c6f-963b-04ff7caf51aa nodeName:}" failed. No retries permitted until 2026-07-18 01:57:28.48639389 +0000 UTC m=+43.111430549 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-service-ca-bundle") pod "router-default-5db757d4dd-mt2r5" (UID: "dc92e8cf-d868-4c6f-963b-04ff7caf51aa") : configmap references non-existent config key: service-ca.crt
Jul 18 01:57:26.486526 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:26.486486    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 18 01:57:26.486617 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:26.486536    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-metrics-certs podName:dc92e8cf-d868-4c6f-963b-04ff7caf51aa nodeName:}" failed. No retries permitted until 2026-07-18 01:57:28.486522006 +0000 UTC m=+43.111558655 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-metrics-certs") pod "router-default-5db757d4dd-mt2r5" (UID: "dc92e8cf-d868-4c6f-963b-04ff7caf51aa") : secret "router-metrics-certs-default" not found
Jul 18 01:57:26.587192 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:26.587156    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/aa61e8ac-7c4a-4625-a8c8-97854f4fe565-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-rxgl4\" (UID: \"aa61e8ac-7c4a-4625-a8c8-97854f4fe565\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rxgl4"
Jul 18 01:57:26.587364 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:26.587343    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 18 01:57:26.587468 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:26.587459    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa61e8ac-7c4a-4625-a8c8-97854f4fe565-networking-console-plugin-cert podName:aa61e8ac-7c4a-4625-a8c8-97854f4fe565 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:28.587417204 +0000 UTC m=+43.212453853 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/aa61e8ac-7c4a-4625-a8c8-97854f4fe565-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-rxgl4" (UID: "aa61e8ac-7c4a-4625-a8c8-97854f4fe565") : secret "networking-console-plugin-cert" not found
Jul 18 01:57:27.261679 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:27.261554    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tw7tf" event={"ID":"66623dd6-501f-42b2-b12a-188cf5520290","Type":"ContainerStarted","Data":"0f7e01e2de70139bd3d468b78e330e41dc0817958c5e646116d177b22e7b6d65"}
Jul 18 01:57:27.278269 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:27.278193    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-tw7tf" podStartSLOduration=3.606872044 podStartE2EDuration="6.278169131s" totalImagesPullingTime="2.671297087s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:57:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:27.276475396 +0000 UTC m=+41.901512064" watchObservedRunningTime="2026-07-18 01:57:27.278169131 +0000 UTC m=+41.903205799"
Jul 18 01:57:28.406655 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:28.406617    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/eaae66f5-7295-483b-a1d3-9077f4e9c82b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-zhjgp\" (UID: \"eaae66f5-7295-483b-a1d3-9077f4e9c82b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-zhjgp"
Jul 18 01:57:28.407182 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:28.406696    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f46f015-6114-42c0-8bf2-0fa0d0e2e778-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-g6ghg\" (UID: \"9f46f015-6114-42c0-8bf2-0fa0d0e2e778\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-g6ghg"
Jul 18 01:57:28.407182 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:28.406743    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 18 01:57:28.407182 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:28.406801    2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 18 01:57:28.407182 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:28.406822    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eaae66f5-7295-483b-a1d3-9077f4e9c82b-cluster-monitoring-operator-tls podName:eaae66f5-7295-483b-a1d3-9077f4e9c82b nodeName:}" failed. No retries permitted until 2026-07-18 01:57:32.406800743 +0000 UTC m=+47.031837393 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/eaae66f5-7295-483b-a1d3-9077f4e9c82b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-zhjgp" (UID: "eaae66f5-7295-483b-a1d3-9077f4e9c82b") : secret "cluster-monitoring-operator-tls" not found
Jul 18 01:57:28.407182 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:28.406848    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f46f015-6114-42c0-8bf2-0fa0d0e2e778-samples-operator-tls podName:9f46f015-6114-42c0-8bf2-0fa0d0e2e778 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:32.406836659 +0000 UTC m=+47.031873304 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9f46f015-6114-42c0-8bf2-0fa0d0e2e778-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-g6ghg" (UID: "9f46f015-6114-42c0-8bf2-0fa0d0e2e778") : secret "samples-operator-tls" not found
Jul 18 01:57:28.508479 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:28.508189    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-service-ca-bundle\") pod \"router-default-5db757d4dd-mt2r5\" (UID: \"dc92e8cf-d868-4c6f-963b-04ff7caf51aa\") " pod="openshift-ingress/router-default-5db757d4dd-mt2r5"
Jul 18 01:57:28.508479 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:28.508305    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-metrics-certs\") pod \"router-default-5db757d4dd-mt2r5\" (UID: \"dc92e8cf-d868-4c6f-963b-04ff7caf51aa\") " pod="openshift-ingress/router-default-5db757d4dd-mt2r5"
Jul 18 01:57:28.508479 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:28.508330    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-service-ca-bundle podName:dc92e8cf-d868-4c6f-963b-04ff7caf51aa nodeName:}" failed. No retries permitted until 2026-07-18 01:57:32.508311517 +0000 UTC m=+47.133348180 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-service-ca-bundle") pod "router-default-5db757d4dd-mt2r5" (UID: "dc92e8cf-d868-4c6f-963b-04ff7caf51aa") : configmap references non-existent config key: service-ca.crt
Jul 18 01:57:28.508479 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:28.508386    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 18 01:57:28.508479 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:28.508421    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-metrics-certs podName:dc92e8cf-d868-4c6f-963b-04ff7caf51aa nodeName:}" failed. No retries permitted until 2026-07-18 01:57:32.508409355 +0000 UTC m=+47.133446004 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-metrics-certs") pod "router-default-5db757d4dd-mt2r5" (UID: "dc92e8cf-d868-4c6f-963b-04ff7caf51aa") : secret "router-metrics-certs-default" not found
Jul 18 01:57:28.609375 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:28.609334    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/aa61e8ac-7c4a-4625-a8c8-97854f4fe565-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-rxgl4\" (UID: \"aa61e8ac-7c4a-4625-a8c8-97854f4fe565\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rxgl4"
Jul 18 01:57:28.609568 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:28.609533    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 18 01:57:28.609647 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:28.609626    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa61e8ac-7c4a-4625-a8c8-97854f4fe565-networking-console-plugin-cert podName:aa61e8ac-7c4a-4625-a8c8-97854f4fe565 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:32.609596314 +0000 UTC m=+47.234632981 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/aa61e8ac-7c4a-4625-a8c8-97854f4fe565-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-rxgl4" (UID: "aa61e8ac-7c4a-4625-a8c8-97854f4fe565") : secret "networking-console-plugin-cert" not found
Jul 18 01:57:29.518865 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:29.518824    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-registry-tls\") pod \"image-registry-76cf94c9f7-87dhh\" (UID: \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\") " pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh"
Jul 18 01:57:29.519324 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:29.518900    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c9fa15c4-1db0-4b36-999d-5507f4cb2d19-metrics-tls\") pod \"dns-default-kj28b\" (UID: \"c9fa15c4-1db0-4b36-999d-5507f4cb2d19\") " pod="openshift-dns/dns-default-kj28b"
Jul 18 01:57:29.519324 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:29.518959    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 01:57:29.519324 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:29.518985    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76cf94c9f7-87dhh: secret "image-registry-tls" not found
Jul 18 01:57:29.519324 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:29.519040    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 01:57:29.519324 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:29.519058    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-registry-tls podName:f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:37.519037158 +0000 UTC m=+52.144073804 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-registry-tls") pod "image-registry-76cf94c9f7-87dhh" (UID: "f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09") : secret "image-registry-tls" not found
Jul 18 01:57:29.519324 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:29.519079    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9fa15c4-1db0-4b36-999d-5507f4cb2d19-metrics-tls podName:c9fa15c4-1db0-4b36-999d-5507f4cb2d19 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:37.519068003 +0000 UTC m=+52.144104661 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c9fa15c4-1db0-4b36-999d-5507f4cb2d19-metrics-tls") pod "dns-default-kj28b" (UID: "c9fa15c4-1db0-4b36-999d-5507f4cb2d19") : secret "dns-default-metrics-tls" not found
Jul 18 01:57:32.446937 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:32.446897    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/eaae66f5-7295-483b-a1d3-9077f4e9c82b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-zhjgp\" (UID: \"eaae66f5-7295-483b-a1d3-9077f4e9c82b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-zhjgp"
Jul 18 01:57:32.447322 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:32.446949    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f46f015-6114-42c0-8bf2-0fa0d0e2e778-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-g6ghg\" (UID: \"9f46f015-6114-42c0-8bf2-0fa0d0e2e778\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-g6ghg"
Jul 18 01:57:32.447322 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:32.447051    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 18 01:57:32.447322 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:32.447071    2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 18 01:57:32.447322 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:32.447120    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eaae66f5-7295-483b-a1d3-9077f4e9c82b-cluster-monitoring-operator-tls podName:eaae66f5-7295-483b-a1d3-9077f4e9c82b nodeName:}" failed. No retries permitted until 2026-07-18 01:57:40.447104757 +0000 UTC m=+55.072141402 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/eaae66f5-7295-483b-a1d3-9077f4e9c82b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-zhjgp" (UID: "eaae66f5-7295-483b-a1d3-9077f4e9c82b") : secret "cluster-monitoring-operator-tls" not found
Jul 18 01:57:32.447322 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:32.447135    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f46f015-6114-42c0-8bf2-0fa0d0e2e778-samples-operator-tls podName:9f46f015-6114-42c0-8bf2-0fa0d0e2e778 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:40.447129595 +0000 UTC m=+55.072166241 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9f46f015-6114-42c0-8bf2-0fa0d0e2e778-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-g6ghg" (UID: "9f46f015-6114-42c0-8bf2-0fa0d0e2e778") : secret "samples-operator-tls" not found
Jul 18 01:57:32.547728 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:32.547705    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-service-ca-bundle\") pod \"router-default-5db757d4dd-mt2r5\" (UID: \"dc92e8cf-d868-4c6f-963b-04ff7caf51aa\") " pod="openshift-ingress/router-default-5db757d4dd-mt2r5"
Jul 18 01:57:32.547810 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:32.547785    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-metrics-certs\") pod \"router-default-5db757d4dd-mt2r5\" (UID: \"dc92e8cf-d868-4c6f-963b-04ff7caf51aa\") " pod="openshift-ingress/router-default-5db757d4dd-mt2r5"
Jul 18 01:57:32.547892 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:32.547875    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-service-ca-bundle podName:dc92e8cf-d868-4c6f-963b-04ff7caf51aa nodeName:}" failed. No retries permitted until 2026-07-18 01:57:40.547859019 +0000 UTC m=+55.172895667 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-service-ca-bundle") pod "router-default-5db757d4dd-mt2r5" (UID: "dc92e8cf-d868-4c6f-963b-04ff7caf51aa") : configmap references non-existent config key: service-ca.crt
Jul 18 01:57:32.547934 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:32.547903    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 18 01:57:32.547966 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:32.547949    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-metrics-certs podName:dc92e8cf-d868-4c6f-963b-04ff7caf51aa nodeName:}" failed. No retries permitted until 2026-07-18 01:57:40.547935195 +0000 UTC m=+55.172971841 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-metrics-certs") pod "router-default-5db757d4dd-mt2r5" (UID: "dc92e8cf-d868-4c6f-963b-04ff7caf51aa") : secret "router-metrics-certs-default" not found
Jul 18 01:57:32.649025 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:32.649002    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/aa61e8ac-7c4a-4625-a8c8-97854f4fe565-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-rxgl4\" (UID: \"aa61e8ac-7c4a-4625-a8c8-97854f4fe565\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rxgl4"
Jul 18 01:57:32.649186 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:32.649166    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 18 01:57:32.649259 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:32.649247    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa61e8ac-7c4a-4625-a8c8-97854f4fe565-networking-console-plugin-cert podName:aa61e8ac-7c4a-4625-a8c8-97854f4fe565 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:40.649227107 +0000 UTC m=+55.274263754 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/aa61e8ac-7c4a-4625-a8c8-97854f4fe565-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-rxgl4" (UID: "aa61e8ac-7c4a-4625-a8c8-97854f4fe565") : secret "networking-console-plugin-cert" not found
Jul 18 01:57:33.279036 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:33.278593    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-69sww" event={"ID":"dcf31fe7-21f9-4a41-9b37-2f8440a7c439","Type":"ContainerStarted","Data":"893c0b27bc38b80d8c29ff99eab08b0b00ce1b8a6122386aa9b0bede552ab4ba"}
Jul 18 01:57:33.280479 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:33.280395    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-xk5l2" event={"ID":"e2ae0fc9-8dc6-4440-9b8e-1cbb96ab4c98","Type":"ContainerStarted","Data":"b60f651b6c7d8f6078e27c7fe0715777fccf36d1d2adf9b7f7aaa28020343fcc"}
Jul 18 01:57:33.282162 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:33.282136    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-tbs7d" event={"ID":"28a9ce44-0900-4265-a0ee-67acdcfde240","Type":"ContainerStarted","Data":"a4edb7a7cbbc729e0fa722760901fdd0ac1dedf4bcf86b63a09515a768a73eb6"}
Jul 18 01:57:33.284027 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:33.284005    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-bfrdg_9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777/console-operator/0.log"
Jul 18 01:57:33.284128 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:33.284047    2576 generic.go:356] "Generic (PLEG): container finished" podID="9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777" containerID="b8a8a405c68a90b95487cf6c6b99ac2316c8148ea8ebd3ae226b14771ea801ad" exitCode=255
Jul 18 01:57:33.284183 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:33.284143    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-bfrdg" event={"ID":"9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777","Type":"ContainerDied","Data":"b8a8a405c68a90b95487cf6c6b99ac2316c8148ea8ebd3ae226b14771ea801ad"}
Jul 18 01:57:33.284358 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:33.284337    2576 scope.go:117] "RemoveContainer" containerID="b8a8a405c68a90b95487cf6c6b99ac2316c8148ea8ebd3ae226b14771ea801ad"
Jul 18 01:57:33.286492 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:33.286468    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qgdh7" event={"ID":"a40d1613-67f5-4603-a488-4895ddbe78f0","Type":"ContainerStarted","Data":"963f5da1ab741fdc05c739031a07fa16a847a085d20c765428a198d3a5eff006"}
Jul 18 01:57:33.288214 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:33.288194    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-smbjg" event={"ID":"73fe74bc-8fc3-4907-874e-5b83af50d2a1","Type":"ContainerStarted","Data":"c6dd76ef87ee7d6dd0f515570eea90e7767ed26086f09788afef9483f22434a7"}
Jul 18 01:57:33.294649 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:33.294606    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-69sww" podStartSLOduration=2.803448872 podStartE2EDuration="9.294589649s" totalImagesPullingTime="6.491140777s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:57:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:33.293420709 +0000 UTC m=+47.918457378" watchObservedRunningTime="2026-07-18 01:57:33.294589649 +0000 UTC m=+47.919626316"
Jul 18 01:57:33.308326 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:33.308277    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-tbs7d" podStartSLOduration=2.930937893 podStartE2EDuration="9.308260915s" totalImagesPullingTime="6.377323022s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:57:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:33.307827063 +0000 UTC m=+47.932863730" watchObservedRunningTime="2026-07-18 01:57:33.308260915 +0000 UTC m=+47.933297584"
Jul 18 01:57:33.325652 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:33.324248    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-smbjg" podStartSLOduration=2.898790949 podStartE2EDuration="9.32422987s" totalImagesPullingTime="6.425438921s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:57:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:33.323708398 +0000 UTC m=+47.948745065" watchObservedRunningTime="2026-07-18 01:57:33.32422987 +0000 UTC m=+47.949266536"
Jul 18 01:57:33.372845 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:33.372786    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qgdh7" podStartSLOduration=3.014520073 podStartE2EDuration="9.372763486s" totalImagesPullingTime="6.358243413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:57:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:33.353333103 +0000 UTC m=+47.978369771" watchObservedRunningTime="2026-07-18 01:57:33.372763486 +0000 UTC m=+47.997800154"
Jul 18 01:57:33.398269 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:33.398213    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-xk5l2" podStartSLOduration=2.642421106 podStartE2EDuration="9.39819031s" totalImagesPullingTime="6.755769204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:57:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:33.373277215 +0000 UTC m=+47.998313883" watchObservedRunningTime="2026-07-18 01:57:33.39819031 +0000 UTC m=+48.023226978"
Jul 18 01:57:34.124741 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:34.124700    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-s78gl"]
Jul 18 01:57:34.160302 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:34.160265    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-s78gl"]
Jul 18 01:57:34.160504 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:34.160345    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-s78gl"
Jul 18 01:57:34.162663 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:34.162637    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 18 01:57:34.162663 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:34.162652    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rd42x\""
Jul 18 01:57:34.162857 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:34.162643    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 18 01:57:34.266214 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:34.266180    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c378a52c-b5db-4a92-bd55-8616ad887e45-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s78gl\" (UID: \"c378a52c-b5db-4a92-bd55-8616ad887e45\") " pod="openshift-insights/insights-runtime-extractor-s78gl"
Jul 18 01:57:34.266214 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:34.266224    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26hzk\" (UniqueName: \"kubernetes.io/projected/c378a52c-b5db-4a92-bd55-8616ad887e45-kube-api-access-26hzk\") pod \"insights-runtime-extractor-s78gl\" (UID: \"c378a52c-b5db-4a92-bd55-8616ad887e45\") " pod="openshift-insights/insights-runtime-extractor-s78gl"
Jul 18 01:57:34.266503 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:34.266469    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c378a52c-b5db-4a92-bd55-8616ad887e45-data-volume\") pod \"insights-runtime-extractor-s78gl\" (UID: \"c378a52c-b5db-4a92-bd55-8616ad887e45\") " pod="openshift-insights/insights-runtime-extractor-s78gl"
Jul 18 01:57:34.266575 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:34.266522    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c378a52c-b5db-4a92-bd55-8616ad887e45-crio-socket\") pod \"insights-runtime-extractor-s78gl\" (UID: \"c378a52c-b5db-4a92-bd55-8616ad887e45\") " pod="openshift-insights/insights-runtime-extractor-s78gl"
Jul 18 01:57:34.266575 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:34.266557    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c378a52c-b5db-4a92-bd55-8616ad887e45-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s78gl\" (UID: \"c378a52c-b5db-4a92-bd55-8616ad887e45\") " pod="openshift-insights/insights-runtime-extractor-s78gl"
Jul 18 01:57:34.293511 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:34.293481    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-bfrdg_9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777/console-operator/1.log"
Jul 18 01:57:34.293955 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:34.293936    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-bfrdg_9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777/console-operator/0.log"
Jul 18 01:57:34.294058 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:34.293976    2576 generic.go:356] "Generic (PLEG): container finished" podID="9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777" containerID="91f44aa339cc351ae75d9d32d94a498e8c90a8da7952e5ed7d975610a27fa026" exitCode=255
Jul 18 01:57:34.294121 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:34.294089    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-bfrdg" event={"ID":"9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777","Type":"ContainerDied","Data":"91f44aa339cc351ae75d9d32d94a498e8c90a8da7952e5ed7d975610a27fa026"}
Jul 18 01:57:34.294205 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:34.294191    2576 scope.go:117] "RemoveContainer" containerID="b8a8a405c68a90b95487cf6c6b99ac2316c8148ea8ebd3ae226b14771ea801ad"
Jul 18 01:57:34.294519 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:34.294310    2576 scope.go:117] "RemoveContainer" containerID="91f44aa339cc351ae75d9d32d94a498e8c90a8da7952e5ed7d975610a27fa026"
Jul 18 01:57:34.294519 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:34.294504    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-bfrdg_openshift-console-operator(9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777)\"" pod="openshift-console-operator/console-operator-565b7b569d-bfrdg" podUID="9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777"
Jul 18 01:57:34.368452 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:34.367710    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c378a52c-b5db-4a92-bd55-8616ad887e45-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s78gl\" (UID: \"c378a52c-b5db-4a92-bd55-8616ad887e45\") " pod="openshift-insights/insights-runtime-extractor-s78gl"
Jul 18 01:57:34.368452 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:34.367763    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26hzk\" (UniqueName: \"kubernetes.io/projected/c378a52c-b5db-4a92-bd55-8616ad887e45-kube-api-access-26hzk\") pod \"insights-runtime-extractor-s78gl\" (UID: \"c378a52c-b5db-4a92-bd55-8616ad887e45\") " pod="openshift-insights/insights-runtime-extractor-s78gl"
Jul 18 01:57:34.368452 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:34.367904    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c378a52c-b5db-4a92-bd55-8616ad887e45-data-volume\") pod \"insights-runtime-extractor-s78gl\" (UID: \"c378a52c-b5db-4a92-bd55-8616ad887e45\") " pod="openshift-insights/insights-runtime-extractor-s78gl"
Jul 18 01:57:34.368452 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:34.367938    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c378a52c-b5db-4a92-bd55-8616ad887e45-crio-socket\") pod \"insights-runtime-extractor-s78gl\" (UID: \"c378a52c-b5db-4a92-bd55-8616ad887e45\") " pod="openshift-insights/insights-runtime-extractor-s78gl"
Jul 18 01:57:34.368452 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:34.367963    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c378a52c-b5db-4a92-bd55-8616ad887e45-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s78gl\" (UID: \"c378a52c-b5db-4a92-bd55-8616ad887e45\") " pod="openshift-insights/insights-runtime-extractor-s78gl"
Jul 18 01:57:34.368452 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:34.368102    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 18 01:57:34.368452 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:34.368176    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c378a52c-b5db-4a92-bd55-8616ad887e45-kube-rbac-proxy-cm podName:c378a52c-b5db-4a92-bd55-8616ad887e45 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:34.868157934 +0000 UTC m=+49.493194593 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c378a52c-b5db-4a92-bd55-8616ad887e45-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s78gl" (UID: "c378a52c-b5db-4a92-bd55-8616ad887e45") : configmap "kube-rbac-proxy" not found
Jul 18 01:57:34.368943 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:34.368525    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 18 01:57:34.368943 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:34.368584    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c378a52c-b5db-4a92-bd55-8616ad887e45-crio-socket\") pod \"insights-runtime-extractor-s78gl\" (UID: \"c378a52c-b5db-4a92-bd55-8616ad887e45\") " pod="openshift-insights/insights-runtime-extractor-s78gl"
Jul 18 01:57:34.368943 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:34.368597    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c378a52c-b5db-4a92-bd55-8616ad887e45-insights-runtime-extractor-tls podName:c378a52c-b5db-4a92-bd55-8616ad887e45 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:34.868580087 +0000 UTC m=+49.493616749 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c378a52c-b5db-4a92-bd55-8616ad887e45-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s78gl" (UID: "c378a52c-b5db-4a92-bd55-8616ad887e45") : secret "insights-runtime-extractor-tls" not found
Jul 18 01:57:34.368943 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:34.368739    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c378a52c-b5db-4a92-bd55-8616ad887e45-data-volume\") pod \"insights-runtime-extractor-s78gl\" (UID: \"c378a52c-b5db-4a92-bd55-8616ad887e45\") " pod="openshift-insights/insights-runtime-extractor-s78gl"
Jul 18 01:57:34.390991 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:34.390955    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26hzk\" (UniqueName: \"kubernetes.io/projected/c378a52c-b5db-4a92-bd55-8616ad887e45-kube-api-access-26hzk\") pod \"insights-runtime-extractor-s78gl\" (UID: \"c378a52c-b5db-4a92-bd55-8616ad887e45\") " pod="openshift-insights/insights-runtime-extractor-s78gl"
Jul 18 01:57:34.874063 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:34.873972    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c378a52c-b5db-4a92-bd55-8616ad887e45-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s78gl\" (UID: \"c378a52c-b5db-4a92-bd55-8616ad887e45\") " pod="openshift-insights/insights-runtime-extractor-s78gl"
Jul 18 01:57:34.874223 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:34.874104    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c378a52c-b5db-4a92-bd55-8616ad887e45-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s78gl\" (UID: \"c378a52c-b5db-4a92-bd55-8616ad887e45\") " pod="openshift-insights/insights-runtime-extractor-s78gl"
Jul 18 01:57:34.874223 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:34.874146    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 18 01:57:34.874223 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:34.874176    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 18 01:57:34.874353 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:34.874227    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c378a52c-b5db-4a92-bd55-8616ad887e45-insights-runtime-extractor-tls podName:c378a52c-b5db-4a92-bd55-8616ad887e45 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:35.874211338 +0000 UTC m=+50.499247983 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c378a52c-b5db-4a92-bd55-8616ad887e45-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s78gl" (UID: "c378a52c-b5db-4a92-bd55-8616ad887e45") : secret "insights-runtime-extractor-tls" not found
Jul 18 01:57:34.874353 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:34.874241    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c378a52c-b5db-4a92-bd55-8616ad887e45-kube-rbac-proxy-cm podName:c378a52c-b5db-4a92-bd55-8616ad887e45 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:35.874235016 +0000 UTC m=+50.499271660 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c378a52c-b5db-4a92-bd55-8616ad887e45-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s78gl" (UID: "c378a52c-b5db-4a92-bd55-8616ad887e45") : configmap "kube-rbac-proxy" not found
Jul 18 01:57:35.229718 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:35.229684    2576 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-bfrdg"
Jul 18 01:57:35.230065 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:35.229734    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-bfrdg"
Jul 18 01:57:35.298359 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:35.298331    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-bfrdg_9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777/console-operator/1.log"
Jul 18 01:57:35.298702 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:35.298684    2576 scope.go:117] "RemoveContainer" containerID="91f44aa339cc351ae75d9d32d94a498e8c90a8da7952e5ed7d975610a27fa026"
Jul 18 01:57:35.298868 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:35.298850    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-bfrdg_openshift-console-operator(9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777)\"" pod="openshift-console-operator/console-operator-565b7b569d-bfrdg" podUID="9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777"
Jul 18 01:57:35.523525 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:35.523456    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-25924_a3f34325-5076-4e41-a22d-5caece5dbece/dns-node-resolver/0.log"
Jul 18 01:57:35.882801 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:35.882775    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c378a52c-b5db-4a92-bd55-8616ad887e45-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s78gl\" (UID: \"c378a52c-b5db-4a92-bd55-8616ad887e45\") " pod="openshift-insights/insights-runtime-extractor-s78gl"
Jul 18 01:57:35.882982 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:35.882876    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 18 01:57:35.882982 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:35.882878    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c378a52c-b5db-4a92-bd55-8616ad887e45-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s78gl\" (UID: \"c378a52c-b5db-4a92-bd55-8616ad887e45\") " pod="openshift-insights/insights-runtime-extractor-s78gl"
Jul 18 01:57:35.882982 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:35.882957    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 18 01:57:35.883114 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:35.882959    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c378a52c-b5db-4a92-bd55-8616ad887e45-kube-rbac-proxy-cm podName:c378a52c-b5db-4a92-bd55-8616ad887e45 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:37.882939944 +0000 UTC m=+52.507976593 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c378a52c-b5db-4a92-bd55-8616ad887e45-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s78gl" (UID: "c378a52c-b5db-4a92-bd55-8616ad887e45") : configmap "kube-rbac-proxy" not found
Jul 18 01:57:35.883114 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:35.883033    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c378a52c-b5db-4a92-bd55-8616ad887e45-insights-runtime-extractor-tls podName:c378a52c-b5db-4a92-bd55-8616ad887e45 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:37.883020028 +0000 UTC m=+52.508056678 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c378a52c-b5db-4a92-bd55-8616ad887e45-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s78gl" (UID: "c378a52c-b5db-4a92-bd55-8616ad887e45") : secret "insights-runtime-extractor-tls" not found
Jul 18 01:57:36.301916 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:36.301841    2576 scope.go:117] "RemoveContainer" containerID="91f44aa339cc351ae75d9d32d94a498e8c90a8da7952e5ed7d975610a27fa026"
Jul 18 01:57:36.302240 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:36.301999    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-bfrdg_openshift-console-operator(9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777)\"" pod="openshift-console-operator/console-operator-565b7b569d-bfrdg" podUID="9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777"
Jul 18 01:57:36.522862 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:36.522838    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-f5kmg_33351d5c-d8c8-4d94-8209-28d15797b7a9/node-ca/0.log"
Jul 18 01:57:37.523538 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:37.523509    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tw7tf_66623dd6-501f-42b2-b12a-188cf5520290/serve-healthcheck-canary/0.log"
Jul 18 01:57:37.598693 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:37.598656    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-registry-tls\") pod \"image-registry-76cf94c9f7-87dhh\" (UID: \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\") " pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh"
Jul 18 01:57:37.598856 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:37.598712    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c9fa15c4-1db0-4b36-999d-5507f4cb2d19-metrics-tls\") pod \"dns-default-kj28b\" (UID: \"c9fa15c4-1db0-4b36-999d-5507f4cb2d19\") " pod="openshift-dns/dns-default-kj28b"
Jul 18 01:57:37.598856 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:37.598810    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 01:57:37.598856 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:37.598829    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76cf94c9f7-87dhh: secret "image-registry-tls" not found
Jul 18 01:57:37.598856 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:37.598843    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 01:57:37.598982 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:37.598898    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9fa15c4-1db0-4b36-999d-5507f4cb2d19-metrics-tls podName:c9fa15c4-1db0-4b36-999d-5507f4cb2d19 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:53.598881875 +0000 UTC m=+68.223918522 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c9fa15c4-1db0-4b36-999d-5507f4cb2d19-metrics-tls") pod "dns-default-kj28b" (UID: "c9fa15c4-1db0-4b36-999d-5507f4cb2d19") : secret "dns-default-metrics-tls" not found
Jul 18 01:57:37.598982 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:37.598915    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-registry-tls podName:f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:53.598907738 +0000 UTC m=+68.223944383 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-registry-tls") pod "image-registry-76cf94c9f7-87dhh" (UID: "f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09") : secret "image-registry-tls" not found
Jul 18 01:57:37.901651 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:37.901614    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c378a52c-b5db-4a92-bd55-8616ad887e45-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s78gl\" (UID: \"c378a52c-b5db-4a92-bd55-8616ad887e45\") " pod="openshift-insights/insights-runtime-extractor-s78gl"
Jul 18 01:57:37.901839 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:37.901718    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c378a52c-b5db-4a92-bd55-8616ad887e45-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s78gl\" (UID: \"c378a52c-b5db-4a92-bd55-8616ad887e45\") " pod="openshift-insights/insights-runtime-extractor-s78gl"
Jul 18 01:57:37.901899 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:37.901864    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 18 01:57:37.901941 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:37.901927    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c378a52c-b5db-4a92-bd55-8616ad887e45-insights-runtime-extractor-tls podName:c378a52c-b5db-4a92-bd55-8616ad887e45 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:41.901911125 +0000 UTC m=+56.526947769 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c378a52c-b5db-4a92-bd55-8616ad887e45-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s78gl" (UID: "c378a52c-b5db-4a92-bd55-8616ad887e45") : secret "insights-runtime-extractor-tls" not found
Jul 18 01:57:37.902123 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:37.902105    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c378a52c-b5db-4a92-bd55-8616ad887e45-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s78gl\" (UID: \"c378a52c-b5db-4a92-bd55-8616ad887e45\") " pod="openshift-insights/insights-runtime-extractor-s78gl"
Jul 18 01:57:40.524680 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:40.524640    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/eaae66f5-7295-483b-a1d3-9077f4e9c82b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-zhjgp\" (UID: \"eaae66f5-7295-483b-a1d3-9077f4e9c82b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-zhjgp"
Jul 18 01:57:40.525092 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:40.524699    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f46f015-6114-42c0-8bf2-0fa0d0e2e778-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-g6ghg\" (UID: \"9f46f015-6114-42c0-8bf2-0fa0d0e2e778\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-g6ghg"
Jul 18 01:57:40.525092 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:40.524798    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 18 01:57:40.525092 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:40.524844    2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 18 01:57:40.525092 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:40.524881    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eaae66f5-7295-483b-a1d3-9077f4e9c82b-cluster-monitoring-operator-tls podName:eaae66f5-7295-483b-a1d3-9077f4e9c82b nodeName:}" failed. No retries permitted until 2026-07-18 01:57:56.524859682 +0000 UTC m=+71.149896346 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/eaae66f5-7295-483b-a1d3-9077f4e9c82b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-zhjgp" (UID: "eaae66f5-7295-483b-a1d3-9077f4e9c82b") : secret "cluster-monitoring-operator-tls" not found
Jul 18 01:57:40.525092 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:40.524897    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f46f015-6114-42c0-8bf2-0fa0d0e2e778-samples-operator-tls podName:9f46f015-6114-42c0-8bf2-0fa0d0e2e778 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:56.524889889 +0000 UTC m=+71.149926534 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9f46f015-6114-42c0-8bf2-0fa0d0e2e778-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-g6ghg" (UID: "9f46f015-6114-42c0-8bf2-0fa0d0e2e778") : secret "samples-operator-tls" not found
Jul 18 01:57:40.625758 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:40.625720    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-service-ca-bundle\") pod \"router-default-5db757d4dd-mt2r5\" (UID: \"dc92e8cf-d868-4c6f-963b-04ff7caf51aa\") " pod="openshift-ingress/router-default-5db757d4dd-mt2r5"
Jul 18 01:57:40.625927 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:40.625813    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-metrics-certs\") pod \"router-default-5db757d4dd-mt2r5\" (UID: \"dc92e8cf-d868-4c6f-963b-04ff7caf51aa\") " pod="openshift-ingress/router-default-5db757d4dd-mt2r5"
Jul 18 01:57:40.625927 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:40.625883    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-service-ca-bundle podName:dc92e8cf-d868-4c6f-963b-04ff7caf51aa nodeName:}" failed. No retries permitted until 2026-07-18 01:57:56.625866148 +0000 UTC m=+71.250902793 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-service-ca-bundle") pod "router-default-5db757d4dd-mt2r5" (UID: "dc92e8cf-d868-4c6f-963b-04ff7caf51aa") : configmap references non-existent config key: service-ca.crt
Jul 18 01:57:40.626017 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:40.625951    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 18 01:57:40.626017 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:40.626005    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-metrics-certs podName:dc92e8cf-d868-4c6f-963b-04ff7caf51aa nodeName:}" failed. No retries permitted until 2026-07-18 01:57:56.625992717 +0000 UTC m=+71.251029363 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-metrics-certs") pod "router-default-5db757d4dd-mt2r5" (UID: "dc92e8cf-d868-4c6f-963b-04ff7caf51aa") : secret "router-metrics-certs-default" not found
Jul 18 01:57:40.727171 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:40.727134    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/aa61e8ac-7c4a-4625-a8c8-97854f4fe565-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-rxgl4\" (UID: \"aa61e8ac-7c4a-4625-a8c8-97854f4fe565\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rxgl4"
Jul 18 01:57:40.727356 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:40.727270    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 18 01:57:40.727356 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:40.727335    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa61e8ac-7c4a-4625-a8c8-97854f4fe565-networking-console-plugin-cert podName:aa61e8ac-7c4a-4625-a8c8-97854f4fe565 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:56.727320331 +0000 UTC m=+71.352356976 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/aa61e8ac-7c4a-4625-a8c8-97854f4fe565-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-rxgl4" (UID: "aa61e8ac-7c4a-4625-a8c8-97854f4fe565") : secret "networking-console-plugin-cert" not found
Jul 18 01:57:41.938139 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:41.938086    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c378a52c-b5db-4a92-bd55-8616ad887e45-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s78gl\" (UID: \"c378a52c-b5db-4a92-bd55-8616ad887e45\") " pod="openshift-insights/insights-runtime-extractor-s78gl"
Jul 18 01:57:41.938638 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:41.938254    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 18 01:57:41.938638 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:57:41.938322    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c378a52c-b5db-4a92-bd55-8616ad887e45-insights-runtime-extractor-tls podName:c378a52c-b5db-4a92-bd55-8616ad887e45 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:49.938305074 +0000 UTC m=+64.563341731 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c378a52c-b5db-4a92-bd55-8616ad887e45-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s78gl" (UID: "c378a52c-b5db-4a92-bd55-8616ad887e45") : secret "insights-runtime-extractor-tls" not found
Jul 18 01:57:48.222341 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:48.222312    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jfzth"
Jul 18 01:57:48.963274 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:48.963243    2576 scope.go:117] "RemoveContainer" containerID="91f44aa339cc351ae75d9d32d94a498e8c90a8da7952e5ed7d975610a27fa026"
Jul 18 01:57:49.344353 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:49.344277    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-bfrdg_9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777/console-operator/1.log"
Jul 18 01:57:49.344353 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:49.344346    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-bfrdg" event={"ID":"9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777","Type":"ContainerStarted","Data":"df792049ffd15226c7ba9ed8d58096b23ac58ff82cf0e21138cd219d441cb207"}
Jul 18 01:57:49.344904 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:49.344652    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-bfrdg"
Jul 18 01:57:49.359141 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:49.359090    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-bfrdg" podStartSLOduration=18.944346923 podStartE2EDuration="25.359074066s" totalImagesPullingTime="6.414727143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:57:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:49.358723744 +0000 UTC m=+63.983760407" watchObservedRunningTime="2026-07-18 01:57:49.359074066 +0000 UTC m=+63.984110734"
Jul 18 01:57:49.513725 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:49.513696    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-bfrdg"
Jul 18 01:57:50.012212 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:50.012173    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c378a52c-b5db-4a92-bd55-8616ad887e45-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s78gl\" (UID: \"c378a52c-b5db-4a92-bd55-8616ad887e45\") " pod="openshift-insights/insights-runtime-extractor-s78gl"
Jul 18 01:57:50.014439 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:50.014409    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c378a52c-b5db-4a92-bd55-8616ad887e45-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s78gl\" (UID: \"c378a52c-b5db-4a92-bd55-8616ad887e45\") " pod="openshift-insights/insights-runtime-extractor-s78gl"
Jul 18 01:57:50.071999 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:50.071965    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rd42x\""
Jul 18 01:57:50.080814 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:50.080790    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-s78gl"
Jul 18 01:57:50.203278 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:50.203246    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-s78gl"]
Jul 18 01:57:50.206269 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:57:50.206247    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc378a52c_b5db_4a92_bd55_8616ad887e45.slice/crio-507f6727f3fa8d5ebca51af156910fe0f861c6f3634e1116369e3b1ec821472e WatchSource:0}: Error finding container 507f6727f3fa8d5ebca51af156910fe0f861c6f3634e1116369e3b1ec821472e: Status 404 returned error can't find the container with id 507f6727f3fa8d5ebca51af156910fe0f861c6f3634e1116369e3b1ec821472e
Jul 18 01:57:50.349158 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:50.349074    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s78gl" event={"ID":"c378a52c-b5db-4a92-bd55-8616ad887e45","Type":"ContainerStarted","Data":"507f6727f3fa8d5ebca51af156910fe0f861c6f3634e1116369e3b1ec821472e"}
Jul 18 01:57:52.032999 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:52.032963    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48beaadf-7df1-497b-af20-2b57bb5a4250-metrics-certs\") pod \"network-metrics-daemon-qgrtk\" (UID: \"48beaadf-7df1-497b-af20-2b57bb5a4250\") " pod="openshift-multus/network-metrics-daemon-qgrtk"
Jul 18 01:57:52.033477 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:52.033008    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a370ba82-a51e-41aa-9ced-19bab3186de4-original-pull-secret\") pod \"global-pull-secret-syncer-jnffd\" (UID: \"a370ba82-a51e-41aa-9ced-19bab3186de4\") " pod="kube-system/global-pull-secret-syncer-jnffd"
Jul 18 01:57:52.035765 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:52.035730    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 18 01:57:52.035885 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:52.035767    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 18 01:57:52.046247 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:52.046221    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a370ba82-a51e-41aa-9ced-19bab3186de4-original-pull-secret\") pod \"global-pull-secret-syncer-jnffd\" (UID: \"a370ba82-a51e-41aa-9ced-19bab3186de4\") " pod="kube-system/global-pull-secret-syncer-jnffd"
Jul 18 01:57:52.046410 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:52.046366    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48beaadf-7df1-497b-af20-2b57bb5a4250-metrics-certs\") pod \"network-metrics-daemon-qgrtk\" (UID: \"48beaadf-7df1-497b-af20-2b57bb5a4250\") " pod="openshift-multus/network-metrics-daemon-qgrtk"
Jul 18 01:57:52.133963 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:52.133927    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtjnd\" (UniqueName: \"kubernetes.io/projected/b212c851-6975-4dcc-bd43-9421f78f6c64-kube-api-access-wtjnd\") pod \"network-check-target-h6pxz\" (UID: \"b212c851-6975-4dcc-bd43-9421f78f6c64\") " pod="openshift-network-diagnostics/network-check-target-h6pxz"
Jul 18 01:57:52.136967 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:52.136940    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtjnd\" (UniqueName: \"kubernetes.io/projected/b212c851-6975-4dcc-bd43-9421f78f6c64-kube-api-access-wtjnd\") pod \"network-check-target-h6pxz\" (UID: \"b212c851-6975-4dcc-bd43-9421f78f6c64\") " pod="openshift-network-diagnostics/network-check-target-h6pxz"
Jul 18 01:57:52.277782 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:52.277751    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-k5xbv\""
Jul 18 01:57:52.285482 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:52.285402    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qgrtk"
Jul 18 01:57:52.285647 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:52.285482    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zr2dl\""
Jul 18 01:57:52.288063 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:52.288035    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jnffd"
Jul 18 01:57:52.293487 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:52.293468    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6pxz"
Jul 18 01:57:53.646816 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:53.646761    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-registry-tls\") pod \"image-registry-76cf94c9f7-87dhh\" (UID: \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\") " pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh"
Jul 18 01:57:53.647237 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:53.646855    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c9fa15c4-1db0-4b36-999d-5507f4cb2d19-metrics-tls\") pod \"dns-default-kj28b\" (UID: \"c9fa15c4-1db0-4b36-999d-5507f4cb2d19\") " pod="openshift-dns/dns-default-kj28b"
Jul 18 01:57:53.649386 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:53.649332    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c9fa15c4-1db0-4b36-999d-5507f4cb2d19-metrics-tls\") pod \"dns-default-kj28b\" (UID: \"c9fa15c4-1db0-4b36-999d-5507f4cb2d19\") " pod="openshift-dns/dns-default-kj28b"
Jul 18 01:57:53.649536 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:53.649474    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-registry-tls\") pod \"image-registry-76cf94c9f7-87dhh\" (UID: \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\") " pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh"
Jul 18 01:57:53.833959 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:53.833929    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-q7jhh\""
Jul 18 01:57:53.841930 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:53.841889    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh"
Jul 18 01:57:53.852711 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:53.852682    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-96mg4\""
Jul 18 01:57:53.860593 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:53.860566    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kj28b"
Jul 18 01:57:55.035820 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:55.035695    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76cf94c9f7-87dhh"]
Jul 18 01:57:55.258119 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:57:55.258084    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3a1d9e2_58d8_4f09_bf0c_5dbfa5123b09.slice/crio-84e0854895d0deb6cdffe112858ea39322e292e461ebd693859256a6f95bb0be WatchSource:0}: Error finding container 84e0854895d0deb6cdffe112858ea39322e292e461ebd693859256a6f95bb0be: Status 404 returned error can't find the container with id 84e0854895d0deb6cdffe112858ea39322e292e461ebd693859256a6f95bb0be
Jul 18 01:57:55.371671 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:55.371609    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh" event={"ID":"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09","Type":"ContainerStarted","Data":"84e0854895d0deb6cdffe112858ea39322e292e461ebd693859256a6f95bb0be"}
Jul 18 01:57:55.423180 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:55.423151    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-kj28b"]
Jul 18 01:57:55.427440 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:57:55.427385    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9fa15c4_1db0_4b36_999d_5507f4cb2d19.slice/crio-cff45cfbc829ba212b9dafa03990d4f3bff98a61623168e19865213d273e71a1 WatchSource:0}: Error finding container cff45cfbc829ba212b9dafa03990d4f3bff98a61623168e19865213d273e71a1: Status 404 returned error can't find the container with id cff45cfbc829ba212b9dafa03990d4f3bff98a61623168e19865213d273e71a1
Jul 18 01:57:55.469672 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:55.469640    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jnffd"]
Jul 18 01:57:55.472417 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:57:55.472390    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda370ba82_a51e_41aa_9ced_19bab3186de4.slice/crio-ba5a8656a6a757754f8b6139a2cc4691d4aed4b5ccb3065215cbd69258b2c6e8 WatchSource:0}: Error finding container ba5a8656a6a757754f8b6139a2cc4691d4aed4b5ccb3065215cbd69258b2c6e8: Status 404 returned error can't find the container with id ba5a8656a6a757754f8b6139a2cc4691d4aed4b5ccb3065215cbd69258b2c6e8
Jul 18 01:57:55.636529 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:55.636498    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-h6pxz"]
Jul 18 01:57:55.639074 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:55.639044    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qgrtk"]
Jul 18 01:57:55.640087 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:57:55.640051    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb212c851_6975_4dcc_bd43_9421f78f6c64.slice/crio-3b7507742e557e2af0791e46bd32db6155a9391b44a212186170eca6f58632c3 WatchSource:0}: Error finding container 3b7507742e557e2af0791e46bd32db6155a9391b44a212186170eca6f58632c3: Status 404 returned error can't find the container with id 3b7507742e557e2af0791e46bd32db6155a9391b44a212186170eca6f58632c3
Jul 18 01:57:55.641628 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:57:55.641600    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48beaadf_7df1_497b_af20_2b57bb5a4250.slice/crio-b60dd371edfdc5e5b0888613ceeea1dfbba96ca3a7d57532f773c468a453ca4d WatchSource:0}: Error finding container b60dd371edfdc5e5b0888613ceeea1dfbba96ca3a7d57532f773c468a453ca4d: Status 404 returned error can't find the container with id b60dd371edfdc5e5b0888613ceeea1dfbba96ca3a7d57532f773c468a453ca4d
Jul 18 01:57:56.377687 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:56.377579    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s78gl" event={"ID":"c378a52c-b5db-4a92-bd55-8616ad887e45","Type":"ContainerStarted","Data":"157652b5821203713ed7bbb89bc4d88ff09c0d1a96e7bca468f719da1550a6f8"}
Jul 18 01:57:56.379781 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:56.379748    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh" event={"ID":"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09","Type":"ContainerStarted","Data":"27e3e887f245acb40584515252bed1b8c8102fa6c77f96c187027e444abc1e4d"}
Jul 18 01:57:56.380115 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:56.380097    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh"
Jul 18 01:57:56.381763 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:56.381735    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kj28b" event={"ID":"c9fa15c4-1db0-4b36-999d-5507f4cb2d19","Type":"ContainerStarted","Data":"cff45cfbc829ba212b9dafa03990d4f3bff98a61623168e19865213d273e71a1"}
Jul 18 01:57:56.386260 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:56.386234    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qgrtk" event={"ID":"48beaadf-7df1-497b-af20-2b57bb5a4250","Type":"ContainerStarted","Data":"b60dd371edfdc5e5b0888613ceeea1dfbba96ca3a7d57532f773c468a453ca4d"}
Jul 18 01:57:56.388362 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:56.388331    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jnffd" event={"ID":"a370ba82-a51e-41aa-9ced-19bab3186de4","Type":"ContainerStarted","Data":"ba5a8656a6a757754f8b6139a2cc4691d4aed4b5ccb3065215cbd69258b2c6e8"}
Jul 18 01:57:56.390711 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:56.390686    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-h6pxz" event={"ID":"b212c851-6975-4dcc-bd43-9421f78f6c64","Type":"ContainerStarted","Data":"a9086d028ee29d4da33484c04e50a4fb34da6bd4dd0b99d2b9ea4aef10fb5bca"}
Jul 18 01:57:56.390803 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:56.390733    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-h6pxz" event={"ID":"b212c851-6975-4dcc-bd43-9421f78f6c64","Type":"ContainerStarted","Data":"3b7507742e557e2af0791e46bd32db6155a9391b44a212186170eca6f58632c3"}
Jul 18 01:57:56.390869 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:56.390850    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-h6pxz"
Jul 18 01:57:56.400894 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:56.400844    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh" podStartSLOduration=70.400824735 podStartE2EDuration="1m10.400824735s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:56:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:56.398894332 +0000 UTC m=+71.023930999" watchObservedRunningTime="2026-07-18 01:57:56.400824735 +0000 UTC m=+71.025861403"
Jul 18 01:57:56.421768 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:56.421649    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-h6pxz" podStartSLOduration=70.421627794 podStartE2EDuration="1m10.421627794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:56:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:56.420117496 +0000 UTC m=+71.045154163" watchObservedRunningTime="2026-07-18 01:57:56.421627794 +0000 UTC m=+71.046664462"
Jul 18 01:57:56.573211 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:56.572742    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/eaae66f5-7295-483b-a1d3-9077f4e9c82b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-zhjgp\" (UID: \"eaae66f5-7295-483b-a1d3-9077f4e9c82b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-zhjgp"
Jul 18 01:57:56.573211 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:56.572822    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f46f015-6114-42c0-8bf2-0fa0d0e2e778-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-g6ghg\" (UID: \"9f46f015-6114-42c0-8bf2-0fa0d0e2e778\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-g6ghg"
Jul 18 01:57:56.578523 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:56.578463    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f46f015-6114-42c0-8bf2-0fa0d0e2e778-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-g6ghg\" (UID: \"9f46f015-6114-42c0-8bf2-0fa0d0e2e778\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-g6ghg"
Jul 18 01:57:56.578681 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:56.578583    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/eaae66f5-7295-483b-a1d3-9077f4e9c82b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-zhjgp\" (UID: \"eaae66f5-7295-483b-a1d3-9077f4e9c82b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-zhjgp"
Jul 18 01:57:56.673942 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:56.673898    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-metrics-certs\") pod \"router-default-5db757d4dd-mt2r5\" (UID: \"dc92e8cf-d868-4c6f-963b-04ff7caf51aa\") " pod="openshift-ingress/router-default-5db757d4dd-mt2r5"
Jul 18 01:57:56.674122 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:56.674004    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-service-ca-bundle\") pod \"router-default-5db757d4dd-mt2r5\" (UID: \"dc92e8cf-d868-4c6f-963b-04ff7caf51aa\") " pod="openshift-ingress/router-default-5db757d4dd-mt2r5"
Jul 18 01:57:56.674823 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:56.674797    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-service-ca-bundle\") pod \"router-default-5db757d4dd-mt2r5\" (UID: \"dc92e8cf-d868-4c6f-963b-04ff7caf51aa\") " pod="openshift-ingress/router-default-5db757d4dd-mt2r5"
Jul 18 01:57:56.677666 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:56.677620    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc92e8cf-d868-4c6f-963b-04ff7caf51aa-metrics-certs\") pod \"router-default-5db757d4dd-mt2r5\" (UID: \"dc92e8cf-d868-4c6f-963b-04ff7caf51aa\") " pod="openshift-ingress/router-default-5db757d4dd-mt2r5"
Jul 18 01:57:56.775046 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:56.774665    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/aa61e8ac-7c4a-4625-a8c8-97854f4fe565-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-rxgl4\" (UID: \"aa61e8ac-7c4a-4625-a8c8-97854f4fe565\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rxgl4"
Jul 18 01:57:56.778518 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:56.778465    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/aa61e8ac-7c4a-4625-a8c8-97854f4fe565-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-rxgl4\" (UID: \"aa61e8ac-7c4a-4625-a8c8-97854f4fe565\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rxgl4"
Jul 18 01:57:56.827849 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:56.827816    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-jkbcn\""
Jul 18 01:57:56.836138 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:56.835891    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-g6ghg"
Jul 18 01:57:56.842776 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:56.842584    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-g6fw2\""
Jul 18 01:57:56.851560 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:56.851385    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-zhjgp"
Jul 18 01:57:56.924571 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:56.924463    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-qvh4j\""
Jul 18 01:57:56.934377 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:56.933972    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5db757d4dd-mt2r5"
Jul 18 01:57:57.022223 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:57.021974    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-hfwkq\""
Jul 18 01:57:57.029725 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:57.029479    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rxgl4"
Jul 18 01:57:57.047380 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:57.047258    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-g6ghg"]
Jul 18 01:57:57.082626 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:57.082541    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-zhjgp"]
Jul 18 01:57:57.090705 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:57:57.090310    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaae66f5_7295_483b_a1d3_9077f4e9c82b.slice/crio-d3c1277e65e9877979509ee622c6ce3dd7a8706d88bc90bf235123414e94892a WatchSource:0}: Error finding container d3c1277e65e9877979509ee622c6ce3dd7a8706d88bc90bf235123414e94892a: Status 404 returned error can't find the container with id d3c1277e65e9877979509ee622c6ce3dd7a8706d88bc90bf235123414e94892a
Jul 18 01:57:57.153497 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:57.153121    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5db757d4dd-mt2r5"]
Jul 18 01:57:57.158850 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:57:57.158810    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc92e8cf_d868_4c6f_963b_04ff7caf51aa.slice/crio-804c772eba6e16cc0d9d85649886774412d22a83cbb57c5a79830dad85ca2ee3 WatchSource:0}: Error finding container 804c772eba6e16cc0d9d85649886774412d22a83cbb57c5a79830dad85ca2ee3: Status 404 returned error can't find the container with id 804c772eba6e16cc0d9d85649886774412d22a83cbb57c5a79830dad85ca2ee3
Jul 18 01:57:57.231479 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:57.231126    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-rxgl4"]
Jul 18 01:57:57.399216 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:57.398517    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qgrtk" event={"ID":"48beaadf-7df1-497b-af20-2b57bb5a4250","Type":"ContainerStarted","Data":"cae045037172c9638c73691ada8fdf8f6a944d8718fa21a958a3f62d8f90439c"}
Jul 18 01:57:57.399216 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:57.398577    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qgrtk" event={"ID":"48beaadf-7df1-497b-af20-2b57bb5a4250","Type":"ContainerStarted","Data":"3f4a352dbc9e8932901d98a96a8303d6e999425a7c050d4a833acc51b5b7f89d"}
Jul 18 01:57:57.401301 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:57.401234    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-zhjgp" event={"ID":"eaae66f5-7295-483b-a1d3-9077f4e9c82b","Type":"ContainerStarted","Data":"d3c1277e65e9877979509ee622c6ce3dd7a8706d88bc90bf235123414e94892a"}
Jul 18 01:57:57.403958 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:57.403875    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-g6ghg" event={"ID":"9f46f015-6114-42c0-8bf2-0fa0d0e2e778","Type":"ContainerStarted","Data":"dec0c46532cde7e53404f4485be4e5e8f75ae33f5cec41f8d43174809f673787"}
Jul 18 01:57:57.406623 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:57.405896    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5db757d4dd-mt2r5" event={"ID":"dc92e8cf-d868-4c6f-963b-04ff7caf51aa","Type":"ContainerStarted","Data":"4e89dab620d89ad2fd3e39138e2af782815c10f65cce9ff28006ce7feb142839"}
Jul 18 01:57:57.406623 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:57.406577    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5db757d4dd-mt2r5" event={"ID":"dc92e8cf-d868-4c6f-963b-04ff7caf51aa","Type":"ContainerStarted","Data":"804c772eba6e16cc0d9d85649886774412d22a83cbb57c5a79830dad85ca2ee3"}
Jul 18 01:57:57.434410 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:57.434267    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5db757d4dd-mt2r5" podStartSLOduration=33.434227963 podStartE2EDuration="33.434227963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:57:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:57.433760365 +0000 UTC m=+72.058797037" watchObservedRunningTime="2026-07-18 01:57:57.434227963 +0000 UTC m=+72.059264632"
Jul 18 01:57:57.435746 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:57.435214    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-qgrtk" podStartSLOduration=70.175427258 podStartE2EDuration="1m11.435203367s" totalImagesPullingTime="1.259776109s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:56:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:57.41527899 +0000 UTC m=+72.040315658" watchObservedRunningTime="2026-07-18 01:57:57.435203367 +0000 UTC m=+72.060240035"
Jul 18 01:57:57.934493 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:57.934457    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5db757d4dd-mt2r5"
Jul 18 01:57:57.937754 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:57.937724    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5db757d4dd-mt2r5"
Jul 18 01:57:58.410059 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:58.410032    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5db757d4dd-mt2r5"
Jul 18 01:57:58.411375 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:58.411348    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5db757d4dd-mt2r5"
Jul 18 01:57:59.145448 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:57:59.145395    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa61e8ac_7c4a_4625_a8c8_97854f4fe565.slice/crio-5c4155cf20aafe6c6f0e4924019c7f46244747de314a0862b423a94046e3e068 WatchSource:0}: Error finding container 5c4155cf20aafe6c6f0e4924019c7f46244747de314a0862b423a94046e3e068: Status 404 returned error can't find the container with id 5c4155cf20aafe6c6f0e4924019c7f46244747de314a0862b423a94046e3e068
Jul 18 01:57:59.416310 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:57:59.416143    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rxgl4" event={"ID":"aa61e8ac-7c4a-4625-a8c8-97854f4fe565","Type":"ContainerStarted","Data":"5c4155cf20aafe6c6f0e4924019c7f46244747de314a0862b423a94046e3e068"}
Jul 18 01:58:02.428832 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:02.428795    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kj28b" event={"ID":"c9fa15c4-1db0-4b36-999d-5507f4cb2d19","Type":"ContainerStarted","Data":"63eb1cfa5f2e2848ee2529377c44cae437f494a9edbb03a3ba0b94e8ef373f0f"}
Jul 18 01:58:02.430354 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:02.430323    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jnffd" event={"ID":"a370ba82-a51e-41aa-9ced-19bab3186de4","Type":"ContainerStarted","Data":"d8de83685ae8324d71eb06d16a3e2d16af4661fc63d24c9648613edc0c91d57d"}
Jul 18 01:58:02.444656 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:02.444601    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-jnffd" podStartSLOduration=70.006024017 podStartE2EDuration="1m16.444580192s" totalImagesPullingTime="6.438556175s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:56:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:02.444473947 +0000 UTC m=+77.069510616" watchObservedRunningTime="2026-07-18 01:58:02.444580192 +0000 UTC m=+77.069616860"
Jul 18 01:58:05.442906 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:05.442854    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s78gl" event={"ID":"c378a52c-b5db-4a92-bd55-8616ad887e45","Type":"ContainerStarted","Data":"e0e71663f36ecb33d262c501842456d484b4a0513a6149874e393953c2d47f9f"}
Jul 18 01:58:05.444927 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:05.444899    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kj28b" event={"ID":"c9fa15c4-1db0-4b36-999d-5507f4cb2d19","Type":"ContainerStarted","Data":"4f400c7c4959c950e810031450839ac7d4b01bd234bce98baff98ebb97b62736"}
Jul 18 01:58:05.445065 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:05.445048    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-kj28b"
Jul 18 01:58:05.446568 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:05.446529    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-zhjgp" event={"ID":"eaae66f5-7295-483b-a1d3-9077f4e9c82b","Type":"ContainerStarted","Data":"ff71433fab79c9aa457aee11394e2360d2afe57d09a6c0b1b8156520abba3c63"}
Jul 18 01:58:05.448485 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:05.448463    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-g6ghg" event={"ID":"9f46f015-6114-42c0-8bf2-0fa0d0e2e778","Type":"ContainerStarted","Data":"e124de999cb2c7b1375902be9f28257f591aa45f168353523733ec87a57b606a"}
Jul 18 01:58:05.448598 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:05.448509    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-g6ghg" event={"ID":"9f46f015-6114-42c0-8bf2-0fa0d0e2e778","Type":"ContainerStarted","Data":"615edb3371bc31c7852053e47e7e716fa777e2dee65290ce478ec7f9991377b9"}
Jul 18 01:58:05.450140 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:05.450115    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rxgl4" event={"ID":"aa61e8ac-7c4a-4625-a8c8-97854f4fe565","Type":"ContainerStarted","Data":"793246db919801759af3ef52c5e22d24e0a1056436b4d2183c3511fc4ce1ce4d"}
Jul 18 01:58:05.465550 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:05.465484    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-kj28b" podStartSLOduration=37.994051012 podStartE2EDuration="44.465466709s" totalImagesPullingTime="6.471415697s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:57:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:05.464621193 +0000 UTC m=+80.089657885" watchObservedRunningTime="2026-07-18 01:58:05.465466709 +0000 UTC m=+80.090503377"
Jul 18 01:58:05.491085 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:05.491018    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-g6ghg" podStartSLOduration=34.265638292 podStartE2EDuration="41.490996969s" totalImagesPullingTime="7.225358677s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:57:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:05.490316565 +0000 UTC m=+80.115353232" watchObservedRunningTime="2026-07-18 01:58:05.490996969 +0000 UTC m=+80.116033648"
Jul 18 01:58:05.513246 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:05.513188    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-zhjgp" podStartSLOduration=34.252625457 podStartE2EDuration="41.51316908s" totalImagesPullingTime="7.260543623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:57:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:05.512376973 +0000 UTC m=+80.137413641" watchObservedRunningTime="2026-07-18 01:58:05.51316908 +0000 UTC m=+80.138205747"
Jul 18 01:58:05.531215 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:05.531153    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rxgl4" podStartSLOduration=36.347123128 podStartE2EDuration="41.531131262s" totalImagesPullingTime="5.184008134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:57:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:05.530064428 +0000 UTC m=+80.155101095" watchObservedRunningTime="2026-07-18 01:58:05.531131262 +0000 UTC m=+80.156167929"
Jul 18 01:58:08.463624 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:08.463473    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s78gl" event={"ID":"c378a52c-b5db-4a92-bd55-8616ad887e45","Type":"ContainerStarted","Data":"6c6bffe158d65e0ab22f26a4a11d4e2a6df011f48db2345599eb7827261c11e7"}
Jul 18 01:58:08.481031 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:08.480978    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-s78gl" podStartSLOduration=16.790340197 podStartE2EDuration="34.480962858s" totalImagesPullingTime="17.690622661s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:57:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:08.479928387 +0000 UTC m=+83.104965054" watchObservedRunningTime="2026-07-18 01:58:08.480962858 +0000 UTC m=+83.105999525"
Jul 18 01:58:12.456829 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:12.456797    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-kj28b"
Jul 18 01:58:17.411389 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:17.411352    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh"
Jul 18 01:58:22.426533 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:22.426496    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-b4x4l"]
Jul 18 01:58:22.431078 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:22.431059    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b4x4l"
Jul 18 01:58:22.433073 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:22.433040    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 18 01:58:22.433187 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:22.433166    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 18 01:58:22.434019 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:22.433993    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-s9bfv\""
Jul 18 01:58:22.434140 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:22.433996    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 18 01:58:22.434140 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:22.434045    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 18 01:58:22.582788 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:22.582751    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/db90bbd1-feff-47f9-a0b5-f00cbe34503a-sys\") pod \"node-exporter-b4x4l\" (UID: \"db90bbd1-feff-47f9-a0b5-f00cbe34503a\") " pod="openshift-monitoring/node-exporter-b4x4l"
Jul 18 01:58:22.582984 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:22.582808    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/db90bbd1-feff-47f9-a0b5-f00cbe34503a-node-exporter-tls\") pod \"node-exporter-b4x4l\" (UID: \"db90bbd1-feff-47f9-a0b5-f00cbe34503a\") " pod="openshift-monitoring/node-exporter-b4x4l"
Jul 18 01:58:22.582984 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:22.582881    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/db90bbd1-feff-47f9-a0b5-f00cbe34503a-node-exporter-wtmp\") pod \"node-exporter-b4x4l\" (UID: \"db90bbd1-feff-47f9-a0b5-f00cbe34503a\") " pod="openshift-monitoring/node-exporter-b4x4l"
Jul 18 01:58:22.582984 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:22.582941    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/db90bbd1-feff-47f9-a0b5-f00cbe34503a-node-exporter-textfile\") pod \"node-exporter-b4x4l\" (UID: \"db90bbd1-feff-47f9-a0b5-f00cbe34503a\") " pod="openshift-monitoring/node-exporter-b4x4l"
Jul 18 01:58:22.583097 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:22.583000    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/db90bbd1-feff-47f9-a0b5-f00cbe34503a-root\") pod \"node-exporter-b4x4l\" (UID: \"db90bbd1-feff-47f9-a0b5-f00cbe34503a\") " pod="openshift-monitoring/node-exporter-b4x4l"
Jul 18 01:58:22.583097 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:22.583033    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/db90bbd1-feff-47f9-a0b5-f00cbe34503a-metrics-client-ca\") pod \"node-exporter-b4x4l\" (UID: \"db90bbd1-feff-47f9-a0b5-f00cbe34503a\") " pod="openshift-monitoring/node-exporter-b4x4l"
Jul 18 01:58:22.583097 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:22.583059    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/db90bbd1-feff-47f9-a0b5-f00cbe34503a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b4x4l\" (UID: \"db90bbd1-feff-47f9-a0b5-f00cbe34503a\") " pod="openshift-monitoring/node-exporter-b4x4l"
Jul 18 01:58:22.583203 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:22.583128    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/db90bbd1-feff-47f9-a0b5-f00cbe34503a-node-exporter-accelerators-collector-config\") pod \"node-exporter-b4x4l\" (UID: \"db90bbd1-feff-47f9-a0b5-f00cbe34503a\") " pod="openshift-monitoring/node-exporter-b4x4l"
Jul 18 01:58:22.583203 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:22.583152    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpqsk\" (UniqueName: \"kubernetes.io/projected/db90bbd1-feff-47f9-a0b5-f00cbe34503a-kube-api-access-vpqsk\") pod \"node-exporter-b4x4l\" (UID: \"db90bbd1-feff-47f9-a0b5-f00cbe34503a\") " pod="openshift-monitoring/node-exporter-b4x4l"
Jul 18 01:58:22.684602 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:22.684515    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpqsk\" (UniqueName: \"kubernetes.io/projected/db90bbd1-feff-47f9-a0b5-f00cbe34503a-kube-api-access-vpqsk\") pod \"node-exporter-b4x4l\" (UID: \"db90bbd1-feff-47f9-a0b5-f00cbe34503a\") " pod="openshift-monitoring/node-exporter-b4x4l"
Jul 18 01:58:22.684602 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:22.684562    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/db90bbd1-feff-47f9-a0b5-f00cbe34503a-sys\") pod \"node-exporter-b4x4l\" (UID: \"db90bbd1-feff-47f9-a0b5-f00cbe34503a\") " pod="openshift-monitoring/node-exporter-b4x4l"
Jul 18 01:58:22.684602 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:22.684583    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/db90bbd1-feff-47f9-a0b5-f00cbe34503a-node-exporter-tls\") pod \"node-exporter-b4x4l\" (UID: \"db90bbd1-feff-47f9-a0b5-f00cbe34503a\") " pod="openshift-monitoring/node-exporter-b4x4l"
Jul 18 01:58:22.684602 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:22.684598    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/db90bbd1-feff-47f9-a0b5-f00cbe34503a-node-exporter-wtmp\") pod \"node-exporter-b4x4l\" (UID: \"db90bbd1-feff-47f9-a0b5-f00cbe34503a\") " pod="openshift-monitoring/node-exporter-b4x4l"
Jul 18 01:58:22.684922 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:22.684621    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/db90bbd1-feff-47f9-a0b5-f00cbe34503a-node-exporter-textfile\") pod \"node-exporter-b4x4l\" (UID: \"db90bbd1-feff-47f9-a0b5-f00cbe34503a\") " pod="openshift-monitoring/node-exporter-b4x4l"
Jul 18 01:58:22.684922 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:22.684643    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/db90bbd1-feff-47f9-a0b5-f00cbe34503a-root\") pod \"node-exporter-b4x4l\" (UID: \"db90bbd1-feff-47f9-a0b5-f00cbe34503a\") " pod="openshift-monitoring/node-exporter-b4x4l"
Jul 18 01:58:22.684922 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:22.684651    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/db90bbd1-feff-47f9-a0b5-f00cbe34503a-sys\") pod \"node-exporter-b4x4l\" (UID: \"db90bbd1-feff-47f9-a0b5-f00cbe34503a\") " pod="openshift-monitoring/node-exporter-b4x4l"
Jul 18 01:58:22.684922 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:22.684662    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/db90bbd1-feff-47f9-a0b5-f00cbe34503a-metrics-client-ca\") pod \"node-exporter-b4x4l\" (UID: \"db90bbd1-feff-47f9-a0b5-f00cbe34503a\") " pod="openshift-monitoring/node-exporter-b4x4l"
Jul 18 01:58:22.684922 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:58:22.684701    2576 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 18 01:58:22.684922 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:22.684711    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/db90bbd1-feff-47f9-a0b5-f00cbe34503a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b4x4l\" (UID: \"db90bbd1-feff-47f9-a0b5-f00cbe34503a\") " pod="openshift-monitoring/node-exporter-b4x4l"
Jul 18 01:58:22.684922 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:22.684714    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/db90bbd1-feff-47f9-a0b5-f00cbe34503a-root\") pod \"node-exporter-b4x4l\" (UID: \"db90bbd1-feff-47f9-a0b5-f00cbe34503a\") " pod="openshift-monitoring/node-exporter-b4x4l"
Jul 18 01:58:22.684922 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:58:22.684778    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db90bbd1-feff-47f9-a0b5-f00cbe34503a-node-exporter-tls podName:db90bbd1-feff-47f9-a0b5-f00cbe34503a nodeName:}" failed. No retries permitted until 2026-07-18 01:58:23.184756746 +0000 UTC m=+97.809793410 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/db90bbd1-feff-47f9-a0b5-f00cbe34503a-node-exporter-tls") pod "node-exporter-b4x4l" (UID: "db90bbd1-feff-47f9-a0b5-f00cbe34503a") : secret "node-exporter-tls" not found
Jul 18 01:58:22.684922 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:22.684823    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/db90bbd1-feff-47f9-a0b5-f00cbe34503a-node-exporter-wtmp\") pod \"node-exporter-b4x4l\" (UID: \"db90bbd1-feff-47f9-a0b5-f00cbe34503a\") " pod="openshift-monitoring/node-exporter-b4x4l"
Jul 18 01:58:22.684922 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:22.684874    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/db90bbd1-feff-47f9-a0b5-f00cbe34503a-node-exporter-accelerators-collector-config\") pod \"node-exporter-b4x4l\" (UID: \"db90bbd1-feff-47f9-a0b5-f00cbe34503a\") " pod="openshift-monitoring/node-exporter-b4x4l"
Jul 18 01:58:22.685248 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:22.684988    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/db90bbd1-feff-47f9-a0b5-f00cbe34503a-node-exporter-textfile\") pod \"node-exporter-b4x4l\" (UID: \"db90bbd1-feff-47f9-a0b5-f00cbe34503a\") " pod="openshift-monitoring/node-exporter-b4x4l"
Jul 18 01:58:22.685283 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:22.685248    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/db90bbd1-feff-47f9-a0b5-f00cbe34503a-metrics-client-ca\") pod \"node-exporter-b4x4l\" (UID: \"db90bbd1-feff-47f9-a0b5-f00cbe34503a\") " pod="openshift-monitoring/node-exporter-b4x4l"
Jul 18 01:58:22.685317 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:22.685292    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/db90bbd1-feff-47f9-a0b5-f00cbe34503a-node-exporter-accelerators-collector-config\") pod \"node-exporter-b4x4l\" (UID: \"db90bbd1-feff-47f9-a0b5-f00cbe34503a\") " pod="openshift-monitoring/node-exporter-b4x4l"
Jul 18 01:58:22.687057 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:22.687029    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/db90bbd1-feff-47f9-a0b5-f00cbe34503a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b4x4l\" (UID: \"db90bbd1-feff-47f9-a0b5-f00cbe34503a\") " pod="openshift-monitoring/node-exporter-b4x4l"
Jul 18 01:58:22.692442 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:22.692409    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpqsk\" (UniqueName: \"kubernetes.io/projected/db90bbd1-feff-47f9-a0b5-f00cbe34503a-kube-api-access-vpqsk\") pod \"node-exporter-b4x4l\" (UID: \"db90bbd1-feff-47f9-a0b5-f00cbe34503a\") " pod="openshift-monitoring/node-exporter-b4x4l"
Jul 18 01:58:23.188984 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:23.188945    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/db90bbd1-feff-47f9-a0b5-f00cbe34503a-node-exporter-tls\") pod \"node-exporter-b4x4l\" (UID: \"db90bbd1-feff-47f9-a0b5-f00cbe34503a\") " pod="openshift-monitoring/node-exporter-b4x4l"
Jul 18 01:58:23.191387 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:23.191363    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/db90bbd1-feff-47f9-a0b5-f00cbe34503a-node-exporter-tls\") pod \"node-exporter-b4x4l\" (UID: \"db90bbd1-feff-47f9-a0b5-f00cbe34503a\") " pod="openshift-monitoring/node-exporter-b4x4l"
Jul 18 01:58:23.340640 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:23.340611    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b4x4l"
Jul 18 01:58:23.353567 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:58:23.353526    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb90bbd1_feff_47f9_a0b5_f00cbe34503a.slice/crio-0631b1dc4deb4146f3099916d3909d2dd377f7b1e37a5dea946d33d058301a82 WatchSource:0}: Error finding container 0631b1dc4deb4146f3099916d3909d2dd377f7b1e37a5dea946d33d058301a82: Status 404 returned error can't find the container with id 0631b1dc4deb4146f3099916d3909d2dd377f7b1e37a5dea946d33d058301a82
Jul 18 01:58:23.514752 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:23.514661    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b4x4l" event={"ID":"db90bbd1-feff-47f9-a0b5-f00cbe34503a","Type":"ContainerStarted","Data":"0631b1dc4deb4146f3099916d3909d2dd377f7b1e37a5dea946d33d058301a82"}
Jul 18 01:58:24.520130 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:24.520094    2576 generic.go:356] "Generic (PLEG): container finished" podID="db90bbd1-feff-47f9-a0b5-f00cbe34503a" containerID="9fe9ca9307cdca74614f955399f8083e89e78643640b0fc233c9dd8a9acc2223" exitCode=0
Jul 18 01:58:24.520499 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:24.520176    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b4x4l" event={"ID":"db90bbd1-feff-47f9-a0b5-f00cbe34503a","Type":"ContainerDied","Data":"9fe9ca9307cdca74614f955399f8083e89e78643640b0fc233c9dd8a9acc2223"}
Jul 18 01:58:25.526168 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:25.526131    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b4x4l" event={"ID":"db90bbd1-feff-47f9-a0b5-f00cbe34503a","Type":"ContainerStarted","Data":"4ca883842f82516af9540202ee67a149836a9bab2f790a15658e16050300ce66"}
Jul 18 01:58:25.526679 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:25.526226    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b4x4l" event={"ID":"db90bbd1-feff-47f9-a0b5-f00cbe34503a","Type":"ContainerStarted","Data":"88b40f26933e95111be3f96fbfe6b47dc4a16ea320641b181b92efa631294f4c"}
Jul 18 01:58:25.545411 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:25.545357    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-b4x4l" podStartSLOduration=2.497672744 podStartE2EDuration="3.545341694s" totalImagesPullingTime="983.797613ms" totalInitContainerRuntime="63.871337ms" isStatefulPod=true podCreationTimestamp="2026-07-18 01:58:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:25.54377679 +0000 UTC m=+100.168813458" watchObservedRunningTime="2026-07-18 01:58:25.545341694 +0000 UTC m=+100.170378361"
Jul 18 01:58:27.245362 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:27.245327    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-76cf94c9f7-87dhh"]
Jul 18 01:58:27.409149 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:27.409120    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-h6pxz"
Jul 18 01:58:43.590685 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:43.590652    2576 generic.go:356] "Generic (PLEG): container finished" podID="28a9ce44-0900-4265-a0ee-67acdcfde240" containerID="a4edb7a7cbbc729e0fa722760901fdd0ac1dedf4bcf86b63a09515a768a73eb6" exitCode=0
Jul 18 01:58:43.591074 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:43.590722    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-tbs7d" event={"ID":"28a9ce44-0900-4265-a0ee-67acdcfde240","Type":"ContainerDied","Data":"a4edb7a7cbbc729e0fa722760901fdd0ac1dedf4bcf86b63a09515a768a73eb6"}
Jul 18 01:58:43.591074 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:43.591052    2576 scope.go:117] "RemoveContainer" containerID="a4edb7a7cbbc729e0fa722760901fdd0ac1dedf4bcf86b63a09515a768a73eb6"
Jul 18 01:58:44.596521 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:44.596488    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-tbs7d" event={"ID":"28a9ce44-0900-4265-a0ee-67acdcfde240","Type":"ContainerStarted","Data":"15350d8a8f3cb6fdf3a74595256291e79de1588cd56c10fbb39cee0068398215"}
Jul 18 01:58:48.611778 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:48.611748    2576 generic.go:356] "Generic (PLEG): container finished" podID="dcf31fe7-21f9-4a41-9b37-2f8440a7c439" containerID="893c0b27bc38b80d8c29ff99eab08b0b00ce1b8a6122386aa9b0bede552ab4ba" exitCode=0
Jul 18 01:58:48.612079 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:48.611819    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-69sww" event={"ID":"dcf31fe7-21f9-4a41-9b37-2f8440a7c439","Type":"ContainerDied","Data":"893c0b27bc38b80d8c29ff99eab08b0b00ce1b8a6122386aa9b0bede552ab4ba"}
Jul 18 01:58:48.612179 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:48.612166    2576 scope.go:117] "RemoveContainer" containerID="893c0b27bc38b80d8c29ff99eab08b0b00ce1b8a6122386aa9b0bede552ab4ba"
Jul 18 01:58:49.106358 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:49.106320    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5db757d4dd-mt2r5_dc92e8cf-d868-4c6f-963b-04ff7caf51aa/router/0.log"
Jul 18 01:58:49.123536 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:49.123513    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tw7tf_66623dd6-501f-42b2-b12a-188cf5520290/serve-healthcheck-canary/0.log"
Jul 18 01:58:49.617368 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:49.617332    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-69sww" event={"ID":"dcf31fe7-21f9-4a41-9b37-2f8440a7c439","Type":"ContainerStarted","Data":"8a4f9588fbd8b2c838a946be3b112f08d96f4caa79979eebbfe736133a0b0757"}
Jul 18 01:58:52.266380 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:52.266316    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh" podUID="f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09" containerName="registry" containerID="cri-o://27e3e887f245acb40584515252bed1b8c8102fa6c77f96c187027e444abc1e4d" gracePeriod=30
Jul 18 01:58:52.510507 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:52.510481    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh"
Jul 18 01:58:52.607333 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:52.607242    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-registry-certificates\") pod \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\" (UID: \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\") "
Jul 18 01:58:52.607333 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:52.607280    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-installation-pull-secrets\") pod \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\" (UID: \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\") "
Jul 18 01:58:52.607333 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:52.607301    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vbt55\" (UniqueName: \"kubernetes.io/projected/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-kube-api-access-vbt55\") pod \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\" (UID: \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\") "
Jul 18 01:58:52.607612 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:52.607340    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-bound-sa-token\") pod \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\" (UID: \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\") "
Jul 18 01:58:52.607612 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:52.607380    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-registry-tls\") pod \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\" (UID: \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\") "
Jul 18 01:58:52.607612 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:52.607407    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-image-registry-private-configuration\") pod \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\" (UID: \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\") "
Jul 18 01:58:52.607612 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:52.607503    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-ca-trust-extracted\") pod \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\" (UID: \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\") "
Jul 18 01:58:52.607612 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:52.607529    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-trusted-ca\") pod \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\" (UID: \"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09\") "
Jul 18 01:58:52.607865 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:52.607713    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09" (UID: "f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:58:52.608124 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:52.608098    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09" (UID: "f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:58:52.609918 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:52.609878    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09" (UID: "f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:58:52.610139 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:52.610116    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-kube-api-access-vbt55" (OuterVolumeSpecName: "kube-api-access-vbt55") pod "f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09" (UID: "f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09"). InnerVolumeSpecName "kube-api-access-vbt55". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:58:52.610212 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:52.610147    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09" (UID: "f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:58:52.610212 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:52.610177    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09" (UID: "f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:58:52.610284 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:52.610225    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09" (UID: "f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:58:52.618350 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:52.618315    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09" (UID: "f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:58:52.628308 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:52.628281    2576 generic.go:356] "Generic (PLEG): container finished" podID="f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09" containerID="27e3e887f245acb40584515252bed1b8c8102fa6c77f96c187027e444abc1e4d" exitCode=0
Jul 18 01:58:52.628450 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:52.628337    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh" event={"ID":"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09","Type":"ContainerDied","Data":"27e3e887f245acb40584515252bed1b8c8102fa6c77f96c187027e444abc1e4d"}
Jul 18 01:58:52.628450 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:52.628377    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh" event={"ID":"f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09","Type":"ContainerDied","Data":"84e0854895d0deb6cdffe112858ea39322e292e461ebd693859256a6f95bb0be"}
Jul 18 01:58:52.628450 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:52.628398    2576 scope.go:117] "RemoveContainer" containerID="27e3e887f245acb40584515252bed1b8c8102fa6c77f96c187027e444abc1e4d"
Jul 18 01:58:52.628450 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:52.628338    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76cf94c9f7-87dhh"
Jul 18 01:58:52.636642 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:52.636623    2576 scope.go:117] "RemoveContainer" containerID="27e3e887f245acb40584515252bed1b8c8102fa6c77f96c187027e444abc1e4d"
Jul 18 01:58:52.636950 ip-10-0-143-8 kubenswrapper[2576]: E0718 01:58:52.636932    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27e3e887f245acb40584515252bed1b8c8102fa6c77f96c187027e444abc1e4d\": container with ID starting with 27e3e887f245acb40584515252bed1b8c8102fa6c77f96c187027e444abc1e4d not found: ID does not exist" containerID="27e3e887f245acb40584515252bed1b8c8102fa6c77f96c187027e444abc1e4d"
Jul 18 01:58:52.636997 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:52.636959    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27e3e887f245acb40584515252bed1b8c8102fa6c77f96c187027e444abc1e4d"} err="failed to get container status \"27e3e887f245acb40584515252bed1b8c8102fa6c77f96c187027e444abc1e4d\": rpc error: code = NotFound desc = could not find container \"27e3e887f245acb40584515252bed1b8c8102fa6c77f96c187027e444abc1e4d\": container with ID starting with 27e3e887f245acb40584515252bed1b8c8102fa6c77f96c187027e444abc1e4d not found: ID does not exist"
Jul 18 01:58:52.648609 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:52.648585    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-76cf94c9f7-87dhh"]
Jul 18 01:58:52.651968 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:52.651948    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-76cf94c9f7-87dhh"]
Jul 18 01:58:52.709052 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:52.709023    2576 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-bound-sa-token\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 01:58:52.709052 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:52.709048    2576 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-registry-tls\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 01:58:52.709052 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:52.709058    2576 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-image-registry-private-configuration\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 01:58:52.709256 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:52.709069    2576 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-ca-trust-extracted\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 01:58:52.709256 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:52.709081    2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-trusted-ca\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 01:58:52.709256 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:52.709089    2576 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-registry-certificates\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 01:58:52.709256 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:52.709098    2576 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-installation-pull-secrets\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 01:58:52.709256 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:52.709108    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbt55\" (UniqueName: \"kubernetes.io/projected/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09-kube-api-access-vbt55\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 01:58:53.967119 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:53.967091    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09" path="/var/lib/kubelet/pods/f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09/volumes"
Jul 18 01:58:59.656498 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:59.656458    2576 generic.go:356] "Generic (PLEG): container finished" podID="73fe74bc-8fc3-4907-874e-5b83af50d2a1" containerID="c6dd76ef87ee7d6dd0f515570eea90e7767ed26086f09788afef9483f22434a7" exitCode=0
Jul 18 01:58:59.657013 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:59.656517    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-smbjg" event={"ID":"73fe74bc-8fc3-4907-874e-5b83af50d2a1","Type":"ContainerDied","Data":"c6dd76ef87ee7d6dd0f515570eea90e7767ed26086f09788afef9483f22434a7"}
Jul 18 01:58:59.657013 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:58:59.656917    2576 scope.go:117] "RemoveContainer" containerID="c6dd76ef87ee7d6dd0f515570eea90e7767ed26086f09788afef9483f22434a7"
Jul 18 01:59:00.662993 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:00.662957    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-smbjg" event={"ID":"73fe74bc-8fc3-4907-874e-5b83af50d2a1","Type":"ContainerStarted","Data":"fc2d1217d881168da8c23f16912b2bfe54412179026bdba2fa7cbd0c69226541"}
Jul 18 01:59:46.645994 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:46.645951    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-58f474ff5c-x87mb"]
Jul 18 01:59:46.646501 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:46.646260    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09" containerName="registry"
Jul 18 01:59:46.646501 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:46.646274    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09" containerName="registry"
Jul 18 01:59:46.646501 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:46.646328    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3a1d9e2-58d8-4f09-bf0c-5dbfa5123b09" containerName="registry"
Jul 18 01:59:46.650595 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:46.650570    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-58f474ff5c-x87mb"
Jul 18 01:59:46.652961 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:46.652936    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 18 01:59:46.652961 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:46.652966    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 18 01:59:46.653144 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:46.653042    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-mdpwc\""
Jul 18 01:59:46.653144 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:46.653057    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 18 01:59:46.653314 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:46.653237    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 18 01:59:46.653508 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:46.653493    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 18 01:59:46.658093 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:46.657807    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 18 01:59:46.661731 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:46.661712    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-58f474ff5c-x87mb"]
Jul 18 01:59:46.732286 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:46.732234    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/432c67b4-af56-43e9-bccb-af44842443c7-telemeter-client-tls\") pod \"telemeter-client-58f474ff5c-x87mb\" (UID: \"432c67b4-af56-43e9-bccb-af44842443c7\") " pod="openshift-monitoring/telemeter-client-58f474ff5c-x87mb"
Jul 18 01:59:46.732286 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:46.732293    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/432c67b4-af56-43e9-bccb-af44842443c7-telemeter-trusted-ca-bundle\") pod \"telemeter-client-58f474ff5c-x87mb\" (UID: \"432c67b4-af56-43e9-bccb-af44842443c7\") " pod="openshift-monitoring/telemeter-client-58f474ff5c-x87mb"
Jul 18 01:59:46.732554 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:46.732375    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/432c67b4-af56-43e9-bccb-af44842443c7-federate-client-tls\") pod \"telemeter-client-58f474ff5c-x87mb\" (UID: \"432c67b4-af56-43e9-bccb-af44842443c7\") " pod="openshift-monitoring/telemeter-client-58f474ff5c-x87mb"
Jul 18 01:59:46.732554 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:46.732408    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/432c67b4-af56-43e9-bccb-af44842443c7-secret-telemeter-client\") pod \"telemeter-client-58f474ff5c-x87mb\" (UID: \"432c67b4-af56-43e9-bccb-af44842443c7\") " pod="openshift-monitoring/telemeter-client-58f474ff5c-x87mb"
Jul 18 01:59:46.732554 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:46.732448    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/432c67b4-af56-43e9-bccb-af44842443c7-metrics-client-ca\") pod \"telemeter-client-58f474ff5c-x87mb\" (UID: \"432c67b4-af56-43e9-bccb-af44842443c7\") " pod="openshift-monitoring/telemeter-client-58f474ff5c-x87mb"
Jul 18 01:59:46.732554 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:46.732468    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6swdl\" (UniqueName: \"kubernetes.io/projected/432c67b4-af56-43e9-bccb-af44842443c7-kube-api-access-6swdl\") pod \"telemeter-client-58f474ff5c-x87mb\" (UID: \"432c67b4-af56-43e9-bccb-af44842443c7\") " pod="openshift-monitoring/telemeter-client-58f474ff5c-x87mb"
Jul 18 01:59:46.732554 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:46.732507    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/432c67b4-af56-43e9-bccb-af44842443c7-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-58f474ff5c-x87mb\" (UID: \"432c67b4-af56-43e9-bccb-af44842443c7\") " pod="openshift-monitoring/telemeter-client-58f474ff5c-x87mb"
Jul 18 01:59:46.732554 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:46.732545    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/432c67b4-af56-43e9-bccb-af44842443c7-serving-certs-ca-bundle\") pod \"telemeter-client-58f474ff5c-x87mb\" (UID: \"432c67b4-af56-43e9-bccb-af44842443c7\") " pod="openshift-monitoring/telemeter-client-58f474ff5c-x87mb"
Jul 18 01:59:46.833726 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:46.833690    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/432c67b4-af56-43e9-bccb-af44842443c7-telemeter-trusted-ca-bundle\") pod \"telemeter-client-58f474ff5c-x87mb\" (UID: \"432c67b4-af56-43e9-bccb-af44842443c7\") " pod="openshift-monitoring/telemeter-client-58f474ff5c-x87mb"
Jul 18 01:59:46.833915 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:46.833742    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/432c67b4-af56-43e9-bccb-af44842443c7-federate-client-tls\") pod \"telemeter-client-58f474ff5c-x87mb\" (UID: \"432c67b4-af56-43e9-bccb-af44842443c7\") " pod="openshift-monitoring/telemeter-client-58f474ff5c-x87mb"
Jul 18 01:59:46.833915 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:46.833775    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/432c67b4-af56-43e9-bccb-af44842443c7-secret-telemeter-client\") pod \"telemeter-client-58f474ff5c-x87mb\" (UID: \"432c67b4-af56-43e9-bccb-af44842443c7\") " pod="openshift-monitoring/telemeter-client-58f474ff5c-x87mb"
Jul 18 01:59:46.833915 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:46.833801    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/432c67b4-af56-43e9-bccb-af44842443c7-metrics-client-ca\") pod \"telemeter-client-58f474ff5c-x87mb\" (UID: \"432c67b4-af56-43e9-bccb-af44842443c7\") " pod="openshift-monitoring/telemeter-client-58f474ff5c-x87mb"
Jul 18 01:59:46.833915 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:46.833829    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6swdl\" (UniqueName: \"kubernetes.io/projected/432c67b4-af56-43e9-bccb-af44842443c7-kube-api-access-6swdl\") pod \"telemeter-client-58f474ff5c-x87mb\" (UID: \"432c67b4-af56-43e9-bccb-af44842443c7\") " pod="openshift-monitoring/telemeter-client-58f474ff5c-x87mb"
Jul 18 01:59:46.833915 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:46.833900    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/432c67b4-af56-43e9-bccb-af44842443c7-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-58f474ff5c-x87mb\" (UID: \"432c67b4-af56-43e9-bccb-af44842443c7\") " pod="openshift-monitoring/telemeter-client-58f474ff5c-x87mb"
Jul 18 01:59:46.834180 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:46.833940    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/432c67b4-af56-43e9-bccb-af44842443c7-serving-certs-ca-bundle\") pod \"telemeter-client-58f474ff5c-x87mb\" (UID: \"432c67b4-af56-43e9-bccb-af44842443c7\") " pod="openshift-monitoring/telemeter-client-58f474ff5c-x87mb"
Jul 18 01:59:46.834180 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:46.833972    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/432c67b4-af56-43e9-bccb-af44842443c7-telemeter-client-tls\") pod \"telemeter-client-58f474ff5c-x87mb\" (UID: \"432c67b4-af56-43e9-bccb-af44842443c7\") " pod="openshift-monitoring/telemeter-client-58f474ff5c-x87mb"
Jul 18 01:59:46.834716 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:46.834692    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/432c67b4-af56-43e9-bccb-af44842443c7-metrics-client-ca\") pod \"telemeter-client-58f474ff5c-x87mb\" (UID: \"432c67b4-af56-43e9-bccb-af44842443c7\") " pod="openshift-monitoring/telemeter-client-58f474ff5c-x87mb"
Jul 18 01:59:46.834900 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:46.834738    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/432c67b4-af56-43e9-bccb-af44842443c7-serving-certs-ca-bundle\") pod \"telemeter-client-58f474ff5c-x87mb\" (UID: \"432c67b4-af56-43e9-bccb-af44842443c7\") " pod="openshift-monitoring/telemeter-client-58f474ff5c-x87mb"
Jul 18 01:59:46.834900 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:46.834783    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/432c67b4-af56-43e9-bccb-af44842443c7-telemeter-trusted-ca-bundle\") pod \"telemeter-client-58f474ff5c-x87mb\" (UID: \"432c67b4-af56-43e9-bccb-af44842443c7\") " pod="openshift-monitoring/telemeter-client-58f474ff5c-x87mb"
Jul 18 01:59:46.836473 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:46.836451    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/432c67b4-af56-43e9-bccb-af44842443c7-federate-client-tls\") pod \"telemeter-client-58f474ff5c-x87mb\" (UID: \"432c67b4-af56-43e9-bccb-af44842443c7\") " pod="openshift-monitoring/telemeter-client-58f474ff5c-x87mb"
Jul 18 01:59:46.836567 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:46.836545    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/432c67b4-af56-43e9-bccb-af44842443c7-telemeter-client-tls\") pod \"telemeter-client-58f474ff5c-x87mb\" (UID: \"432c67b4-af56-43e9-bccb-af44842443c7\") " pod="openshift-monitoring/telemeter-client-58f474ff5c-x87mb"
Jul 18 01:59:46.836739 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:46.836720    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/432c67b4-af56-43e9-bccb-af44842443c7-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-58f474ff5c-x87mb\" (UID: \"432c67b4-af56-43e9-bccb-af44842443c7\") " pod="openshift-monitoring/telemeter-client-58f474ff5c-x87mb"
Jul 18 01:59:46.836796 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:46.836728    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/432c67b4-af56-43e9-bccb-af44842443c7-secret-telemeter-client\") pod \"telemeter-client-58f474ff5c-x87mb\" (UID: \"432c67b4-af56-43e9-bccb-af44842443c7\") " pod="openshift-monitoring/telemeter-client-58f474ff5c-x87mb"
Jul 18 01:59:46.841830 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:46.841806    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6swdl\" (UniqueName: \"kubernetes.io/projected/432c67b4-af56-43e9-bccb-af44842443c7-kube-api-access-6swdl\") pod \"telemeter-client-58f474ff5c-x87mb\" (UID: \"432c67b4-af56-43e9-bccb-af44842443c7\") " pod="openshift-monitoring/telemeter-client-58f474ff5c-x87mb"
Jul 18 01:59:46.961188 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:46.961105    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-58f474ff5c-x87mb"
Jul 18 01:59:47.105138 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:47.105105    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-58f474ff5c-x87mb"]
Jul 18 01:59:47.108865 ip-10-0-143-8 kubenswrapper[2576]: W0718 01:59:47.108836    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod432c67b4_af56_43e9_bccb_af44842443c7.slice/crio-ffe3cf81b01e32a7151e263326795ab2dbd059940d0f03d0cdb09792e72b5360 WatchSource:0}: Error finding container ffe3cf81b01e32a7151e263326795ab2dbd059940d0f03d0cdb09792e72b5360: Status 404 returned error can't find the container with id ffe3cf81b01e32a7151e263326795ab2dbd059940d0f03d0cdb09792e72b5360
Jul 18 01:59:47.829714 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:47.829676    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-58f474ff5c-x87mb" event={"ID":"432c67b4-af56-43e9-bccb-af44842443c7","Type":"ContainerStarted","Data":"ffe3cf81b01e32a7151e263326795ab2dbd059940d0f03d0cdb09792e72b5360"}
Jul 18 01:59:49.838778 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:49.838743    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-58f474ff5c-x87mb" event={"ID":"432c67b4-af56-43e9-bccb-af44842443c7","Type":"ContainerStarted","Data":"2fd59a19d302e0995856ec0f8662ea5920c52e8b328968a290f4c843075fc275"}
Jul 18 01:59:50.844577 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:50.844537    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-58f474ff5c-x87mb" event={"ID":"432c67b4-af56-43e9-bccb-af44842443c7","Type":"ContainerStarted","Data":"19f9029da51a36dad3b08897faf7d1e068f23070b6e68522614324a0c7c00899"}
Jul 18 01:59:50.845055 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:50.844596    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-58f474ff5c-x87mb" event={"ID":"432c67b4-af56-43e9-bccb-af44842443c7","Type":"ContainerStarted","Data":"c3221a3f381cf34cacdd24d9b30055a2428633ed9fb92b88f06e5f4e5fc91aa7"}
Jul 18 01:59:50.866223 ip-10-0-143-8 kubenswrapper[2576]: I0718 01:59:50.866165    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-58f474ff5c-x87mb" podStartSLOduration=2.180009603 podStartE2EDuration="4.866149075s" totalImagesPullingTime="2.686139472s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:59:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:59:50.864460359 +0000 UTC m=+185.489497019" watchObservedRunningTime="2026-07-18 01:59:50.866149075 +0000 UTC m=+185.491185746"
Jul 18 02:01:05.347220 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:05.347186    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67c6896796-xgqbf"]
Jul 18 02:01:05.350482 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:05.350461    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67c6896796-xgqbf"
Jul 18 02:01:05.353613 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:05.353588    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 18 02:01:05.353722 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:05.353587    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 18 02:01:05.353801 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:05.353787    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-hv5z9\""
Jul 18 02:01:05.353887 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:05.353867    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 18 02:01:05.354096 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:05.354079    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 18 02:01:05.361931 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:05.361910    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67c6896796-xgqbf"]
Jul 18 02:01:05.446393 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:05.446364    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/04c8a8bf-e29b-412e-b8d5-b5c350c6db0a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-67c6896796-xgqbf\" (UID: \"04c8a8bf-e29b-412e-b8d5-b5c350c6db0a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67c6896796-xgqbf"
Jul 18 02:01:05.446590 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:05.446405    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gw5j6\" (UniqueName: \"kubernetes.io/projected/04c8a8bf-e29b-412e-b8d5-b5c350c6db0a-kube-api-access-gw5j6\") pod \"managed-serviceaccount-addon-agent-67c6896796-xgqbf\" (UID: \"04c8a8bf-e29b-412e-b8d5-b5c350c6db0a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67c6896796-xgqbf"
Jul 18 02:01:05.546912 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:05.546876    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/04c8a8bf-e29b-412e-b8d5-b5c350c6db0a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-67c6896796-xgqbf\" (UID: \"04c8a8bf-e29b-412e-b8d5-b5c350c6db0a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67c6896796-xgqbf"
Jul 18 02:01:05.547061 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:05.546922    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gw5j6\" (UniqueName: \"kubernetes.io/projected/04c8a8bf-e29b-412e-b8d5-b5c350c6db0a-kube-api-access-gw5j6\") pod \"managed-serviceaccount-addon-agent-67c6896796-xgqbf\" (UID: \"04c8a8bf-e29b-412e-b8d5-b5c350c6db0a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67c6896796-xgqbf"
Jul 18 02:01:05.549331 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:05.549308    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/04c8a8bf-e29b-412e-b8d5-b5c350c6db0a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-67c6896796-xgqbf\" (UID: \"04c8a8bf-e29b-412e-b8d5-b5c350c6db0a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67c6896796-xgqbf"
Jul 18 02:01:05.554899 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:05.554876    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gw5j6\" (UniqueName: \"kubernetes.io/projected/04c8a8bf-e29b-412e-b8d5-b5c350c6db0a-kube-api-access-gw5j6\") pod \"managed-serviceaccount-addon-agent-67c6896796-xgqbf\" (UID: \"04c8a8bf-e29b-412e-b8d5-b5c350c6db0a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67c6896796-xgqbf"
Jul 18 02:01:05.672964 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:05.672920    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67c6896796-xgqbf"
Jul 18 02:01:05.798878 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:05.798846    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67c6896796-xgqbf"]
Jul 18 02:01:05.802192 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:01:05.802165    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod04c8a8bf_e29b_412e_b8d5_b5c350c6db0a.slice/crio-fe9c7c46229b3160fe9f1fc4097c77da49b2e482bc2bde0e808e0abae30edc33 WatchSource:0}: Error finding container fe9c7c46229b3160fe9f1fc4097c77da49b2e482bc2bde0e808e0abae30edc33: Status 404 returned error can't find the container with id fe9c7c46229b3160fe9f1fc4097c77da49b2e482bc2bde0e808e0abae30edc33
Jul 18 02:01:06.115715 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:06.115632    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67c6896796-xgqbf" event={"ID":"04c8a8bf-e29b-412e-b8d5-b5c350c6db0a","Type":"ContainerStarted","Data":"fe9c7c46229b3160fe9f1fc4097c77da49b2e482bc2bde0e808e0abae30edc33"}
Jul 18 02:01:09.131292 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:09.131256    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67c6896796-xgqbf" event={"ID":"04c8a8bf-e29b-412e-b8d5-b5c350c6db0a","Type":"ContainerStarted","Data":"143411bfc79ba43cc55000a00edc099dfbee065ec96a27291c334b0e2e1aca88"}
Jul 18 02:01:09.147997 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:09.147945    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67c6896796-xgqbf" podStartSLOduration=1.619622191 podStartE2EDuration="4.14793094s" totalImagesPullingTime="2.528308749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:01:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:01:09.145779015 +0000 UTC m=+263.770815665" watchObservedRunningTime="2026-07-18 02:01:09.14793094 +0000 UTC m=+263.772967607"
Jul 18 02:01:10.055020 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:10.054981    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dhq8x"]
Jul 18 02:01:10.058183 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:10.058161    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dhq8x"
Jul 18 02:01:10.060619 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:10.060597    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 18 02:01:10.060748 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:10.060640    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-zljcg\""
Jul 18 02:01:10.060748 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:10.060639    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 18 02:01:10.070220 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:10.070196    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dhq8x"]
Jul 18 02:01:10.082833 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:10.082807    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d5453717-65e1-4db5-82ea-5a0967dd5230-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-dhq8x\" (UID: \"d5453717-65e1-4db5-82ea-5a0967dd5230\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dhq8x"
Jul 18 02:01:10.082958 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:10.082850    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hdzq\" (UniqueName: \"kubernetes.io/projected/d5453717-65e1-4db5-82ea-5a0967dd5230-kube-api-access-9hdzq\") pod \"cert-manager-operator-controller-manager-55c444bf86-dhq8x\" (UID: \"d5453717-65e1-4db5-82ea-5a0967dd5230\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dhq8x"
Jul 18 02:01:10.184050 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:10.184013    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d5453717-65e1-4db5-82ea-5a0967dd5230-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-dhq8x\" (UID: \"d5453717-65e1-4db5-82ea-5a0967dd5230\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dhq8x"
Jul 18 02:01:10.184554 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:10.184070    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hdzq\" (UniqueName: \"kubernetes.io/projected/d5453717-65e1-4db5-82ea-5a0967dd5230-kube-api-access-9hdzq\") pod \"cert-manager-operator-controller-manager-55c444bf86-dhq8x\" (UID: \"d5453717-65e1-4db5-82ea-5a0967dd5230\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dhq8x"
Jul 18 02:01:10.184554 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:10.184502    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d5453717-65e1-4db5-82ea-5a0967dd5230-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-dhq8x\" (UID: \"d5453717-65e1-4db5-82ea-5a0967dd5230\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dhq8x"
Jul 18 02:01:10.200365 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:10.200336    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hdzq\" (UniqueName: \"kubernetes.io/projected/d5453717-65e1-4db5-82ea-5a0967dd5230-kube-api-access-9hdzq\") pod \"cert-manager-operator-controller-manager-55c444bf86-dhq8x\" (UID: \"d5453717-65e1-4db5-82ea-5a0967dd5230\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dhq8x"
Jul 18 02:01:10.367151 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:10.367053    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dhq8x"
Jul 18 02:01:10.494237 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:10.494208    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dhq8x"]
Jul 18 02:01:10.497972 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:01:10.497945    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5453717_65e1_4db5_82ea_5a0967dd5230.slice/crio-dcc8c794d736f703c9d3f163b7be38461510dae05764a283c8527f76481c9f7f WatchSource:0}: Error finding container dcc8c794d736f703c9d3f163b7be38461510dae05764a283c8527f76481c9f7f: Status 404 returned error can't find the container with id dcc8c794d736f703c9d3f163b7be38461510dae05764a283c8527f76481c9f7f
Jul 18 02:01:11.140273 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:11.140237    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dhq8x" event={"ID":"d5453717-65e1-4db5-82ea-5a0967dd5230","Type":"ContainerStarted","Data":"dcc8c794d736f703c9d3f163b7be38461510dae05764a283c8527f76481c9f7f"}
Jul 18 02:01:14.154139 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:14.154105    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dhq8x" event={"ID":"d5453717-65e1-4db5-82ea-5a0967dd5230","Type":"ContainerStarted","Data":"4379c16b133072b41e9c3f3e5434acac303ce471dc15e98f89aee2d6f92c48b5"}
Jul 18 02:01:14.173561 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:14.173500    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dhq8x" podStartSLOduration=1.464055661 podStartE2EDuration="4.173447951s" totalImagesPullingTime="2.70939229s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:01:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:01:14.171777792 +0000 UTC m=+268.796814459" watchObservedRunningTime="2026-07-18 02:01:14.173447951 +0000 UTC m=+268.798484618"
Jul 18 02:01:14.225027 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:14.224998    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-xp9s9"]
Jul 18 02:01:14.230904 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:14.230874    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-xp9s9"
Jul 18 02:01:14.233715 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:14.233690    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 18 02:01:14.233944 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:14.233929    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-qlf8f\""
Jul 18 02:01:14.234223 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:14.234193    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 18 02:01:14.237743 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:14.237581    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-xp9s9"]
Jul 18 02:01:14.321698 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:14.321650    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/217f8cd9-2bde-4932-8420-26f21576929d-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-xp9s9\" (UID: \"217f8cd9-2bde-4932-8420-26f21576929d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xp9s9"
Jul 18 02:01:14.321883 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:14.321706    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7j9m5\" (UniqueName: \"kubernetes.io/projected/217f8cd9-2bde-4932-8420-26f21576929d-kube-api-access-7j9m5\") pod \"cert-manager-webhook-6486ffd997-xp9s9\" (UID: \"217f8cd9-2bde-4932-8420-26f21576929d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xp9s9"
Jul 18 02:01:14.422704 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:14.422614    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/217f8cd9-2bde-4932-8420-26f21576929d-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-xp9s9\" (UID: \"217f8cd9-2bde-4932-8420-26f21576929d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xp9s9"
Jul 18 02:01:14.422704 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:14.422662    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7j9m5\" (UniqueName: \"kubernetes.io/projected/217f8cd9-2bde-4932-8420-26f21576929d-kube-api-access-7j9m5\") pod \"cert-manager-webhook-6486ffd997-xp9s9\" (UID: \"217f8cd9-2bde-4932-8420-26f21576929d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xp9s9"
Jul 18 02:01:14.434461 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:14.434416    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/217f8cd9-2bde-4932-8420-26f21576929d-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-xp9s9\" (UID: \"217f8cd9-2bde-4932-8420-26f21576929d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xp9s9"
Jul 18 02:01:14.434584 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:14.434496    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j9m5\" (UniqueName: \"kubernetes.io/projected/217f8cd9-2bde-4932-8420-26f21576929d-kube-api-access-7j9m5\") pod \"cert-manager-webhook-6486ffd997-xp9s9\" (UID: \"217f8cd9-2bde-4932-8420-26f21576929d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xp9s9"
Jul 18 02:01:14.540630 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:14.540592    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-xp9s9"
Jul 18 02:01:14.681870 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:14.681843    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-xp9s9"]
Jul 18 02:01:14.683984 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:01:14.683952    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod217f8cd9_2bde_4932_8420_26f21576929d.slice/crio-cf8ae0c7b96f04cd3c6dcd538e81906eca6e83b3062fcffddcb711aa451d2ba5 WatchSource:0}: Error finding container cf8ae0c7b96f04cd3c6dcd538e81906eca6e83b3062fcffddcb711aa451d2ba5: Status 404 returned error can't find the container with id cf8ae0c7b96f04cd3c6dcd538e81906eca6e83b3062fcffddcb711aa451d2ba5
Jul 18 02:01:15.159194 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:15.159160    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-xp9s9" event={"ID":"217f8cd9-2bde-4932-8420-26f21576929d","Type":"ContainerStarted","Data":"cf8ae0c7b96f04cd3c6dcd538e81906eca6e83b3062fcffddcb711aa451d2ba5"}
Jul 18 02:01:19.178554 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:19.178520    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-xp9s9" event={"ID":"217f8cd9-2bde-4932-8420-26f21576929d","Type":"ContainerStarted","Data":"1aa8f4a0d9537e31ac4fddfccae375751acb5e1e4b8ad836fc4c3c272cf19d33"}
Jul 18 02:01:19.178991 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:19.178594    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-xp9s9"
Jul 18 02:01:19.195695 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:19.195652    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-xp9s9" podStartSLOduration=1.611653929 podStartE2EDuration="5.19563788s" totalImagesPullingTime="3.583983951s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:01:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:01:19.194359311 +0000 UTC m=+273.819395978" watchObservedRunningTime="2026-07-18 02:01:19.19563788 +0000 UTC m=+273.820674547"
Jul 18 02:01:25.184771 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:25.184740    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-xp9s9"
Jul 18 02:01:27.387885 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:27.387849    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-shll7"]
Jul 18 02:01:27.391167 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:27.391148    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-shll7"
Jul 18 02:01:27.393604 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:27.393583    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 18 02:01:27.394408 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:27.394386    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-zfthz\""
Jul 18 02:01:27.394488 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:27.394387    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 18 02:01:27.399846 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:27.399817    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-shll7"]
Jul 18 02:01:27.538210 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:27.538170    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6gbp\" (UniqueName: \"kubernetes.io/projected/f614ea92-0430-4e36-a587-d77ab9f14e92-kube-api-access-m6gbp\") pod \"openshift-lws-operator-fd8ccff4c-shll7\" (UID: \"f614ea92-0430-4e36-a587-d77ab9f14e92\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-shll7"
Jul 18 02:01:27.538210 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:27.538212    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f614ea92-0430-4e36-a587-d77ab9f14e92-tmp\") pod \"openshift-lws-operator-fd8ccff4c-shll7\" (UID: \"f614ea92-0430-4e36-a587-d77ab9f14e92\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-shll7"
Jul 18 02:01:27.639365 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:27.639267    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6gbp\" (UniqueName: \"kubernetes.io/projected/f614ea92-0430-4e36-a587-d77ab9f14e92-kube-api-access-m6gbp\") pod \"openshift-lws-operator-fd8ccff4c-shll7\" (UID: \"f614ea92-0430-4e36-a587-d77ab9f14e92\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-shll7"
Jul 18 02:01:27.639365 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:27.639315    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f614ea92-0430-4e36-a587-d77ab9f14e92-tmp\") pod \"openshift-lws-operator-fd8ccff4c-shll7\" (UID: \"f614ea92-0430-4e36-a587-d77ab9f14e92\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-shll7"
Jul 18 02:01:27.639766 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:27.639743    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f614ea92-0430-4e36-a587-d77ab9f14e92-tmp\") pod \"openshift-lws-operator-fd8ccff4c-shll7\" (UID: \"f614ea92-0430-4e36-a587-d77ab9f14e92\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-shll7"
Jul 18 02:01:27.647477 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:27.647455    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6gbp\" (UniqueName: \"kubernetes.io/projected/f614ea92-0430-4e36-a587-d77ab9f14e92-kube-api-access-m6gbp\") pod \"openshift-lws-operator-fd8ccff4c-shll7\" (UID: \"f614ea92-0430-4e36-a587-d77ab9f14e92\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-shll7"
Jul 18 02:01:27.701421 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:27.701387    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-shll7"
Jul 18 02:01:27.827209 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:27.827183    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-shll7"]
Jul 18 02:01:27.829365 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:01:27.829333    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf614ea92_0430_4e36_a587_d77ab9f14e92.slice/crio-f26e678d69d00145ed35d3fcf3711fb09ff7f44eaa0fc0c1a560bc0f89be1d82 WatchSource:0}: Error finding container f26e678d69d00145ed35d3fcf3711fb09ff7f44eaa0fc0c1a560bc0f89be1d82: Status 404 returned error can't find the container with id f26e678d69d00145ed35d3fcf3711fb09ff7f44eaa0fc0c1a560bc0f89be1d82
Jul 18 02:01:28.223401 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:28.223363    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-shll7" event={"ID":"f614ea92-0430-4e36-a587-d77ab9f14e92","Type":"ContainerStarted","Data":"f26e678d69d00145ed35d3fcf3711fb09ff7f44eaa0fc0c1a560bc0f89be1d82"}
Jul 18 02:01:31.238456 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:31.238408    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-shll7" event={"ID":"f614ea92-0430-4e36-a587-d77ab9f14e92","Type":"ContainerStarted","Data":"58edc51546d4a211b953612ca8d51b9d2e9d7edf7df9d48325021787ce07cfd4"}
Jul 18 02:01:31.256684 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:31.256630    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-shll7" podStartSLOduration=1.769765111 podStartE2EDuration="4.256613043s" totalImagesPullingTime="2.486847932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:01:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:01:31.255782437 +0000 UTC m=+285.880819118" watchObservedRunningTime="2026-07-18 02:01:31.256613043 +0000 UTC m=+285.881649709"
Jul 18 02:01:45.792498 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:45.792466    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-bfrdg_9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777/console-operator/1.log"
Jul 18 02:01:45.799501 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:45.799475    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-bfrdg_9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777/console-operator/1.log"
Jul 18 02:01:45.802769 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:45.802739    2576 kubelet.go:1648] "Image garbage collection succeeded"
Jul 18 02:01:50.684529 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:50.684498    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-bgz4c"]
Jul 18 02:01:50.686926 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:50.686912    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-bgz4c"
Jul 18 02:01:50.689929 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:50.689907    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 18 02:01:50.690031 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:50.690008    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-r5mvq\""
Jul 18 02:01:50.690296 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:50.690278    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 18 02:01:50.690366 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:50.690279    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 18 02:01:50.690421 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:50.690375    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 18 02:01:50.715702 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:50.715676    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-bgz4c"]
Jul 18 02:01:50.752184 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:50.752149    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc672fd7-3070-477a-9817-00d9fce807d0-tmp\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-bgz4c\" (UID: \"fc672fd7-3070-477a-9817-00d9fce807d0\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-bgz4c"
Jul 18 02:01:50.752360 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:50.752197    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fc672fd7-3070-477a-9817-00d9fce807d0-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-bgz4c\" (UID: \"fc672fd7-3070-477a-9817-00d9fce807d0\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-bgz4c"
Jul 18 02:01:50.752360 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:50.752269    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fc672fd7-3070-477a-9817-00d9fce807d0-manifests\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-bgz4c\" (UID: \"fc672fd7-3070-477a-9817-00d9fce807d0\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-bgz4c"
Jul 18 02:01:50.752360 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:50.752326    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fc672fd7-3070-477a-9817-00d9fce807d0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-bgz4c\" (UID: \"fc672fd7-3070-477a-9817-00d9fce807d0\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-bgz4c"
Jul 18 02:01:50.752360 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:50.752353    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzhbz\" (UniqueName: \"kubernetes.io/projected/fc672fd7-3070-477a-9817-00d9fce807d0-kube-api-access-mzhbz\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-bgz4c\" (UID: \"fc672fd7-3070-477a-9817-00d9fce807d0\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-bgz4c"
Jul 18 02:01:50.853469 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:50.853405    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc672fd7-3070-477a-9817-00d9fce807d0-tmp\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-bgz4c\" (UID: \"fc672fd7-3070-477a-9817-00d9fce807d0\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-bgz4c"
Jul 18 02:01:50.853670 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:50.853487    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fc672fd7-3070-477a-9817-00d9fce807d0-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-bgz4c\" (UID: \"fc672fd7-3070-477a-9817-00d9fce807d0\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-bgz4c"
Jul 18 02:01:50.853670 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:50.853520    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fc672fd7-3070-477a-9817-00d9fce807d0-manifests\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-bgz4c\" (UID: \"fc672fd7-3070-477a-9817-00d9fce807d0\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-bgz4c"
Jul 18 02:01:50.853670 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:50.853537    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fc672fd7-3070-477a-9817-00d9fce807d0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-bgz4c\" (UID: \"fc672fd7-3070-477a-9817-00d9fce807d0\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-bgz4c"
Jul 18 02:01:50.853670 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:50.853561    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzhbz\" (UniqueName: \"kubernetes.io/projected/fc672fd7-3070-477a-9817-00d9fce807d0-kube-api-access-mzhbz\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-bgz4c\" (UID: \"fc672fd7-3070-477a-9817-00d9fce807d0\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-bgz4c"
Jul 18 02:01:50.853993 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:50.853966    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fc672fd7-3070-477a-9817-00d9fce807d0-manifests\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-bgz4c\" (UID: \"fc672fd7-3070-477a-9817-00d9fce807d0\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-bgz4c"
Jul 18 02:01:50.855765 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:50.855740    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc672fd7-3070-477a-9817-00d9fce807d0-tmp\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-bgz4c\" (UID: \"fc672fd7-3070-477a-9817-00d9fce807d0\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-bgz4c"
Jul 18 02:01:50.855965 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:50.855949    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fc672fd7-3070-477a-9817-00d9fce807d0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-bgz4c\" (UID: \"fc672fd7-3070-477a-9817-00d9fce807d0\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-bgz4c"
Jul 18 02:01:50.856027 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:50.856006    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fc672fd7-3070-477a-9817-00d9fce807d0-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-bgz4c\" (UID: \"fc672fd7-3070-477a-9817-00d9fce807d0\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-bgz4c"
Jul 18 02:01:50.862412 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:50.862389    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzhbz\" (UniqueName: \"kubernetes.io/projected/fc672fd7-3070-477a-9817-00d9fce807d0-kube-api-access-mzhbz\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-bgz4c\" (UID: \"fc672fd7-3070-477a-9817-00d9fce807d0\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-bgz4c"
Jul 18 02:01:50.997582 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:50.997487    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-bgz4c"
Jul 18 02:01:51.151295 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:01:51.146144    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc672fd7_3070_477a_9817_00d9fce807d0.slice/crio-44d28cdef9f03d3b57bc8fe37060b2d45515fd9241d09d9e9bd7cda0ca68521e WatchSource:0}: Error finding container 44d28cdef9f03d3b57bc8fe37060b2d45515fd9241d09d9e9bd7cda0ca68521e: Status 404 returned error can't find the container with id 44d28cdef9f03d3b57bc8fe37060b2d45515fd9241d09d9e9bd7cda0ca68521e
Jul 18 02:01:51.151295 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:51.149118    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 02:01:51.155516 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:51.154315    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-bgz4c"]
Jul 18 02:01:51.332228 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:51.332138    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-bgz4c" event={"ID":"fc672fd7-3070-477a-9817-00d9fce807d0","Type":"ContainerStarted","Data":"44d28cdef9f03d3b57bc8fe37060b2d45515fd9241d09d9e9bd7cda0ca68521e"}
Jul 18 02:01:55.351518 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:55.351474    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-bgz4c" event={"ID":"fc672fd7-3070-477a-9817-00d9fce807d0","Type":"ContainerStarted","Data":"3effbe987cfde253c8ae1eb8fa9b49cf410f6d2d9beb2e7050e2b36a6d02ce3c"}
Jul 18 02:01:56.357686 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:56.357648    2576 generic.go:356] "Generic (PLEG): container finished" podID="fc672fd7-3070-477a-9817-00d9fce807d0" containerID="3effbe987cfde253c8ae1eb8fa9b49cf410f6d2d9beb2e7050e2b36a6d02ce3c" exitCode=0
Jul 18 02:01:56.358096 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:56.357733    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-bgz4c" event={"ID":"fc672fd7-3070-477a-9817-00d9fce807d0","Type":"ContainerDied","Data":"3effbe987cfde253c8ae1eb8fa9b49cf410f6d2d9beb2e7050e2b36a6d02ce3c"}
Jul 18 02:01:57.364225 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:57.364186    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-bgz4c" event={"ID":"fc672fd7-3070-477a-9817-00d9fce807d0","Type":"ContainerStarted","Data":"a077ca501327a34c4dca39b0ac3569e40565569a301304988a3b50b43d90044f"}
Jul 18 02:01:57.364682 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:57.364289    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-bgz4c"
Jul 18 02:01:57.384082 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:01:57.384032    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-bgz4c" podStartSLOduration=2.765812898 podStartE2EDuration="7.384015953s" totalImagesPullingTime="3.520993892s" totalInitContainerRuntime="1.097209163s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:01:50 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:01:57.383359484 +0000 UTC m=+312.008396153" watchObservedRunningTime="2026-07-18 02:01:57.384015953 +0000 UTC m=+312.009052620"
Jul 18 02:02:08.370598 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:08.370567    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-bgz4c"
Jul 18 02:02:10.749145 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:10.749105    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-pl52c"]
Jul 18 02:02:10.752483 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:10.752462    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-pl52c"
Jul 18 02:02:10.754932 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:10.754913    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-dtrgh\""
Jul 18 02:02:10.755027 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:10.754960    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 18 02:02:10.765931 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:10.765905    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-pl52c"]
Jul 18 02:02:10.830151 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:10.830110    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ea7eea6e-5b86-44c5-a721-c0823e3f305b-cert\") pod \"odh-model-controller-7998c67dc4-pl52c\" (UID: \"ea7eea6e-5b86-44c5-a721-c0823e3f305b\") " pod="opendatahub/odh-model-controller-7998c67dc4-pl52c"
Jul 18 02:02:10.830151 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:10.830151    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksx7s\" (UniqueName: \"kubernetes.io/projected/ea7eea6e-5b86-44c5-a721-c0823e3f305b-kube-api-access-ksx7s\") pod \"odh-model-controller-7998c67dc4-pl52c\" (UID: \"ea7eea6e-5b86-44c5-a721-c0823e3f305b\") " pod="opendatahub/odh-model-controller-7998c67dc4-pl52c"
Jul 18 02:02:10.931029 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:10.930991    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ea7eea6e-5b86-44c5-a721-c0823e3f305b-cert\") pod \"odh-model-controller-7998c67dc4-pl52c\" (UID: \"ea7eea6e-5b86-44c5-a721-c0823e3f305b\") " pod="opendatahub/odh-model-controller-7998c67dc4-pl52c"
Jul 18 02:02:10.931029 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:10.931030    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksx7s\" (UniqueName: \"kubernetes.io/projected/ea7eea6e-5b86-44c5-a721-c0823e3f305b-kube-api-access-ksx7s\") pod \"odh-model-controller-7998c67dc4-pl52c\" (UID: \"ea7eea6e-5b86-44c5-a721-c0823e3f305b\") " pod="opendatahub/odh-model-controller-7998c67dc4-pl52c"
Jul 18 02:02:10.933623 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:10.933599    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ea7eea6e-5b86-44c5-a721-c0823e3f305b-cert\") pod \"odh-model-controller-7998c67dc4-pl52c\" (UID: \"ea7eea6e-5b86-44c5-a721-c0823e3f305b\") " pod="opendatahub/odh-model-controller-7998c67dc4-pl52c"
Jul 18 02:02:10.938944 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:10.938916    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksx7s\" (UniqueName: \"kubernetes.io/projected/ea7eea6e-5b86-44c5-a721-c0823e3f305b-kube-api-access-ksx7s\") pod \"odh-model-controller-7998c67dc4-pl52c\" (UID: \"ea7eea6e-5b86-44c5-a721-c0823e3f305b\") " pod="opendatahub/odh-model-controller-7998c67dc4-pl52c"
Jul 18 02:02:11.062631 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:11.062536    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-pl52c"
Jul 18 02:02:11.194135 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:11.193934    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-pl52c"]
Jul 18 02:02:11.196910 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:02:11.196884    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea7eea6e_5b86_44c5_a721_c0823e3f305b.slice/crio-28be08eceb64fa714c0c698cd59d813c537460116679044e6a71a95ec757bf19 WatchSource:0}: Error finding container 28be08eceb64fa714c0c698cd59d813c537460116679044e6a71a95ec757bf19: Status 404 returned error can't find the container with id 28be08eceb64fa714c0c698cd59d813c537460116679044e6a71a95ec757bf19
Jul 18 02:02:11.311401 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:11.311372    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-c47pl"]
Jul 18 02:02:11.315843 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:11.315802    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-c47pl"
Jul 18 02:02:11.318063 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:11.318045    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 18 02:02:11.326657 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:11.326632    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-c47pl"]
Jul 18 02:02:11.422832 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:11.422800    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-pl52c" event={"ID":"ea7eea6e-5b86-44c5-a721-c0823e3f305b","Type":"ContainerStarted","Data":"28be08eceb64fa714c0c698cd59d813c537460116679044e6a71a95ec757bf19"}
Jul 18 02:02:11.435340 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:11.435304    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c123cde0-8a31-4ca0-823d-9a777092f96c-tls-certs\") pod \"model-serving-api-54d89bd5fc-c47pl\" (UID: \"c123cde0-8a31-4ca0-823d-9a777092f96c\") " pod="opendatahub/model-serving-api-54d89bd5fc-c47pl"
Jul 18 02:02:11.435340 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:11.435338    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l9l2\" (UniqueName: \"kubernetes.io/projected/c123cde0-8a31-4ca0-823d-9a777092f96c-kube-api-access-2l9l2\") pod \"model-serving-api-54d89bd5fc-c47pl\" (UID: \"c123cde0-8a31-4ca0-823d-9a777092f96c\") " pod="opendatahub/model-serving-api-54d89bd5fc-c47pl"
Jul 18 02:02:11.535978 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:11.535938    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c123cde0-8a31-4ca0-823d-9a777092f96c-tls-certs\") pod \"model-serving-api-54d89bd5fc-c47pl\" (UID: \"c123cde0-8a31-4ca0-823d-9a777092f96c\") " pod="opendatahub/model-serving-api-54d89bd5fc-c47pl"
Jul 18 02:02:11.536166 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:11.535986    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2l9l2\" (UniqueName: \"kubernetes.io/projected/c123cde0-8a31-4ca0-823d-9a777092f96c-kube-api-access-2l9l2\") pod \"model-serving-api-54d89bd5fc-c47pl\" (UID: \"c123cde0-8a31-4ca0-823d-9a777092f96c\") " pod="opendatahub/model-serving-api-54d89bd5fc-c47pl"
Jul 18 02:02:11.538319 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:11.538299    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c123cde0-8a31-4ca0-823d-9a777092f96c-tls-certs\") pod \"model-serving-api-54d89bd5fc-c47pl\" (UID: \"c123cde0-8a31-4ca0-823d-9a777092f96c\") " pod="opendatahub/model-serving-api-54d89bd5fc-c47pl"
Jul 18 02:02:11.543646 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:11.543626    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l9l2\" (UniqueName: \"kubernetes.io/projected/c123cde0-8a31-4ca0-823d-9a777092f96c-kube-api-access-2l9l2\") pod \"model-serving-api-54d89bd5fc-c47pl\" (UID: \"c123cde0-8a31-4ca0-823d-9a777092f96c\") " pod="opendatahub/model-serving-api-54d89bd5fc-c47pl"
Jul 18 02:02:11.626365 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:11.626272    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-c47pl"
Jul 18 02:02:11.751514 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:11.751490    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-c47pl"]
Jul 18 02:02:11.753830 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:02:11.753803    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc123cde0_8a31_4ca0_823d_9a777092f96c.slice/crio-1851effa9398475376477c7861b367e20c8e405626c7c7184fa30a684a2ee696 WatchSource:0}: Error finding container 1851effa9398475376477c7861b367e20c8e405626c7c7184fa30a684a2ee696: Status 404 returned error can't find the container with id 1851effa9398475376477c7861b367e20c8e405626c7c7184fa30a684a2ee696
Jul 18 02:02:12.429808 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:12.429727    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-c47pl" event={"ID":"c123cde0-8a31-4ca0-823d-9a777092f96c","Type":"ContainerStarted","Data":"1851effa9398475376477c7861b367e20c8e405626c7c7184fa30a684a2ee696"}
Jul 18 02:02:14.439791 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:14.439754    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-pl52c" event={"ID":"ea7eea6e-5b86-44c5-a721-c0823e3f305b","Type":"ContainerStarted","Data":"9fc5cd6bbad4e67fb0d791303c5a42ec6cde1f02422ef88ee40f1b25d57063c0"}
Jul 18 02:02:14.440244 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:14.439846    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-pl52c"
Jul 18 02:02:14.446368 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:14.446337    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-c47pl" event={"ID":"c123cde0-8a31-4ca0-823d-9a777092f96c","Type":"ContainerStarted","Data":"0fca4cf680d28200b167347f9e370534487b31beac8e0fbe855cf5bccb046220"}
Jul 18 02:02:14.446561 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:14.446513    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-c47pl"
Jul 18 02:02:14.474400 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:14.474348    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-c47pl" podStartSLOduration=0.917909809 podStartE2EDuration="3.474331043s" totalImagesPullingTime="2.556421234s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:02:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:02:14.472884216 +0000 UTC m=+329.097920882" watchObservedRunningTime="2026-07-18 02:02:14.474331043 +0000 UTC m=+329.099367709"
Jul 18 02:02:14.474938 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:14.474906    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-pl52c" podStartSLOduration=1.358775188 podStartE2EDuration="4.474897872s" totalImagesPullingTime="3.116122684s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:02:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:02:14.458414443 +0000 UTC m=+329.083451113" watchObservedRunningTime="2026-07-18 02:02:14.474897872 +0000 UTC m=+329.099934539"
Jul 18 02:02:19.784554 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:19.784515    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-jvkvx"]
Jul 18 02:02:19.787700 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:19.787676    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jvkvx"
Jul 18 02:02:19.791072 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:19.791029    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 18 02:02:19.791197 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:19.791113    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-78kdm\""
Jul 18 02:02:19.809493 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:19.809466    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-gx5mr"]
Jul 18 02:02:19.812726 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:19.812710    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gx5mr"
Jul 18 02:02:19.816638 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:19.816614    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-j9cbc\""
Jul 18 02:02:19.817277 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:19.817256    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-jvkvx"]
Jul 18 02:02:19.817581 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:19.817539    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 18 02:02:19.844502 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:19.844468    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-gx5mr"]
Jul 18 02:02:19.915416 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:19.915368    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f9ad3be-1156-45cd-924c-a3cb36bdef79-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-gx5mr\" (UID: \"2f9ad3be-1156-45cd-924c-a3cb36bdef79\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gx5mr"
Jul 18 02:02:19.915416 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:19.915420    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6f7vd\" (UniqueName: \"kubernetes.io/projected/47c4d24f-e732-45d7-8fd9-51bae2a117a8-kube-api-access-6f7vd\") pod \"kserve-controller-manager-d6f4b65b8-jvkvx\" (UID: \"47c4d24f-e732-45d7-8fd9-51bae2a117a8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jvkvx"
Jul 18 02:02:19.915679 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:19.915460    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pt2sc\" (UniqueName: \"kubernetes.io/projected/2f9ad3be-1156-45cd-924c-a3cb36bdef79-kube-api-access-pt2sc\") pod \"llmisvc-controller-manager-777c6bd7d5-gx5mr\" (UID: \"2f9ad3be-1156-45cd-924c-a3cb36bdef79\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gx5mr"
Jul 18 02:02:19.915679 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:19.915540    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/47c4d24f-e732-45d7-8fd9-51bae2a117a8-cert\") pod \"kserve-controller-manager-d6f4b65b8-jvkvx\" (UID: \"47c4d24f-e732-45d7-8fd9-51bae2a117a8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jvkvx"
Jul 18 02:02:20.016312 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:20.016271    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/47c4d24f-e732-45d7-8fd9-51bae2a117a8-cert\") pod \"kserve-controller-manager-d6f4b65b8-jvkvx\" (UID: \"47c4d24f-e732-45d7-8fd9-51bae2a117a8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jvkvx"
Jul 18 02:02:20.016510 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:20.016347    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f9ad3be-1156-45cd-924c-a3cb36bdef79-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-gx5mr\" (UID: \"2f9ad3be-1156-45cd-924c-a3cb36bdef79\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gx5mr"
Jul 18 02:02:20.016510 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:20.016388    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6f7vd\" (UniqueName: \"kubernetes.io/projected/47c4d24f-e732-45d7-8fd9-51bae2a117a8-kube-api-access-6f7vd\") pod \"kserve-controller-manager-d6f4b65b8-jvkvx\" (UID: \"47c4d24f-e732-45d7-8fd9-51bae2a117a8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jvkvx"
Jul 18 02:02:20.016510 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:20.016416    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt2sc\" (UniqueName: \"kubernetes.io/projected/2f9ad3be-1156-45cd-924c-a3cb36bdef79-kube-api-access-pt2sc\") pod \"llmisvc-controller-manager-777c6bd7d5-gx5mr\" (UID: \"2f9ad3be-1156-45cd-924c-a3cb36bdef79\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gx5mr"
Jul 18 02:02:20.016510 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:02:20.016452    2576 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 18 02:02:20.016510 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:02:20.016494    2576 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 18 02:02:20.016699 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:02:20.016534    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47c4d24f-e732-45d7-8fd9-51bae2a117a8-cert podName:47c4d24f-e732-45d7-8fd9-51bae2a117a8 nodeName:}" failed. No retries permitted until 2026-07-18 02:02:20.516512859 +0000 UTC m=+335.141549518 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/47c4d24f-e732-45d7-8fd9-51bae2a117a8-cert") pod "kserve-controller-manager-d6f4b65b8-jvkvx" (UID: "47c4d24f-e732-45d7-8fd9-51bae2a117a8") : secret "kserve-webhook-server-cert" not found
Jul 18 02:02:20.016699 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:02:20.016551    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f9ad3be-1156-45cd-924c-a3cb36bdef79-cert podName:2f9ad3be-1156-45cd-924c-a3cb36bdef79 nodeName:}" failed. No retries permitted until 2026-07-18 02:02:20.516543082 +0000 UTC m=+335.141579727 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2f9ad3be-1156-45cd-924c-a3cb36bdef79-cert") pod "llmisvc-controller-manager-777c6bd7d5-gx5mr" (UID: "2f9ad3be-1156-45cd-924c-a3cb36bdef79") : secret "llmisvc-webhook-server-cert" not found
Jul 18 02:02:20.025265 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:20.025232    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pt2sc\" (UniqueName: \"kubernetes.io/projected/2f9ad3be-1156-45cd-924c-a3cb36bdef79-kube-api-access-pt2sc\") pod \"llmisvc-controller-manager-777c6bd7d5-gx5mr\" (UID: \"2f9ad3be-1156-45cd-924c-a3cb36bdef79\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gx5mr"
Jul 18 02:02:20.025265 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:20.025237    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6f7vd\" (UniqueName: \"kubernetes.io/projected/47c4d24f-e732-45d7-8fd9-51bae2a117a8-kube-api-access-6f7vd\") pod \"kserve-controller-manager-d6f4b65b8-jvkvx\" (UID: \"47c4d24f-e732-45d7-8fd9-51bae2a117a8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jvkvx"
Jul 18 02:02:20.521300 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:20.521253    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f9ad3be-1156-45cd-924c-a3cb36bdef79-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-gx5mr\" (UID: \"2f9ad3be-1156-45cd-924c-a3cb36bdef79\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gx5mr"
Jul 18 02:02:20.521535 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:20.521327    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/47c4d24f-e732-45d7-8fd9-51bae2a117a8-cert\") pod \"kserve-controller-manager-d6f4b65b8-jvkvx\" (UID: \"47c4d24f-e732-45d7-8fd9-51bae2a117a8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jvkvx"
Jul 18 02:02:20.523919 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:20.523888    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/47c4d24f-e732-45d7-8fd9-51bae2a117a8-cert\") pod \"kserve-controller-manager-d6f4b65b8-jvkvx\" (UID: \"47c4d24f-e732-45d7-8fd9-51bae2a117a8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jvkvx"
Jul 18 02:02:20.524056 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:20.523953    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f9ad3be-1156-45cd-924c-a3cb36bdef79-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-gx5mr\" (UID: \"2f9ad3be-1156-45cd-924c-a3cb36bdef79\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gx5mr"
Jul 18 02:02:20.698152 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:20.698115    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jvkvx"
Jul 18 02:02:20.723573 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:20.723536    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gx5mr"
Jul 18 02:02:20.865865 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:20.865837    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-jvkvx"]
Jul 18 02:02:20.867567 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:02:20.867540    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47c4d24f_e732_45d7_8fd9_51bae2a117a8.slice/crio-c4bf75fce10e7d231d4eb27253f057bc04c625d2bfc98329b319c0bfdca174bf WatchSource:0}: Error finding container c4bf75fce10e7d231d4eb27253f057bc04c625d2bfc98329b319c0bfdca174bf: Status 404 returned error can't find the container with id c4bf75fce10e7d231d4eb27253f057bc04c625d2bfc98329b319c0bfdca174bf
Jul 18 02:02:20.874300 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:20.874275    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-gx5mr"]
Jul 18 02:02:20.876496 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:02:20.876474    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2f9ad3be_1156_45cd_924c_a3cb36bdef79.slice/crio-a938f89a30fa82fe21c61ea3189c62a43f34f70d1fc61e1fe97e46ffbf5103eb WatchSource:0}: Error finding container a938f89a30fa82fe21c61ea3189c62a43f34f70d1fc61e1fe97e46ffbf5103eb: Status 404 returned error can't find the container with id a938f89a30fa82fe21c61ea3189c62a43f34f70d1fc61e1fe97e46ffbf5103eb
Jul 18 02:02:21.112534 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:21.112455    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-748465cbd5-42p4f"]
Jul 18 02:02:21.117065 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:21.117047    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-748465cbd5-42p4f"
Jul 18 02:02:21.120006 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:21.119977    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 18 02:02:21.120006 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:21.119995    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 18 02:02:21.120193 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:21.119995    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-7gsxn\""
Jul 18 02:02:21.129080 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:21.129056    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-748465cbd5-42p4f"]
Jul 18 02:02:21.228258 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:21.228219    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/635497de-6dc6-4d91-bd8a-76287723a4e5-tmp\") pod \"kube-auth-proxy-748465cbd5-42p4f\" (UID: \"635497de-6dc6-4d91-bd8a-76287723a4e5\") " pod="openshift-ingress/kube-auth-proxy-748465cbd5-42p4f"
Jul 18 02:02:21.228463 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:21.228264    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/635497de-6dc6-4d91-bd8a-76287723a4e5-tls-certs\") pod \"kube-auth-proxy-748465cbd5-42p4f\" (UID: \"635497de-6dc6-4d91-bd8a-76287723a4e5\") " pod="openshift-ingress/kube-auth-proxy-748465cbd5-42p4f"
Jul 18 02:02:21.228463 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:21.228299    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b52bs\" (UniqueName: \"kubernetes.io/projected/635497de-6dc6-4d91-bd8a-76287723a4e5-kube-api-access-b52bs\") pod \"kube-auth-proxy-748465cbd5-42p4f\" (UID: \"635497de-6dc6-4d91-bd8a-76287723a4e5\") " pod="openshift-ingress/kube-auth-proxy-748465cbd5-42p4f"
Jul 18 02:02:21.329566 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:21.329529    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/635497de-6dc6-4d91-bd8a-76287723a4e5-tmp\") pod \"kube-auth-proxy-748465cbd5-42p4f\" (UID: \"635497de-6dc6-4d91-bd8a-76287723a4e5\") " pod="openshift-ingress/kube-auth-proxy-748465cbd5-42p4f"
Jul 18 02:02:21.329566 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:21.329568    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/635497de-6dc6-4d91-bd8a-76287723a4e5-tls-certs\") pod \"kube-auth-proxy-748465cbd5-42p4f\" (UID: \"635497de-6dc6-4d91-bd8a-76287723a4e5\") " pod="openshift-ingress/kube-auth-proxy-748465cbd5-42p4f"
Jul 18 02:02:21.329769 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:21.329591    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b52bs\" (UniqueName: \"kubernetes.io/projected/635497de-6dc6-4d91-bd8a-76287723a4e5-kube-api-access-b52bs\") pod \"kube-auth-proxy-748465cbd5-42p4f\" (UID: \"635497de-6dc6-4d91-bd8a-76287723a4e5\") " pod="openshift-ingress/kube-auth-proxy-748465cbd5-42p4f"
Jul 18 02:02:21.329769 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:02:21.329686    2576 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 18 02:02:21.329769 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:02:21.329754    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/635497de-6dc6-4d91-bd8a-76287723a4e5-tls-certs podName:635497de-6dc6-4d91-bd8a-76287723a4e5 nodeName:}" failed. No retries permitted until 2026-07-18 02:02:21.82973884 +0000 UTC m=+336.454775490 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/635497de-6dc6-4d91-bd8a-76287723a4e5-tls-certs") pod "kube-auth-proxy-748465cbd5-42p4f" (UID: "635497de-6dc6-4d91-bd8a-76287723a4e5") : secret "kube-auth-proxy-tls" not found
Jul 18 02:02:21.331750 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:21.331724    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/635497de-6dc6-4d91-bd8a-76287723a4e5-tmp\") pod \"kube-auth-proxy-748465cbd5-42p4f\" (UID: \"635497de-6dc6-4d91-bd8a-76287723a4e5\") " pod="openshift-ingress/kube-auth-proxy-748465cbd5-42p4f"
Jul 18 02:02:21.354207 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:21.354173    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b52bs\" (UniqueName: \"kubernetes.io/projected/635497de-6dc6-4d91-bd8a-76287723a4e5-kube-api-access-b52bs\") pod \"kube-auth-proxy-748465cbd5-42p4f\" (UID: \"635497de-6dc6-4d91-bd8a-76287723a4e5\") " pod="openshift-ingress/kube-auth-proxy-748465cbd5-42p4f"
Jul 18 02:02:21.477475 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:21.477418    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gx5mr" event={"ID":"2f9ad3be-1156-45cd-924c-a3cb36bdef79","Type":"ContainerStarted","Data":"a938f89a30fa82fe21c61ea3189c62a43f34f70d1fc61e1fe97e46ffbf5103eb"}
Jul 18 02:02:21.478568 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:21.478547    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jvkvx" event={"ID":"47c4d24f-e732-45d7-8fd9-51bae2a117a8","Type":"ContainerStarted","Data":"c4bf75fce10e7d231d4eb27253f057bc04c625d2bfc98329b319c0bfdca174bf"}
Jul 18 02:02:21.833610 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:21.833534    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/635497de-6dc6-4d91-bd8a-76287723a4e5-tls-certs\") pod \"kube-auth-proxy-748465cbd5-42p4f\" (UID: \"635497de-6dc6-4d91-bd8a-76287723a4e5\") " pod="openshift-ingress/kube-auth-proxy-748465cbd5-42p4f"
Jul 18 02:02:21.835952 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:21.835927    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/635497de-6dc6-4d91-bd8a-76287723a4e5-tls-certs\") pod \"kube-auth-proxy-748465cbd5-42p4f\" (UID: \"635497de-6dc6-4d91-bd8a-76287723a4e5\") " pod="openshift-ingress/kube-auth-proxy-748465cbd5-42p4f"
Jul 18 02:02:22.027144 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:22.027110    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-748465cbd5-42p4f"
Jul 18 02:02:22.332037 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:22.331996    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-748465cbd5-42p4f"]
Jul 18 02:02:22.335647 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:02:22.335609    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod635497de_6dc6_4d91_bd8a_76287723a4e5.slice/crio-f5119c2c4905229da8aefe2f744aa80187181d5091ff820dd7f17e818da86a43 WatchSource:0}: Error finding container f5119c2c4905229da8aefe2f744aa80187181d5091ff820dd7f17e818da86a43: Status 404 returned error can't find the container with id f5119c2c4905229da8aefe2f744aa80187181d5091ff820dd7f17e818da86a43
Jul 18 02:02:22.492842 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:22.492750    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-748465cbd5-42p4f" event={"ID":"635497de-6dc6-4d91-bd8a-76287723a4e5","Type":"ContainerStarted","Data":"f5119c2c4905229da8aefe2f744aa80187181d5091ff820dd7f17e818da86a43"}
Jul 18 02:02:22.723726 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:22.723690    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbvb8"]
Jul 18 02:02:22.727564 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:22.727538    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbvb8"
Jul 18 02:02:22.730220 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:22.730089    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 18 02:02:22.730220 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:22.730105    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 18 02:02:22.730220 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:22.730105    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 18 02:02:22.730220 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:22.730144    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 18 02:02:22.730517 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:22.730240    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-v64qt\""
Jul 18 02:02:22.743123 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:22.743058    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbvb8"]
Jul 18 02:02:22.845664 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:22.845621    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/30e6fba1-ee78-4462-aace-3adb6da9919b-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-gbvb8\" (UID: \"30e6fba1-ee78-4462-aace-3adb6da9919b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbvb8"
Jul 18 02:02:22.845880 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:22.845685    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/30e6fba1-ee78-4462-aace-3adb6da9919b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-gbvb8\" (UID: \"30e6fba1-ee78-4462-aace-3adb6da9919b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbvb8"
Jul 18 02:02:22.845880 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:22.845778    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30e6fba1-ee78-4462-aace-3adb6da9919b-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-gbvb8\" (UID: \"30e6fba1-ee78-4462-aace-3adb6da9919b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbvb8"
Jul 18 02:02:22.846012 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:22.845890    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/30e6fba1-ee78-4462-aace-3adb6da9919b-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-gbvb8\" (UID: \"30e6fba1-ee78-4462-aace-3adb6da9919b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbvb8"
Jul 18 02:02:22.846012 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:22.845944    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/30e6fba1-ee78-4462-aace-3adb6da9919b-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-gbvb8\" (UID: \"30e6fba1-ee78-4462-aace-3adb6da9919b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbvb8"
Jul 18 02:02:22.846116 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:22.846022    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/30e6fba1-ee78-4462-aace-3adb6da9919b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-gbvb8\" (UID: \"30e6fba1-ee78-4462-aace-3adb6da9919b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbvb8"
Jul 18 02:02:22.846116 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:22.846049    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9v28n\" (UniqueName: \"kubernetes.io/projected/30e6fba1-ee78-4462-aace-3adb6da9919b-kube-api-access-9v28n\") pod \"istiod-openshift-gateway-94bb8fbfd-gbvb8\" (UID: \"30e6fba1-ee78-4462-aace-3adb6da9919b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbvb8"
Jul 18 02:02:22.947058 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:22.947015    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30e6fba1-ee78-4462-aace-3adb6da9919b-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-gbvb8\" (UID: \"30e6fba1-ee78-4462-aace-3adb6da9919b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbvb8"
Jul 18 02:02:22.947247 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:22.947114    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/30e6fba1-ee78-4462-aace-3adb6da9919b-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-gbvb8\" (UID: \"30e6fba1-ee78-4462-aace-3adb6da9919b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbvb8"
Jul 18 02:02:22.947247 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:22.947141    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/30e6fba1-ee78-4462-aace-3adb6da9919b-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-gbvb8\" (UID: \"30e6fba1-ee78-4462-aace-3adb6da9919b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbvb8"
Jul 18 02:02:22.947247 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:22.947185    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/30e6fba1-ee78-4462-aace-3adb6da9919b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-gbvb8\" (UID: \"30e6fba1-ee78-4462-aace-3adb6da9919b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbvb8"
Jul 18 02:02:22.947247 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:22.947209    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v28n\" (UniqueName: \"kubernetes.io/projected/30e6fba1-ee78-4462-aace-3adb6da9919b-kube-api-access-9v28n\") pod \"istiod-openshift-gateway-94bb8fbfd-gbvb8\" (UID: \"30e6fba1-ee78-4462-aace-3adb6da9919b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbvb8"
Jul 18 02:02:22.947487 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:22.947253    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/30e6fba1-ee78-4462-aace-3adb6da9919b-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-gbvb8\" (UID: \"30e6fba1-ee78-4462-aace-3adb6da9919b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbvb8"
Jul 18 02:02:22.947487 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:22.947292    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/30e6fba1-ee78-4462-aace-3adb6da9919b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-gbvb8\" (UID: \"30e6fba1-ee78-4462-aace-3adb6da9919b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbvb8"
Jul 18 02:02:22.948048 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:22.947984    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/30e6fba1-ee78-4462-aace-3adb6da9919b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-gbvb8\" (UID: \"30e6fba1-ee78-4462-aace-3adb6da9919b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbvb8"
Jul 18 02:02:22.950158 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:22.950131    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/30e6fba1-ee78-4462-aace-3adb6da9919b-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-gbvb8\" (UID: \"30e6fba1-ee78-4462-aace-3adb6da9919b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbvb8"
Jul 18 02:02:22.950333 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:22.950307    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/30e6fba1-ee78-4462-aace-3adb6da9919b-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-gbvb8\" (UID: \"30e6fba1-ee78-4462-aace-3adb6da9919b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbvb8"
Jul 18 02:02:22.950510 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:22.950490    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/30e6fba1-ee78-4462-aace-3adb6da9919b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-gbvb8\" (UID: \"30e6fba1-ee78-4462-aace-3adb6da9919b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbvb8"
Jul 18 02:02:22.950813 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:22.950792    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/30e6fba1-ee78-4462-aace-3adb6da9919b-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-gbvb8\" (UID: \"30e6fba1-ee78-4462-aace-3adb6da9919b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbvb8"
Jul 18 02:02:22.959899 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:22.959873    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30e6fba1-ee78-4462-aace-3adb6da9919b-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-gbvb8\" (UID: \"30e6fba1-ee78-4462-aace-3adb6da9919b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbvb8"
Jul 18 02:02:22.960129 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:22.960097    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9v28n\" (UniqueName: \"kubernetes.io/projected/30e6fba1-ee78-4462-aace-3adb6da9919b-kube-api-access-9v28n\") pod \"istiod-openshift-gateway-94bb8fbfd-gbvb8\" (UID: \"30e6fba1-ee78-4462-aace-3adb6da9919b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbvb8"
Jul 18 02:02:23.039441 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:23.039310    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbvb8"
Jul 18 02:02:23.212882 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:23.212805    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbvb8"]
Jul 18 02:02:23.827241 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:02:23.827202    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30e6fba1_ee78_4462_aace_3adb6da9919b.slice/crio-fb36ba740e667b01794edd5484826393d8712d1ea5b9ee843524bd3a3ebf6713 WatchSource:0}: Error finding container fb36ba740e667b01794edd5484826393d8712d1ea5b9ee843524bd3a3ebf6713: Status 404 returned error can't find the container with id fb36ba740e667b01794edd5484826393d8712d1ea5b9ee843524bd3a3ebf6713
Jul 18 02:02:24.505605 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:24.505562    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbvb8" event={"ID":"30e6fba1-ee78-4462-aace-3adb6da9919b","Type":"ContainerStarted","Data":"fb36ba740e667b01794edd5484826393d8712d1ea5b9ee843524bd3a3ebf6713"}
Jul 18 02:02:25.457218 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:25.457188    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-c47pl"
Jul 18 02:02:25.469666 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:25.469286    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-pl52c"
Jul 18 02:02:25.515700 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:25.515650    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gx5mr" event={"ID":"2f9ad3be-1156-45cd-924c-a3cb36bdef79","Type":"ContainerStarted","Data":"ddcf346733597005ee86a79dd95edca5050fd86abd717887a9ccb2068feb64b8"}
Jul 18 02:02:25.516906 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:25.516550    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gx5mr"
Jul 18 02:02:25.518564 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:25.518500    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jvkvx" event={"ID":"47c4d24f-e732-45d7-8fd9-51bae2a117a8","Type":"ContainerStarted","Data":"a76838e4d4b21689f832c745e32deae8a578f88c27a7bea3d615b69432c62d72"}
Jul 18 02:02:25.519025 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:25.519004    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jvkvx"
Jul 18 02:02:25.522780 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:25.522160    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-748465cbd5-42p4f" event={"ID":"635497de-6dc6-4d91-bd8a-76287723a4e5","Type":"ContainerStarted","Data":"691261d5b56fb7cf85fde0be9cfc3e487c177c69ed759c9dec4537dcd368fdea"}
Jul 18 02:02:25.535793 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:25.535735    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gx5mr" podStartSLOduration=2.336287828 podStartE2EDuration="6.535713917s" totalImagesPullingTime="4.199426089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:02:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:02:25.533254933 +0000 UTC m=+340.158291602" watchObservedRunningTime="2026-07-18 02:02:25.535713917 +0000 UTC m=+340.160750574"
Jul 18 02:02:25.551787 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:25.551728    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-748465cbd5-42p4f" podStartSLOduration=1.807909521 podStartE2EDuration="4.551709891s" totalImagesPullingTime="2.74380037s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:02:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:02:25.548852933 +0000 UTC m=+340.173889601" watchObservedRunningTime="2026-07-18 02:02:25.551709891 +0000 UTC m=+340.176746558"
Jul 18 02:02:25.566816 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:25.566754    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jvkvx" podStartSLOduration=2.360688976 podStartE2EDuration="6.566730323s" totalImagesPullingTime="4.206041347s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:02:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:02:25.564160143 +0000 UTC m=+340.189196812" watchObservedRunningTime="2026-07-18 02:02:25.566730323 +0000 UTC m=+340.191766991"
Jul 18 02:02:27.534229 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:27.534182    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 18 02:02:27.534554 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:27.534258    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 18 02:02:28.541720 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:28.541683    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbvb8" event={"ID":"30e6fba1-ee78-4462-aace-3adb6da9919b","Type":"ContainerStarted","Data":"30e5fcdebeed6a483948307ec46642aea4370692f91fd06434a8c606a3d5dbe0"}
Jul 18 02:02:28.542192 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:28.541828    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbvb8"
Jul 18 02:02:28.543500 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:28.543467    2576 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-gbvb8 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 18 02:02:28.543620 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:28.543522    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbvb8" podUID="30e6fba1-ee78-4462-aace-3adb6da9919b" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 18 02:02:28.563070 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:28.563023    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbvb8" podStartSLOduration=2.858842647 podStartE2EDuration="6.56300715s" totalImagesPullingTime="3.704164503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:02:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:02:28.561673998 +0000 UTC m=+343.186710664" watchObservedRunningTime="2026-07-18 02:02:28.56300715 +0000 UTC m=+343.188043818"
Jul 18 02:02:29.547725 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:29.547693    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbvb8"
Jul 18 02:02:57.538362 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:57.538276    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gx5mr"
Jul 18 02:02:57.541349 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:02:57.541328    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jvkvx"
Jul 18 02:03:50.449580 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:03:50.449546    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-hlwwp"]
Jul 18 02:03:50.452979 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:03:50.452960    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hlwwp"
Jul 18 02:03:50.457150 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:03:50.457123    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 18 02:03:50.457842 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:03:50.457819    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 18 02:03:50.457943 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:03:50.457828    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 18 02:03:50.457943 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:03:50.457826    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-6kv55\""
Jul 18 02:03:50.464724 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:03:50.464703    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-hlwwp"]
Jul 18 02:03:50.501625 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:03:50.501594    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvxwp\" (UniqueName: \"kubernetes.io/projected/09763225-37ca-40e1-b910-8c7e45a3237e-kube-api-access-lvxwp\") pod \"dns-operator-controller-manager-864dd78c4c-hlwwp\" (UID: \"09763225-37ca-40e1-b910-8c7e45a3237e\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hlwwp"
Jul 18 02:03:50.602532 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:03:50.602501    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvxwp\" (UniqueName: \"kubernetes.io/projected/09763225-37ca-40e1-b910-8c7e45a3237e-kube-api-access-lvxwp\") pod \"dns-operator-controller-manager-864dd78c4c-hlwwp\" (UID: \"09763225-37ca-40e1-b910-8c7e45a3237e\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hlwwp"
Jul 18 02:03:50.613793 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:03:50.613758    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvxwp\" (UniqueName: \"kubernetes.io/projected/09763225-37ca-40e1-b910-8c7e45a3237e-kube-api-access-lvxwp\") pod \"dns-operator-controller-manager-864dd78c4c-hlwwp\" (UID: \"09763225-37ca-40e1-b910-8c7e45a3237e\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hlwwp"
Jul 18 02:03:50.763499 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:03:50.763393    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hlwwp"
Jul 18 02:03:50.904402 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:03:50.904370    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-hlwwp"]
Jul 18 02:03:50.908201 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:03:50.908165    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09763225_37ca_40e1_b910_8c7e45a3237e.slice/crio-1f2422c36f41bad9f231342a1391913c2f49bc05af9f7e3bd20425e9f3e05786 WatchSource:0}: Error finding container 1f2422c36f41bad9f231342a1391913c2f49bc05af9f7e3bd20425e9f3e05786: Status 404 returned error can't find the container with id 1f2422c36f41bad9f231342a1391913c2f49bc05af9f7e3bd20425e9f3e05786
Jul 18 02:03:50.922148 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:03:50.922116    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hlwwp" event={"ID":"09763225-37ca-40e1-b910-8c7e45a3237e","Type":"ContainerStarted","Data":"1f2422c36f41bad9f231342a1391913c2f49bc05af9f7e3bd20425e9f3e05786"}
Jul 18 02:03:52.937481 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:03:52.937445    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hlwwp" event={"ID":"09763225-37ca-40e1-b910-8c7e45a3237e","Type":"ContainerStarted","Data":"0f218d5852e5ef6c4e5a60c7783d0e0c3b655cd1f70efbff67885632266c8528"}
Jul 18 02:03:52.937874 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:03:52.937578    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hlwwp"
Jul 18 02:03:52.957698 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:03:52.957646    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hlwwp" podStartSLOduration=1.107993176 podStartE2EDuration="2.957630643s" totalImagesPullingTime="1.849637467s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:03:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:03:52.956015721 +0000 UTC m=+427.581052391" watchObservedRunningTime="2026-07-18 02:03:52.957630643 +0000 UTC m=+427.582667310"
Jul 18 02:03:55.007092 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:03:55.007055    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjsqt"]
Jul 18 02:03:55.010503 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:03:55.010482    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjsqt"
Jul 18 02:03:55.013783 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:03:55.013761    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-8xdm8\""
Jul 18 02:03:55.024167 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:03:55.024141    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjsqt"]
Jul 18 02:03:55.141699 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:03:55.141664    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhmzq\" (UniqueName: \"kubernetes.io/projected/5e5d0854-c4d4-44ce-a657-6774fc1e3f94-kube-api-access-bhmzq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hjsqt\" (UID: \"5e5d0854-c4d4-44ce-a657-6774fc1e3f94\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjsqt"
Jul 18 02:03:55.141875 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:03:55.141711    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5e5d0854-c4d4-44ce-a657-6774fc1e3f94-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hjsqt\" (UID: \"5e5d0854-c4d4-44ce-a657-6774fc1e3f94\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjsqt"
Jul 18 02:03:55.242342 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:03:55.242308    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhmzq\" (UniqueName: \"kubernetes.io/projected/5e5d0854-c4d4-44ce-a657-6774fc1e3f94-kube-api-access-bhmzq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hjsqt\" (UID: \"5e5d0854-c4d4-44ce-a657-6774fc1e3f94\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjsqt"
Jul 18 02:03:55.242563 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:03:55.242354    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5e5d0854-c4d4-44ce-a657-6774fc1e3f94-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hjsqt\" (UID: \"5e5d0854-c4d4-44ce-a657-6774fc1e3f94\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjsqt"
Jul 18 02:03:55.242724 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:03:55.242704    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5e5d0854-c4d4-44ce-a657-6774fc1e3f94-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hjsqt\" (UID: \"5e5d0854-c4d4-44ce-a657-6774fc1e3f94\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjsqt"
Jul 18 02:03:55.252019 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:03:55.251996    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhmzq\" (UniqueName: \"kubernetes.io/projected/5e5d0854-c4d4-44ce-a657-6774fc1e3f94-kube-api-access-bhmzq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hjsqt\" (UID: \"5e5d0854-c4d4-44ce-a657-6774fc1e3f94\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjsqt"
Jul 18 02:03:55.320477 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:03:55.320347    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjsqt"
Jul 18 02:03:55.462065 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:03:55.461977    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjsqt"]
Jul 18 02:03:55.465057 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:03:55.465031    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e5d0854_c4d4_44ce_a657_6774fc1e3f94.slice/crio-a96e0cf51ea4b41a21a66cf665badcf62770defc603455ea2943521cc927b4e4 WatchSource:0}: Error finding container a96e0cf51ea4b41a21a66cf665badcf62770defc603455ea2943521cc927b4e4: Status 404 returned error can't find the container with id a96e0cf51ea4b41a21a66cf665badcf62770defc603455ea2943521cc927b4e4
Jul 18 02:03:55.953098 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:03:55.953055    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjsqt" event={"ID":"5e5d0854-c4d4-44ce-a657-6774fc1e3f94","Type":"ContainerStarted","Data":"a96e0cf51ea4b41a21a66cf665badcf62770defc603455ea2943521cc927b4e4"}
Jul 18 02:03:59.974014 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:03:59.973929    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjsqt" event={"ID":"5e5d0854-c4d4-44ce-a657-6774fc1e3f94","Type":"ContainerStarted","Data":"1e8b9445b456d8b57dede70569062a0ed988267d2f8c548e59f9d8a3e99722c7"}
Jul 18 02:03:59.974390 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:03:59.974071    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjsqt"
Jul 18 02:03:59.993143 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:03:59.993094    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjsqt" podStartSLOduration=1.7432644449999999 podStartE2EDuration="5.99307512s" totalImagesPullingTime="4.249810675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:03:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:03:59.992248668 +0000 UTC m=+434.617285338" watchObservedRunningTime="2026-07-18 02:03:59.99307512 +0000 UTC m=+434.618111788"
Jul 18 02:04:03.945475 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:03.945418    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hlwwp"
Jul 18 02:04:10.981618 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:10.981537    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjsqt"
Jul 18 02:04:12.641538 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:12.641501    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjsqt"]
Jul 18 02:04:12.641961 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:12.641672    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjsqt" podUID="5e5d0854-c4d4-44ce-a657-6774fc1e3f94" containerName="manager" containerID="cri-o://1e8b9445b456d8b57dede70569062a0ed988267d2f8c548e59f9d8a3e99722c7" gracePeriod=2
Jul 18 02:04:12.645699 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:12.645669    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjsqt"]
Jul 18 02:04:12.645858 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:04:12.645812    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-hjsqt\" is forbidden: User \"system:node:ip-10-0-143-8.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-8.ec2.internal' and this object" podUID="5e5d0854-c4d4-44ce-a657-6774fc1e3f94" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjsqt"
Jul 18 02:04:12.674844 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:12.674810    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxjjt"]
Jul 18 02:04:12.675213 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:12.675195    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e5d0854-c4d4-44ce-a657-6774fc1e3f94" containerName="manager"
Jul 18 02:04:12.675213 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:12.675213    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e5d0854-c4d4-44ce-a657-6774fc1e3f94" containerName="manager"
Jul 18 02:04:12.675327 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:12.675282    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e5d0854-c4d4-44ce-a657-6774fc1e3f94" containerName="manager"
Jul 18 02:04:12.678622 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:12.678600    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxjjt"
Jul 18 02:04:12.680774 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:04:12.680739    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-hjsqt\" is forbidden: User \"system:node:ip-10-0-143-8.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-8.ec2.internal' and this object" podUID="5e5d0854-c4d4-44ce-a657-6774fc1e3f94" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjsqt"
Jul 18 02:04:12.693120 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:12.693093    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxjjt"]
Jul 18 02:04:12.796834 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:12.796802    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vdxs\" (UniqueName: \"kubernetes.io/projected/3c3f2d16-1e1e-436e-824f-5ee70f3d9e40-kube-api-access-8vdxs\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rxjjt\" (UID: \"3c3f2d16-1e1e-436e-824f-5ee70f3d9e40\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxjjt"
Jul 18 02:04:12.797015 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:12.796876    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3c3f2d16-1e1e-436e-824f-5ee70f3d9e40-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rxjjt\" (UID: \"3c3f2d16-1e1e-436e-824f-5ee70f3d9e40\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxjjt"
Jul 18 02:04:12.889485 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:12.889460    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjsqt"
Jul 18 02:04:12.891527 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:04:12.891498    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-hjsqt\" is forbidden: User \"system:node:ip-10-0-143-8.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-8.ec2.internal' and this object" podUID="5e5d0854-c4d4-44ce-a657-6774fc1e3f94" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjsqt"
Jul 18 02:04:12.897921 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:12.897862    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3c3f2d16-1e1e-436e-824f-5ee70f3d9e40-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rxjjt\" (UID: \"3c3f2d16-1e1e-436e-824f-5ee70f3d9e40\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxjjt"
Jul 18 02:04:12.897921 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:12.897919    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vdxs\" (UniqueName: \"kubernetes.io/projected/3c3f2d16-1e1e-436e-824f-5ee70f3d9e40-kube-api-access-8vdxs\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rxjjt\" (UID: \"3c3f2d16-1e1e-436e-824f-5ee70f3d9e40\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxjjt"
Jul 18 02:04:12.898199 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:12.898183    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3c3f2d16-1e1e-436e-824f-5ee70f3d9e40-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rxjjt\" (UID: \"3c3f2d16-1e1e-436e-824f-5ee70f3d9e40\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxjjt"
Jul 18 02:04:12.906260 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:12.906240    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vdxs\" (UniqueName: \"kubernetes.io/projected/3c3f2d16-1e1e-436e-824f-5ee70f3d9e40-kube-api-access-8vdxs\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rxjjt\" (UID: \"3c3f2d16-1e1e-436e-824f-5ee70f3d9e40\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxjjt"
Jul 18 02:04:12.998475 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:12.998413    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bhmzq\" (UniqueName: \"kubernetes.io/projected/5e5d0854-c4d4-44ce-a657-6774fc1e3f94-kube-api-access-bhmzq\") pod \"5e5d0854-c4d4-44ce-a657-6774fc1e3f94\" (UID: \"5e5d0854-c4d4-44ce-a657-6774fc1e3f94\") "
Jul 18 02:04:12.998618 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:12.998516    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5e5d0854-c4d4-44ce-a657-6774fc1e3f94-extensions-socket-volume\") pod \"5e5d0854-c4d4-44ce-a657-6774fc1e3f94\" (UID: \"5e5d0854-c4d4-44ce-a657-6774fc1e3f94\") "
Jul 18 02:04:12.999054 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:12.999021    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e5d0854-c4d4-44ce-a657-6774fc1e3f94-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5e5d0854-c4d4-44ce-a657-6774fc1e3f94" (UID: "5e5d0854-c4d4-44ce-a657-6774fc1e3f94"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:04:13.000747 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:13.000725    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e5d0854-c4d4-44ce-a657-6774fc1e3f94-kube-api-access-bhmzq" (OuterVolumeSpecName: "kube-api-access-bhmzq") pod "5e5d0854-c4d4-44ce-a657-6774fc1e3f94" (UID: "5e5d0854-c4d4-44ce-a657-6774fc1e3f94"). InnerVolumeSpecName "kube-api-access-bhmzq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:04:13.024004 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:13.023972    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxjjt"
Jul 18 02:04:13.036973 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:13.036939    2576 generic.go:356] "Generic (PLEG): container finished" podID="5e5d0854-c4d4-44ce-a657-6774fc1e3f94" containerID="1e8b9445b456d8b57dede70569062a0ed988267d2f8c548e59f9d8a3e99722c7" exitCode=0
Jul 18 02:04:13.037086 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:13.036997    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjsqt"
Jul 18 02:04:13.037086 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:13.037037    2576 scope.go:117] "RemoveContainer" containerID="1e8b9445b456d8b57dede70569062a0ed988267d2f8c548e59f9d8a3e99722c7"
Jul 18 02:04:13.039489 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:04:13.039459    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-hjsqt\" is forbidden: User \"system:node:ip-10-0-143-8.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-8.ec2.internal' and this object" podUID="5e5d0854-c4d4-44ce-a657-6774fc1e3f94" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjsqt"
Jul 18 02:04:13.047663 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:13.047642    2576 scope.go:117] "RemoveContainer" containerID="1e8b9445b456d8b57dede70569062a0ed988267d2f8c548e59f9d8a3e99722c7"
Jul 18 02:04:13.047753 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:04:13.047714    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-hjsqt\" is forbidden: User \"system:node:ip-10-0-143-8.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-8.ec2.internal' and this object" podUID="5e5d0854-c4d4-44ce-a657-6774fc1e3f94" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjsqt"
Jul 18 02:04:13.047962 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:04:13.047939    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e8b9445b456d8b57dede70569062a0ed988267d2f8c548e59f9d8a3e99722c7\": container with ID starting with 1e8b9445b456d8b57dede70569062a0ed988267d2f8c548e59f9d8a3e99722c7 not found: ID does not exist" containerID="1e8b9445b456d8b57dede70569062a0ed988267d2f8c548e59f9d8a3e99722c7"
Jul 18 02:04:13.048024 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:13.047977    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e8b9445b456d8b57dede70569062a0ed988267d2f8c548e59f9d8a3e99722c7"} err="failed to get container status \"1e8b9445b456d8b57dede70569062a0ed988267d2f8c548e59f9d8a3e99722c7\": rpc error: code = NotFound desc = could not find container \"1e8b9445b456d8b57dede70569062a0ed988267d2f8c548e59f9d8a3e99722c7\": container with ID starting with 1e8b9445b456d8b57dede70569062a0ed988267d2f8c548e59f9d8a3e99722c7 not found: ID does not exist"
Jul 18 02:04:13.099726 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:13.099677    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5e5d0854-c4d4-44ce-a657-6774fc1e3f94-extensions-socket-volume\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:04:13.099726 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:13.099706    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhmzq\" (UniqueName: \"kubernetes.io/projected/5e5d0854-c4d4-44ce-a657-6774fc1e3f94-kube-api-access-bhmzq\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:04:13.168013 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:13.167989    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxjjt"]
Jul 18 02:04:13.170297 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:04:13.170273    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c3f2d16_1e1e_436e_824f_5ee70f3d9e40.slice/crio-4f73f874824b53c4ad0cdf126e13a7d6ceb7d98a0adecebda28b75ff026d4c3a WatchSource:0}: Error finding container 4f73f874824b53c4ad0cdf126e13a7d6ceb7d98a0adecebda28b75ff026d4c3a: Status 404 returned error can't find the container with id 4f73f874824b53c4ad0cdf126e13a7d6ceb7d98a0adecebda28b75ff026d4c3a
Jul 18 02:04:13.970027 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:13.969990    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e5d0854-c4d4-44ce-a657-6774fc1e3f94" path="/var/lib/kubelet/pods/5e5d0854-c4d4-44ce-a657-6774fc1e3f94/volumes"
Jul 18 02:04:14.044010 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:14.043976    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxjjt" event={"ID":"3c3f2d16-1e1e-436e-824f-5ee70f3d9e40","Type":"ContainerStarted","Data":"cc00a9954a18488ee218735af515f1471e68eaf2eaea677ac11393520719c329"}
Jul 18 02:04:14.044174 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:14.044029    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxjjt" event={"ID":"3c3f2d16-1e1e-436e-824f-5ee70f3d9e40","Type":"ContainerStarted","Data":"4f73f874824b53c4ad0cdf126e13a7d6ceb7d98a0adecebda28b75ff026d4c3a"}
Jul 18 02:04:14.044174 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:14.044109    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxjjt"
Jul 18 02:04:14.066231 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:14.066183    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxjjt" podStartSLOduration=2.0661657350000002 podStartE2EDuration="2.066165735s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:04:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:04:14.064640525 +0000 UTC m=+448.689677233" watchObservedRunningTime="2026-07-18 02:04:14.066165735 +0000 UTC m=+448.691202396"
Jul 18 02:04:25.050866 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:25.050835    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxjjt"
Jul 18 02:04:41.293631 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:41.293587    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxjjt"]
Jul 18 02:04:41.294198 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:41.293877    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxjjt" podUID="3c3f2d16-1e1e-436e-824f-5ee70f3d9e40" containerName="manager" containerID="cri-o://cc00a9954a18488ee218735af515f1471e68eaf2eaea677ac11393520719c329" gracePeriod=10
Jul 18 02:04:41.551644 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:41.551578    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxjjt"
Jul 18 02:04:41.557730 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:41.557697    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7tj9r"]
Jul 18 02:04:41.558039 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:41.558026    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c3f2d16-1e1e-436e-824f-5ee70f3d9e40" containerName="manager"
Jul 18 02:04:41.558080 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:41.558040    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c3f2d16-1e1e-436e-824f-5ee70f3d9e40" containerName="manager"
Jul 18 02:04:41.558114 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:41.558110    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c3f2d16-1e1e-436e-824f-5ee70f3d9e40" containerName="manager"
Jul 18 02:04:41.560866 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:41.560850    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7tj9r"
Jul 18 02:04:41.572625 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:41.572598    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7tj9r"]
Jul 18 02:04:41.618589 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:41.618546    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5214dfeb-feb7-41d4-9041-e8ee71ff099b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7tj9r\" (UID: \"5214dfeb-feb7-41d4-9041-e8ee71ff099b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7tj9r"
Jul 18 02:04:41.618761 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:41.618627    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhr8k\" (UniqueName: \"kubernetes.io/projected/5214dfeb-feb7-41d4-9041-e8ee71ff099b-kube-api-access-rhr8k\") pod \"kuadrant-operator-controller-manager-86878c769f-7tj9r\" (UID: \"5214dfeb-feb7-41d4-9041-e8ee71ff099b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7tj9r"
Jul 18 02:04:41.719247 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:41.719208    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3c3f2d16-1e1e-436e-824f-5ee70f3d9e40-extensions-socket-volume\") pod \"3c3f2d16-1e1e-436e-824f-5ee70f3d9e40\" (UID: \"3c3f2d16-1e1e-436e-824f-5ee70f3d9e40\") "
Jul 18 02:04:41.719487 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:41.719305    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8vdxs\" (UniqueName: \"kubernetes.io/projected/3c3f2d16-1e1e-436e-824f-5ee70f3d9e40-kube-api-access-8vdxs\") pod \"3c3f2d16-1e1e-436e-824f-5ee70f3d9e40\" (UID: \"3c3f2d16-1e1e-436e-824f-5ee70f3d9e40\") "
Jul 18 02:04:41.719487 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:41.719466    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5214dfeb-feb7-41d4-9041-e8ee71ff099b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7tj9r\" (UID: \"5214dfeb-feb7-41d4-9041-e8ee71ff099b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7tj9r"
Jul 18 02:04:41.719626 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:41.719525    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhr8k\" (UniqueName: \"kubernetes.io/projected/5214dfeb-feb7-41d4-9041-e8ee71ff099b-kube-api-access-rhr8k\") pod \"kuadrant-operator-controller-manager-86878c769f-7tj9r\" (UID: \"5214dfeb-feb7-41d4-9041-e8ee71ff099b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7tj9r"
Jul 18 02:04:41.719674 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:41.719620    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c3f2d16-1e1e-436e-824f-5ee70f3d9e40-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3c3f2d16-1e1e-436e-824f-5ee70f3d9e40" (UID: "3c3f2d16-1e1e-436e-824f-5ee70f3d9e40"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:04:41.719881 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:41.719856    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5214dfeb-feb7-41d4-9041-e8ee71ff099b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7tj9r\" (UID: \"5214dfeb-feb7-41d4-9041-e8ee71ff099b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7tj9r"
Jul 18 02:04:41.721444 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:41.721408    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c3f2d16-1e1e-436e-824f-5ee70f3d9e40-kube-api-access-8vdxs" (OuterVolumeSpecName: "kube-api-access-8vdxs") pod "3c3f2d16-1e1e-436e-824f-5ee70f3d9e40" (UID: "3c3f2d16-1e1e-436e-824f-5ee70f3d9e40"). InnerVolumeSpecName "kube-api-access-8vdxs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:04:41.728202 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:41.728182    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhr8k\" (UniqueName: \"kubernetes.io/projected/5214dfeb-feb7-41d4-9041-e8ee71ff099b-kube-api-access-rhr8k\") pod \"kuadrant-operator-controller-manager-86878c769f-7tj9r\" (UID: \"5214dfeb-feb7-41d4-9041-e8ee71ff099b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7tj9r"
Jul 18 02:04:41.820252 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:41.820159    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vdxs\" (UniqueName: \"kubernetes.io/projected/3c3f2d16-1e1e-436e-824f-5ee70f3d9e40-kube-api-access-8vdxs\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:04:41.820252 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:41.820194    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3c3f2d16-1e1e-436e-824f-5ee70f3d9e40-extensions-socket-volume\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:04:41.870612 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:41.870578    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7tj9r"
Jul 18 02:04:42.024283 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:42.024257    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7tj9r"]
Jul 18 02:04:42.026325 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:04:42.026299    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5214dfeb_feb7_41d4_9041_e8ee71ff099b.slice/crio-ae17aa4cfa5a13b3292e0ff2902ae455a48d2b695b0e4b6af38a8ac26ea58ff4 WatchSource:0}: Error finding container ae17aa4cfa5a13b3292e0ff2902ae455a48d2b695b0e4b6af38a8ac26ea58ff4: Status 404 returned error can't find the container with id ae17aa4cfa5a13b3292e0ff2902ae455a48d2b695b0e4b6af38a8ac26ea58ff4
Jul 18 02:04:42.174896 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:42.174857    2576 generic.go:356] "Generic (PLEG): container finished" podID="3c3f2d16-1e1e-436e-824f-5ee70f3d9e40" containerID="cc00a9954a18488ee218735af515f1471e68eaf2eaea677ac11393520719c329" exitCode=0
Jul 18 02:04:42.175078 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:42.174897    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxjjt"
Jul 18 02:04:42.175078 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:42.174941    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxjjt" event={"ID":"3c3f2d16-1e1e-436e-824f-5ee70f3d9e40","Type":"ContainerDied","Data":"cc00a9954a18488ee218735af515f1471e68eaf2eaea677ac11393520719c329"}
Jul 18 02:04:42.175078 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:42.174999    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxjjt" event={"ID":"3c3f2d16-1e1e-436e-824f-5ee70f3d9e40","Type":"ContainerDied","Data":"4f73f874824b53c4ad0cdf126e13a7d6ceb7d98a0adecebda28b75ff026d4c3a"}
Jul 18 02:04:42.175078 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:42.175035    2576 scope.go:117] "RemoveContainer" containerID="cc00a9954a18488ee218735af515f1471e68eaf2eaea677ac11393520719c329"
Jul 18 02:04:42.176855 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:42.176825    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7tj9r" event={"ID":"5214dfeb-feb7-41d4-9041-e8ee71ff099b","Type":"ContainerStarted","Data":"28d9682945bcb4b65365e87df8c8895cb27fa0816567b38f7aae7930b941b1b5"}
Jul 18 02:04:42.177003 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:42.176884    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7tj9r"
Jul 18 02:04:42.177003 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:42.176909    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7tj9r" event={"ID":"5214dfeb-feb7-41d4-9041-e8ee71ff099b","Type":"ContainerStarted","Data":"ae17aa4cfa5a13b3292e0ff2902ae455a48d2b695b0e4b6af38a8ac26ea58ff4"}
Jul 18 02:04:42.184422 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:42.184398    2576 scope.go:117] "RemoveContainer" containerID="cc00a9954a18488ee218735af515f1471e68eaf2eaea677ac11393520719c329"
Jul 18 02:04:42.184774 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:04:42.184752    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc00a9954a18488ee218735af515f1471e68eaf2eaea677ac11393520719c329\": container with ID starting with cc00a9954a18488ee218735af515f1471e68eaf2eaea677ac11393520719c329 not found: ID does not exist" containerID="cc00a9954a18488ee218735af515f1471e68eaf2eaea677ac11393520719c329"
Jul 18 02:04:42.184844 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:42.184782    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc00a9954a18488ee218735af515f1471e68eaf2eaea677ac11393520719c329"} err="failed to get container status \"cc00a9954a18488ee218735af515f1471e68eaf2eaea677ac11393520719c329\": rpc error: code = NotFound desc = could not find container \"cc00a9954a18488ee218735af515f1471e68eaf2eaea677ac11393520719c329\": container with ID starting with cc00a9954a18488ee218735af515f1471e68eaf2eaea677ac11393520719c329 not found: ID does not exist"
Jul 18 02:04:42.193156 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:42.193130    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxjjt"]
Jul 18 02:04:42.197532 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:42.197510    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rxjjt"]
Jul 18 02:04:42.211820 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:42.211744    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7tj9r" podStartSLOduration=1.211724672 podStartE2EDuration="1.211724672s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:04:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:04:42.211167065 +0000 UTC m=+476.836203732" watchObservedRunningTime="2026-07-18 02:04:42.211724672 +0000 UTC m=+476.836761340"
Jul 18 02:04:43.969223 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:43.969190    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c3f2d16-1e1e-436e-824f-5ee70f3d9e40" path="/var/lib/kubelet/pods/3c3f2d16-1e1e-436e-824f-5ee70f3d9e40/volumes"
Jul 18 02:04:53.185835 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:04:53.185801    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7tj9r"
Jul 18 02:05:00.304171 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:00.304133    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-s85pk"]
Jul 18 02:05:00.308940 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:00.308917    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-s85pk"
Jul 18 02:05:00.311859 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:00.311841    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-p4c9m\""
Jul 18 02:05:00.311859 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:00.311853    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 18 02:05:00.318678 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:00.318651    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-s85pk"]
Jul 18 02:05:00.353643 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:00.353605    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzkrj\" (UniqueName: \"kubernetes.io/projected/6a6b316d-75d0-4ed7-9d9f-7b58346511fd-kube-api-access-pzkrj\") pod \"limitador-limitador-64c94fd64f-s85pk\" (UID: \"6a6b316d-75d0-4ed7-9d9f-7b58346511fd\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-s85pk"
Jul 18 02:05:00.353841 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:00.353692    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6a6b316d-75d0-4ed7-9d9f-7b58346511fd-config-file\") pod \"limitador-limitador-64c94fd64f-s85pk\" (UID: \"6a6b316d-75d0-4ed7-9d9f-7b58346511fd\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-s85pk"
Jul 18 02:05:00.410420 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:00.410380    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-s85pk"]
Jul 18 02:05:00.454785 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:00.454745    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6a6b316d-75d0-4ed7-9d9f-7b58346511fd-config-file\") pod \"limitador-limitador-64c94fd64f-s85pk\" (UID: \"6a6b316d-75d0-4ed7-9d9f-7b58346511fd\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-s85pk"
Jul 18 02:05:00.454985 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:00.454850    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzkrj\" (UniqueName: \"kubernetes.io/projected/6a6b316d-75d0-4ed7-9d9f-7b58346511fd-kube-api-access-pzkrj\") pod \"limitador-limitador-64c94fd64f-s85pk\" (UID: \"6a6b316d-75d0-4ed7-9d9f-7b58346511fd\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-s85pk"
Jul 18 02:05:00.455414 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:00.455395    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6a6b316d-75d0-4ed7-9d9f-7b58346511fd-config-file\") pod \"limitador-limitador-64c94fd64f-s85pk\" (UID: \"6a6b316d-75d0-4ed7-9d9f-7b58346511fd\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-s85pk"
Jul 18 02:05:00.463125 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:00.463105    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzkrj\" (UniqueName: \"kubernetes.io/projected/6a6b316d-75d0-4ed7-9d9f-7b58346511fd-kube-api-access-pzkrj\") pod \"limitador-limitador-64c94fd64f-s85pk\" (UID: \"6a6b316d-75d0-4ed7-9d9f-7b58346511fd\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-s85pk"
Jul 18 02:05:00.620486 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:00.620396    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-s85pk"
Jul 18 02:05:00.766399 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:00.766373    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-s85pk"]
Jul 18 02:05:00.768817 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:05:00.768789    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a6b316d_75d0_4ed7_9d9f_7b58346511fd.slice/crio-ba1496e227b2448f80b959b6b7e4b66aed503bc80cd889b5872ba044d5066226 WatchSource:0}: Error finding container ba1496e227b2448f80b959b6b7e4b66aed503bc80cd889b5872ba044d5066226: Status 404 returned error can't find the container with id ba1496e227b2448f80b959b6b7e4b66aed503bc80cd889b5872ba044d5066226
Jul 18 02:05:01.268731 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:01.268698    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-s85pk" event={"ID":"6a6b316d-75d0-4ed7-9d9f-7b58346511fd","Type":"ContainerStarted","Data":"ba1496e227b2448f80b959b6b7e4b66aed503bc80cd889b5872ba044d5066226"}
Jul 18 02:05:04.286192 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:04.286157    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-s85pk" event={"ID":"6a6b316d-75d0-4ed7-9d9f-7b58346511fd","Type":"ContainerStarted","Data":"8b83f21c53b84323a54323e797763b0b1da18e3ee97c68e8507b8853792bdb0c"}
Jul 18 02:05:04.286624 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:04.286281    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-s85pk"
Jul 18 02:05:04.301412 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:04.301363    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-s85pk" podStartSLOduration=1.575075988 podStartE2EDuration="4.301346806s" totalImagesPullingTime="2.726270818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:05:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:05:04.300725714 +0000 UTC m=+498.925762393" watchObservedRunningTime="2026-07-18 02:05:04.301346806 +0000 UTC m=+498.926383509"
Jul 18 02:05:15.293676 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:15.293641    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-s85pk"
Jul 18 02:05:16.458880 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:16.458843    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-s85pk"]
Jul 18 02:05:16.459329 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:16.459028    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-s85pk" podUID="6a6b316d-75d0-4ed7-9d9f-7b58346511fd" containerName="limitador" containerID="cri-o://8b83f21c53b84323a54323e797763b0b1da18e3ee97c68e8507b8853792bdb0c" gracePeriod=30
Jul 18 02:05:17.032702 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:17.032673    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-s85pk"
Jul 18 02:05:17.089478 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:17.089412    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6a6b316d-75d0-4ed7-9d9f-7b58346511fd-config-file\") pod \"6a6b316d-75d0-4ed7-9d9f-7b58346511fd\" (UID: \"6a6b316d-75d0-4ed7-9d9f-7b58346511fd\") "
Jul 18 02:05:17.089645 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:17.089503    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pzkrj\" (UniqueName: \"kubernetes.io/projected/6a6b316d-75d0-4ed7-9d9f-7b58346511fd-kube-api-access-pzkrj\") pod \"6a6b316d-75d0-4ed7-9d9f-7b58346511fd\" (UID: \"6a6b316d-75d0-4ed7-9d9f-7b58346511fd\") "
Jul 18 02:05:17.089803 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:17.089777    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a6b316d-75d0-4ed7-9d9f-7b58346511fd-config-file" (OuterVolumeSpecName: "config-file") pod "6a6b316d-75d0-4ed7-9d9f-7b58346511fd" (UID: "6a6b316d-75d0-4ed7-9d9f-7b58346511fd"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:05:17.091579 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:17.091548    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a6b316d-75d0-4ed7-9d9f-7b58346511fd-kube-api-access-pzkrj" (OuterVolumeSpecName: "kube-api-access-pzkrj") pod "6a6b316d-75d0-4ed7-9d9f-7b58346511fd" (UID: "6a6b316d-75d0-4ed7-9d9f-7b58346511fd"). InnerVolumeSpecName "kube-api-access-pzkrj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:05:17.190377 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:17.190330    2576 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6a6b316d-75d0-4ed7-9d9f-7b58346511fd-config-file\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:05:17.190377 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:17.190368    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pzkrj\" (UniqueName: \"kubernetes.io/projected/6a6b316d-75d0-4ed7-9d9f-7b58346511fd-kube-api-access-pzkrj\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:05:17.351423 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:17.351330    2576 generic.go:356] "Generic (PLEG): container finished" podID="6a6b316d-75d0-4ed7-9d9f-7b58346511fd" containerID="8b83f21c53b84323a54323e797763b0b1da18e3ee97c68e8507b8853792bdb0c" exitCode=0
Jul 18 02:05:17.351423 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:17.351377    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-s85pk"
Jul 18 02:05:17.351423 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:17.351405    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-s85pk" event={"ID":"6a6b316d-75d0-4ed7-9d9f-7b58346511fd","Type":"ContainerDied","Data":"8b83f21c53b84323a54323e797763b0b1da18e3ee97c68e8507b8853792bdb0c"}
Jul 18 02:05:17.351690 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:17.351477    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-s85pk" event={"ID":"6a6b316d-75d0-4ed7-9d9f-7b58346511fd","Type":"ContainerDied","Data":"ba1496e227b2448f80b959b6b7e4b66aed503bc80cd889b5872ba044d5066226"}
Jul 18 02:05:17.351690 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:17.351508    2576 scope.go:117] "RemoveContainer" containerID="8b83f21c53b84323a54323e797763b0b1da18e3ee97c68e8507b8853792bdb0c"
Jul 18 02:05:17.360504 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:17.360483    2576 scope.go:117] "RemoveContainer" containerID="8b83f21c53b84323a54323e797763b0b1da18e3ee97c68e8507b8853792bdb0c"
Jul 18 02:05:17.360776 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:05:17.360757    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b83f21c53b84323a54323e797763b0b1da18e3ee97c68e8507b8853792bdb0c\": container with ID starting with 8b83f21c53b84323a54323e797763b0b1da18e3ee97c68e8507b8853792bdb0c not found: ID does not exist" containerID="8b83f21c53b84323a54323e797763b0b1da18e3ee97c68e8507b8853792bdb0c"
Jul 18 02:05:17.360826 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:17.360786    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b83f21c53b84323a54323e797763b0b1da18e3ee97c68e8507b8853792bdb0c"} err="failed to get container status \"8b83f21c53b84323a54323e797763b0b1da18e3ee97c68e8507b8853792bdb0c\": rpc error: code = NotFound desc = could not find container \"8b83f21c53b84323a54323e797763b0b1da18e3ee97c68e8507b8853792bdb0c\": container with ID starting with 8b83f21c53b84323a54323e797763b0b1da18e3ee97c68e8507b8853792bdb0c not found: ID does not exist"
Jul 18 02:05:17.375290 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:17.375254    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-s85pk"]
Jul 18 02:05:17.376805 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:17.376784    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-s85pk"]
Jul 18 02:05:17.968813 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:17.968780    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a6b316d-75d0-4ed7-9d9f-7b58346511fd" path="/var/lib/kubelet/pods/6a6b316d-75d0-4ed7-9d9f-7b58346511fd/volumes"
Jul 18 02:05:21.600768 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:21.600734    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-qvntt"]
Jul 18 02:05:21.601205 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:21.601187    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a6b316d-75d0-4ed7-9d9f-7b58346511fd" containerName="limitador"
Jul 18 02:05:21.601278 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:21.601208    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a6b316d-75d0-4ed7-9d9f-7b58346511fd" containerName="limitador"
Jul 18 02:05:21.601381 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:21.601326    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a6b316d-75d0-4ed7-9d9f-7b58346511fd" containerName="limitador"
Jul 18 02:05:21.605757 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:21.605736    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qvntt"
Jul 18 02:05:21.608236 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:21.608212    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 18 02:05:21.608236 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:21.608226    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-lcm58\""
Jul 18 02:05:21.608409 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:21.608267    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 02:05:21.608409 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:21.608293    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 18 02:05:21.613240 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:21.613214    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-qvntt"]
Jul 18 02:05:21.731320 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:21.731278    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c8b3bc8c-aabb-48b1-9c9f-3915bc2095f1-data\") pod \"postgres-69fb4594cf-qvntt\" (UID: \"c8b3bc8c-aabb-48b1-9c9f-3915bc2095f1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-qvntt"
Jul 18 02:05:21.731541 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:21.731362    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kb9m9\" (UniqueName: \"kubernetes.io/projected/c8b3bc8c-aabb-48b1-9c9f-3915bc2095f1-kube-api-access-kb9m9\") pod \"postgres-69fb4594cf-qvntt\" (UID: \"c8b3bc8c-aabb-48b1-9c9f-3915bc2095f1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-qvntt"
Jul 18 02:05:21.832771 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:21.832714    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kb9m9\" (UniqueName: \"kubernetes.io/projected/c8b3bc8c-aabb-48b1-9c9f-3915bc2095f1-kube-api-access-kb9m9\") pod \"postgres-69fb4594cf-qvntt\" (UID: \"c8b3bc8c-aabb-48b1-9c9f-3915bc2095f1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-qvntt"
Jul 18 02:05:21.832961 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:21.832838    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c8b3bc8c-aabb-48b1-9c9f-3915bc2095f1-data\") pod \"postgres-69fb4594cf-qvntt\" (UID: \"c8b3bc8c-aabb-48b1-9c9f-3915bc2095f1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-qvntt"
Jul 18 02:05:21.833194 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:21.833178    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c8b3bc8c-aabb-48b1-9c9f-3915bc2095f1-data\") pod \"postgres-69fb4594cf-qvntt\" (UID: \"c8b3bc8c-aabb-48b1-9c9f-3915bc2095f1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-qvntt"
Jul 18 02:05:21.841148 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:21.841110    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kb9m9\" (UniqueName: \"kubernetes.io/projected/c8b3bc8c-aabb-48b1-9c9f-3915bc2095f1-kube-api-access-kb9m9\") pod \"postgres-69fb4594cf-qvntt\" (UID: \"c8b3bc8c-aabb-48b1-9c9f-3915bc2095f1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-qvntt"
Jul 18 02:05:21.916700 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:21.916663    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qvntt"
Jul 18 02:05:22.049853 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:22.049826    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-qvntt"]
Jul 18 02:05:22.052493 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:05:22.052460    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8b3bc8c_aabb_48b1_9c9f_3915bc2095f1.slice/crio-8e88d9b9a6f08ed5e4c7ed5f85880933d60fe0c3911a57aebdc6adc35fe0b804 WatchSource:0}: Error finding container 8e88d9b9a6f08ed5e4c7ed5f85880933d60fe0c3911a57aebdc6adc35fe0b804: Status 404 returned error can't find the container with id 8e88d9b9a6f08ed5e4c7ed5f85880933d60fe0c3911a57aebdc6adc35fe0b804
Jul 18 02:05:22.389622 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:22.389585    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qvntt" event={"ID":"c8b3bc8c-aabb-48b1-9c9f-3915bc2095f1","Type":"ContainerStarted","Data":"8e88d9b9a6f08ed5e4c7ed5f85880933d60fe0c3911a57aebdc6adc35fe0b804"}
Jul 18 02:05:28.424962 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:28.424924    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qvntt" event={"ID":"c8b3bc8c-aabb-48b1-9c9f-3915bc2095f1","Type":"ContainerStarted","Data":"b5ce5b1aa834be20e7e7ef7e6775865a82bc53f157e792acdbbe3922c5c15bf6"}
Jul 18 02:05:28.425475 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:28.425017    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qvntt"
Jul 18 02:05:28.440806 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:28.440756    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qvntt" podStartSLOduration=1.8783371089999998 podStartE2EDuration="7.440740569s" totalImagesPullingTime="5.56240346s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:05:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:05:28.43966681 +0000 UTC m=+523.064703479" watchObservedRunningTime="2026-07-18 02:05:28.440740569 +0000 UTC m=+523.065777235"
Jul 18 02:05:34.460374 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:34.460341    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qvntt"
Jul 18 02:05:41.723171 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:41.723090    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-66b54cf598-n59fp"]
Jul 18 02:05:41.726516 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:41.726489    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66b54cf598-n59fp"
Jul 18 02:05:41.728992 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:41.728968    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 18 02:05:41.729136 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:41.728992    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-n7v75\""
Jul 18 02:05:41.729136 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:41.729021    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 18 02:05:41.735670 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:41.735642    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-66b54cf598-n59fp"]
Jul 18 02:05:41.818567 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:41.818519    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znb4l\" (UniqueName: \"kubernetes.io/projected/93b7a318-bb95-4dc5-96a9-f6d33be222f4-kube-api-access-znb4l\") pod \"maas-controller-66b54cf598-n59fp\" (UID: \"93b7a318-bb95-4dc5-96a9-f6d33be222f4\") " pod="opendatahub/maas-controller-66b54cf598-n59fp"
Jul 18 02:05:41.818744 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:41.818579    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/93b7a318-bb95-4dc5-96a9-f6d33be222f4-webhook-cert\") pod \"maas-controller-66b54cf598-n59fp\" (UID: \"93b7a318-bb95-4dc5-96a9-f6d33be222f4\") " pod="opendatahub/maas-controller-66b54cf598-n59fp"
Jul 18 02:05:41.881265 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:41.881232    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-86c7b54d87-6v7bz"]
Jul 18 02:05:41.884716 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:41.884691    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86c7b54d87-6v7bz"
Jul 18 02:05:41.895344 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:41.895319    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-86c7b54d87-6v7bz"]
Jul 18 02:05:41.919383 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:41.919349    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znb4l\" (UniqueName: \"kubernetes.io/projected/93b7a318-bb95-4dc5-96a9-f6d33be222f4-kube-api-access-znb4l\") pod \"maas-controller-66b54cf598-n59fp\" (UID: \"93b7a318-bb95-4dc5-96a9-f6d33be222f4\") " pod="opendatahub/maas-controller-66b54cf598-n59fp"
Jul 18 02:05:41.919601 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:41.919401    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/93b7a318-bb95-4dc5-96a9-f6d33be222f4-webhook-cert\") pod \"maas-controller-66b54cf598-n59fp\" (UID: \"93b7a318-bb95-4dc5-96a9-f6d33be222f4\") " pod="opendatahub/maas-controller-66b54cf598-n59fp"
Jul 18 02:05:41.919601 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:05:41.919527    2576 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 18 02:05:41.919601 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:05:41.919601    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93b7a318-bb95-4dc5-96a9-f6d33be222f4-webhook-cert podName:93b7a318-bb95-4dc5-96a9-f6d33be222f4 nodeName:}" failed. No retries permitted until 2026-07-18 02:05:42.41958418 +0000 UTC m=+537.044620828 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/93b7a318-bb95-4dc5-96a9-f6d33be222f4-webhook-cert") pod "maas-controller-66b54cf598-n59fp" (UID: "93b7a318-bb95-4dc5-96a9-f6d33be222f4") : secret "maas-controller-webhook-cert" not found
Jul 18 02:05:41.927695 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:41.927665    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znb4l\" (UniqueName: \"kubernetes.io/projected/93b7a318-bb95-4dc5-96a9-f6d33be222f4-kube-api-access-znb4l\") pod \"maas-controller-66b54cf598-n59fp\" (UID: \"93b7a318-bb95-4dc5-96a9-f6d33be222f4\") " pod="opendatahub/maas-controller-66b54cf598-n59fp"
Jul 18 02:05:42.020796 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:42.020707    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d9956f87-7b14-4d1e-a576-e62060a9065e-webhook-cert\") pod \"maas-controller-86c7b54d87-6v7bz\" (UID: \"d9956f87-7b14-4d1e-a576-e62060a9065e\") " pod="opendatahub/maas-controller-86c7b54d87-6v7bz"
Jul 18 02:05:42.020947 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:42.020818    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvk6s\" (UniqueName: \"kubernetes.io/projected/d9956f87-7b14-4d1e-a576-e62060a9065e-kube-api-access-nvk6s\") pod \"maas-controller-86c7b54d87-6v7bz\" (UID: \"d9956f87-7b14-4d1e-a576-e62060a9065e\") " pod="opendatahub/maas-controller-86c7b54d87-6v7bz"
Jul 18 02:05:42.121713 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:42.121671    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d9956f87-7b14-4d1e-a576-e62060a9065e-webhook-cert\") pod \"maas-controller-86c7b54d87-6v7bz\" (UID: \"d9956f87-7b14-4d1e-a576-e62060a9065e\") " pod="opendatahub/maas-controller-86c7b54d87-6v7bz"
Jul 18 02:05:42.121858 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:42.121770    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvk6s\" (UniqueName: \"kubernetes.io/projected/d9956f87-7b14-4d1e-a576-e62060a9065e-kube-api-access-nvk6s\") pod \"maas-controller-86c7b54d87-6v7bz\" (UID: \"d9956f87-7b14-4d1e-a576-e62060a9065e\") " pod="opendatahub/maas-controller-86c7b54d87-6v7bz"
Jul 18 02:05:42.121858 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:05:42.121820    2576 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 18 02:05:42.121926 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:05:42.121902    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9956f87-7b14-4d1e-a576-e62060a9065e-webhook-cert podName:d9956f87-7b14-4d1e-a576-e62060a9065e nodeName:}" failed. No retries permitted until 2026-07-18 02:05:42.62188194 +0000 UTC m=+537.246918604 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/d9956f87-7b14-4d1e-a576-e62060a9065e-webhook-cert") pod "maas-controller-86c7b54d87-6v7bz" (UID: "d9956f87-7b14-4d1e-a576-e62060a9065e") : secret "maas-controller-webhook-cert" not found
Jul 18 02:05:42.130234 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:42.130207    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvk6s\" (UniqueName: \"kubernetes.io/projected/d9956f87-7b14-4d1e-a576-e62060a9065e-kube-api-access-nvk6s\") pod \"maas-controller-86c7b54d87-6v7bz\" (UID: \"d9956f87-7b14-4d1e-a576-e62060a9065e\") " pod="opendatahub/maas-controller-86c7b54d87-6v7bz"
Jul 18 02:05:42.424668 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:42.424615    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/93b7a318-bb95-4dc5-96a9-f6d33be222f4-webhook-cert\") pod \"maas-controller-66b54cf598-n59fp\" (UID: \"93b7a318-bb95-4dc5-96a9-f6d33be222f4\") " pod="opendatahub/maas-controller-66b54cf598-n59fp"
Jul 18 02:05:42.427027 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:42.426999    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/93b7a318-bb95-4dc5-96a9-f6d33be222f4-webhook-cert\") pod \"maas-controller-66b54cf598-n59fp\" (UID: \"93b7a318-bb95-4dc5-96a9-f6d33be222f4\") " pod="opendatahub/maas-controller-66b54cf598-n59fp"
Jul 18 02:05:42.626169 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:42.626135    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d9956f87-7b14-4d1e-a576-e62060a9065e-webhook-cert\") pod \"maas-controller-86c7b54d87-6v7bz\" (UID: \"d9956f87-7b14-4d1e-a576-e62060a9065e\") " pod="opendatahub/maas-controller-86c7b54d87-6v7bz"
Jul 18 02:05:42.628604 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:42.628574    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d9956f87-7b14-4d1e-a576-e62060a9065e-webhook-cert\") pod \"maas-controller-86c7b54d87-6v7bz\" (UID: \"d9956f87-7b14-4d1e-a576-e62060a9065e\") " pod="opendatahub/maas-controller-86c7b54d87-6v7bz"
Jul 18 02:05:42.637901 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:42.637876    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66b54cf598-n59fp"
Jul 18 02:05:42.771825 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:42.771798    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-66b54cf598-n59fp"]
Jul 18 02:05:42.773117 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:05:42.773092    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93b7a318_bb95_4dc5_96a9_f6d33be222f4.slice/crio-0243e3f129a69043dbf201df485d02a5e9d758074e13980f67d031515fa47894 WatchSource:0}: Error finding container 0243e3f129a69043dbf201df485d02a5e9d758074e13980f67d031515fa47894: Status 404 returned error can't find the container with id 0243e3f129a69043dbf201df485d02a5e9d758074e13980f67d031515fa47894
Jul 18 02:05:42.795087 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:42.795061    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86c7b54d87-6v7bz"
Jul 18 02:05:42.928919 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:42.928890    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-86c7b54d87-6v7bz"]
Jul 18 02:05:42.931470 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:05:42.931443    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9956f87_7b14_4d1e_a576_e62060a9065e.slice/crio-32db286f0efcc71adc9ce4d157ea65e629fe8c83536a460d894777d255680374 WatchSource:0}: Error finding container 32db286f0efcc71adc9ce4d157ea65e629fe8c83536a460d894777d255680374: Status 404 returned error can't find the container with id 32db286f0efcc71adc9ce4d157ea65e629fe8c83536a460d894777d255680374
Jul 18 02:05:43.499082 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:43.499007    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86c7b54d87-6v7bz" event={"ID":"d9956f87-7b14-4d1e-a576-e62060a9065e","Type":"ContainerStarted","Data":"32db286f0efcc71adc9ce4d157ea65e629fe8c83536a460d894777d255680374"}
Jul 18 02:05:43.500475 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:43.500444    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66b54cf598-n59fp" event={"ID":"93b7a318-bb95-4dc5-96a9-f6d33be222f4","Type":"ContainerStarted","Data":"0243e3f129a69043dbf201df485d02a5e9d758074e13980f67d031515fa47894"}
Jul 18 02:05:45.513398 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:45.513358    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86c7b54d87-6v7bz" event={"ID":"d9956f87-7b14-4d1e-a576-e62060a9065e","Type":"ContainerStarted","Data":"a27feec072150da01b79a243721647f214b32743aad0bd991e7c6b20870d585f"}
Jul 18 02:05:45.513871 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:45.513591    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-86c7b54d87-6v7bz"
Jul 18 02:05:45.515034 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:45.515006    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66b54cf598-n59fp" event={"ID":"93b7a318-bb95-4dc5-96a9-f6d33be222f4","Type":"ContainerStarted","Data":"ff733f96bbb403a081f6370b04457478b0eb2350b3e6ad914848ab60959f34c3"}
Jul 18 02:05:45.515154 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:45.515075    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-66b54cf598-n59fp"
Jul 18 02:05:45.536932 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:45.536875    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-86c7b54d87-6v7bz" podStartSLOduration=2.104553682 podStartE2EDuration="4.536857743s" totalImagesPullingTime="2.432304061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:05:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:05:45.535124445 +0000 UTC m=+540.160161129" watchObservedRunningTime="2026-07-18 02:05:45.536857743 +0000 UTC m=+540.161894422"
Jul 18 02:05:45.561831 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:45.561770    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-66b54cf598-n59fp" podStartSLOduration=1.980395251 podStartE2EDuration="4.561754939s" totalImagesPullingTime="2.581359688s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:05:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:05:45.561098884 +0000 UTC m=+540.186135547" watchObservedRunningTime="2026-07-18 02:05:45.561754939 +0000 UTC m=+540.186791607"
Jul 18 02:05:56.522260 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:56.522231    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-66b54cf598-n59fp"
Jul 18 02:05:56.522675 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:56.522297    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-86c7b54d87-6v7bz"
Jul 18 02:05:56.585730 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:56.585691    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-66b54cf598-n59fp"]
Jul 18 02:05:56.585931 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:56.585907    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-66b54cf598-n59fp" podUID="93b7a318-bb95-4dc5-96a9-f6d33be222f4" containerName="manager" containerID="cri-o://ff733f96bbb403a081f6370b04457478b0eb2350b3e6ad914848ab60959f34c3" gracePeriod=30
Jul 18 02:05:56.841008 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:56.840982    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66b54cf598-n59fp"
Jul 18 02:05:56.846604 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:56.846574    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/93b7a318-bb95-4dc5-96a9-f6d33be222f4-webhook-cert\") pod \"93b7a318-bb95-4dc5-96a9-f6d33be222f4\" (UID: \"93b7a318-bb95-4dc5-96a9-f6d33be222f4\") "
Jul 18 02:05:56.846720 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:56.846654    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-znb4l\" (UniqueName: \"kubernetes.io/projected/93b7a318-bb95-4dc5-96a9-f6d33be222f4-kube-api-access-znb4l\") pod \"93b7a318-bb95-4dc5-96a9-f6d33be222f4\" (UID: \"93b7a318-bb95-4dc5-96a9-f6d33be222f4\") "
Jul 18 02:05:56.848706 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:56.848670    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93b7a318-bb95-4dc5-96a9-f6d33be222f4-kube-api-access-znb4l" (OuterVolumeSpecName: "kube-api-access-znb4l") pod "93b7a318-bb95-4dc5-96a9-f6d33be222f4" (UID: "93b7a318-bb95-4dc5-96a9-f6d33be222f4"). InnerVolumeSpecName "kube-api-access-znb4l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:05:56.848706 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:56.848679    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93b7a318-bb95-4dc5-96a9-f6d33be222f4-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "93b7a318-bb95-4dc5-96a9-f6d33be222f4" (UID: "93b7a318-bb95-4dc5-96a9-f6d33be222f4"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:05:56.947370 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:56.947332    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-znb4l\" (UniqueName: \"kubernetes.io/projected/93b7a318-bb95-4dc5-96a9-f6d33be222f4-kube-api-access-znb4l\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:05:56.947370 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:56.947364    2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/93b7a318-bb95-4dc5-96a9-f6d33be222f4-webhook-cert\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:05:57.574055 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:57.574015    2576 generic.go:356] "Generic (PLEG): container finished" podID="93b7a318-bb95-4dc5-96a9-f6d33be222f4" containerID="ff733f96bbb403a081f6370b04457478b0eb2350b3e6ad914848ab60959f34c3" exitCode=0
Jul 18 02:05:57.574494 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:57.574061    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66b54cf598-n59fp"
Jul 18 02:05:57.574494 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:57.574095    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66b54cf598-n59fp" event={"ID":"93b7a318-bb95-4dc5-96a9-f6d33be222f4","Type":"ContainerDied","Data":"ff733f96bbb403a081f6370b04457478b0eb2350b3e6ad914848ab60959f34c3"}
Jul 18 02:05:57.574494 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:57.574150    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66b54cf598-n59fp" event={"ID":"93b7a318-bb95-4dc5-96a9-f6d33be222f4","Type":"ContainerDied","Data":"0243e3f129a69043dbf201df485d02a5e9d758074e13980f67d031515fa47894"}
Jul 18 02:05:57.574494 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:57.574172    2576 scope.go:117] "RemoveContainer" containerID="ff733f96bbb403a081f6370b04457478b0eb2350b3e6ad914848ab60959f34c3"
Jul 18 02:05:57.582713 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:57.582695    2576 scope.go:117] "RemoveContainer" containerID="ff733f96bbb403a081f6370b04457478b0eb2350b3e6ad914848ab60959f34c3"
Jul 18 02:05:57.582987 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:05:57.582969    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff733f96bbb403a081f6370b04457478b0eb2350b3e6ad914848ab60959f34c3\": container with ID starting with ff733f96bbb403a081f6370b04457478b0eb2350b3e6ad914848ab60959f34c3 not found: ID does not exist" containerID="ff733f96bbb403a081f6370b04457478b0eb2350b3e6ad914848ab60959f34c3"
Jul 18 02:05:57.583044 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:57.582995    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff733f96bbb403a081f6370b04457478b0eb2350b3e6ad914848ab60959f34c3"} err="failed to get container status \"ff733f96bbb403a081f6370b04457478b0eb2350b3e6ad914848ab60959f34c3\": rpc error: code = NotFound desc = could not find container \"ff733f96bbb403a081f6370b04457478b0eb2350b3e6ad914848ab60959f34c3\": container with ID starting with ff733f96bbb403a081f6370b04457478b0eb2350b3e6ad914848ab60959f34c3 not found: ID does not exist"
Jul 18 02:05:57.594818 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:57.594783    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-66b54cf598-n59fp"]
Jul 18 02:05:57.599288 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:57.599264    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-66b54cf598-n59fp"]
Jul 18 02:05:57.969480 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:05:57.969446    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93b7a318-bb95-4dc5-96a9-f6d33be222f4" path="/var/lib/kubelet/pods/93b7a318-bb95-4dc5-96a9-f6d33be222f4/volumes"
Jul 18 02:06:17.855800 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:17.855758    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5c45978f8c-zkfkw"]
Jul 18 02:06:17.856284 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:17.856157    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93b7a318-bb95-4dc5-96a9-f6d33be222f4" containerName="manager"
Jul 18 02:06:17.856284 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:17.856170    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="93b7a318-bb95-4dc5-96a9-f6d33be222f4" containerName="manager"
Jul 18 02:06:17.856284 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:17.856249    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="93b7a318-bb95-4dc5-96a9-f6d33be222f4" containerName="manager"
Jul 18 02:06:17.865744 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:17.865718    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c45978f8c-zkfkw"
Jul 18 02:06:17.868167 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:17.868140    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 18 02:06:17.868688 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:17.868670    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-rbxdz\""
Jul 18 02:06:17.870149 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:17.870123    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c45978f8c-zkfkw"]
Jul 18 02:06:17.928335 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:17.928295    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e782267-e161-40fd-90c3-2974b1fc3fd7-openshift-service-ca\") pod \"maas-api-5c45978f8c-zkfkw\" (UID: \"3e782267-e161-40fd-90c3-2974b1fc3fd7\") " pod="odh-ai-gateway-infra/maas-api-5c45978f8c-zkfkw"
Jul 18 02:06:17.928549 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:17.928349    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e782267-e161-40fd-90c3-2974b1fc3fd7-maas-api-tls\") pod \"maas-api-5c45978f8c-zkfkw\" (UID: \"3e782267-e161-40fd-90c3-2974b1fc3fd7\") " pod="odh-ai-gateway-infra/maas-api-5c45978f8c-zkfkw"
Jul 18 02:06:17.928549 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:17.928483    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gn8p8\" (UniqueName: \"kubernetes.io/projected/3e782267-e161-40fd-90c3-2974b1fc3fd7-kube-api-access-gn8p8\") pod \"maas-api-5c45978f8c-zkfkw\" (UID: \"3e782267-e161-40fd-90c3-2974b1fc3fd7\") " pod="odh-ai-gateway-infra/maas-api-5c45978f8c-zkfkw"
Jul 18 02:06:18.029769 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:18.029735    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e782267-e161-40fd-90c3-2974b1fc3fd7-openshift-service-ca\") pod \"maas-api-5c45978f8c-zkfkw\" (UID: \"3e782267-e161-40fd-90c3-2974b1fc3fd7\") " pod="odh-ai-gateway-infra/maas-api-5c45978f8c-zkfkw"
Jul 18 02:06:18.029944 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:18.029775    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e782267-e161-40fd-90c3-2974b1fc3fd7-maas-api-tls\") pod \"maas-api-5c45978f8c-zkfkw\" (UID: \"3e782267-e161-40fd-90c3-2974b1fc3fd7\") " pod="odh-ai-gateway-infra/maas-api-5c45978f8c-zkfkw"
Jul 18 02:06:18.029944 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:06:18.029878    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Jul 18 02:06:18.029944 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:18.029905    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn8p8\" (UniqueName: \"kubernetes.io/projected/3e782267-e161-40fd-90c3-2974b1fc3fd7-kube-api-access-gn8p8\") pod \"maas-api-5c45978f8c-zkfkw\" (UID: \"3e782267-e161-40fd-90c3-2974b1fc3fd7\") " pod="odh-ai-gateway-infra/maas-api-5c45978f8c-zkfkw"
Jul 18 02:06:18.029944 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:06:18.029936    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e782267-e161-40fd-90c3-2974b1fc3fd7-maas-api-tls podName:3e782267-e161-40fd-90c3-2974b1fc3fd7 nodeName:}" failed. No retries permitted until 2026-07-18 02:06:18.529918945 +0000 UTC m=+573.154955590 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3e782267-e161-40fd-90c3-2974b1fc3fd7-maas-api-tls") pod "maas-api-5c45978f8c-zkfkw" (UID: "3e782267-e161-40fd-90c3-2974b1fc3fd7") : secret "maas-api-serving-cert" not found
Jul 18 02:06:18.030459 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:18.030411    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e782267-e161-40fd-90c3-2974b1fc3fd7-openshift-service-ca\") pod \"maas-api-5c45978f8c-zkfkw\" (UID: \"3e782267-e161-40fd-90c3-2974b1fc3fd7\") " pod="odh-ai-gateway-infra/maas-api-5c45978f8c-zkfkw"
Jul 18 02:06:18.039534 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:18.039508    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn8p8\" (UniqueName: \"kubernetes.io/projected/3e782267-e161-40fd-90c3-2974b1fc3fd7-kube-api-access-gn8p8\") pod \"maas-api-5c45978f8c-zkfkw\" (UID: \"3e782267-e161-40fd-90c3-2974b1fc3fd7\") " pod="odh-ai-gateway-infra/maas-api-5c45978f8c-zkfkw"
Jul 18 02:06:18.415579 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:18.415546    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-mwlv7"]
Jul 18 02:06:18.418706 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:18.418684    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-mwlv7"
Jul 18 02:06:18.421767 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:18.421752    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 18 02:06:18.421858 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:18.421831    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-dgccx\""
Jul 18 02:06:18.430929 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:18.430899    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-mwlv7"]
Jul 18 02:06:18.535296 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:18.535261    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e782267-e161-40fd-90c3-2974b1fc3fd7-maas-api-tls\") pod \"maas-api-5c45978f8c-zkfkw\" (UID: \"3e782267-e161-40fd-90c3-2974b1fc3fd7\") " pod="odh-ai-gateway-infra/maas-api-5c45978f8c-zkfkw"
Jul 18 02:06:18.535464 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:18.535359    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h859m\" (UniqueName: \"kubernetes.io/projected/65c0faa6-4db2-4c9b-a49c-001bbfe83956-kube-api-access-h859m\") pod \"payload-pre-processing-7677b774bb-mwlv7\" (UID: \"65c0faa6-4db2-4c9b-a49c-001bbfe83956\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-mwlv7"
Jul 18 02:06:18.535464 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:18.535413    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65c0faa6-4db2-4c9b-a49c-001bbfe83956-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-mwlv7\" (UID: \"65c0faa6-4db2-4c9b-a49c-001bbfe83956\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-mwlv7"
Jul 18 02:06:18.535464 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:06:18.535416    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Jul 18 02:06:18.535591 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:06:18.535513    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e782267-e161-40fd-90c3-2974b1fc3fd7-maas-api-tls podName:3e782267-e161-40fd-90c3-2974b1fc3fd7 nodeName:}" failed. No retries permitted until 2026-07-18 02:06:19.535492504 +0000 UTC m=+574.160529166 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3e782267-e161-40fd-90c3-2974b1fc3fd7-maas-api-tls") pod "maas-api-5c45978f8c-zkfkw" (UID: "3e782267-e161-40fd-90c3-2974b1fc3fd7") : secret "maas-api-serving-cert" not found
Jul 18 02:06:18.635994 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:18.635959    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65c0faa6-4db2-4c9b-a49c-001bbfe83956-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-mwlv7\" (UID: \"65c0faa6-4db2-4c9b-a49c-001bbfe83956\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-mwlv7"
Jul 18 02:06:18.636173 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:18.636068    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h859m\" (UniqueName: \"kubernetes.io/projected/65c0faa6-4db2-4c9b-a49c-001bbfe83956-kube-api-access-h859m\") pod \"payload-pre-processing-7677b774bb-mwlv7\" (UID: \"65c0faa6-4db2-4c9b-a49c-001bbfe83956\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-mwlv7"
Jul 18 02:06:18.636740 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:18.636720    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65c0faa6-4db2-4c9b-a49c-001bbfe83956-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-mwlv7\" (UID: \"65c0faa6-4db2-4c9b-a49c-001bbfe83956\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-mwlv7"
Jul 18 02:06:18.644705 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:18.644681    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h859m\" (UniqueName: \"kubernetes.io/projected/65c0faa6-4db2-4c9b-a49c-001bbfe83956-kube-api-access-h859m\") pod \"payload-pre-processing-7677b774bb-mwlv7\" (UID: \"65c0faa6-4db2-4c9b-a49c-001bbfe83956\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-mwlv7"
Jul 18 02:06:18.729809 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:18.729710    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-mwlv7"
Jul 18 02:06:18.865596 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:18.865568    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-mwlv7"]
Jul 18 02:06:18.868594 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:06:18.868563    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65c0faa6_4db2_4c9b_a49c_001bbfe83956.slice/crio-32c1cbcd30f726da3ea25a0d9fda64561b907e7762247584bf68c0a295b9d91a WatchSource:0}: Error finding container 32c1cbcd30f726da3ea25a0d9fda64561b907e7762247584bf68c0a295b9d91a: Status 404 returned error can't find the container with id 32c1cbcd30f726da3ea25a0d9fda64561b907e7762247584bf68c0a295b9d91a
Jul 18 02:06:19.543880 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:19.543833    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e782267-e161-40fd-90c3-2974b1fc3fd7-maas-api-tls\") pod \"maas-api-5c45978f8c-zkfkw\" (UID: \"3e782267-e161-40fd-90c3-2974b1fc3fd7\") " pod="odh-ai-gateway-infra/maas-api-5c45978f8c-zkfkw"
Jul 18 02:06:19.548750 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:19.548693    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e782267-e161-40fd-90c3-2974b1fc3fd7-maas-api-tls\") pod \"maas-api-5c45978f8c-zkfkw\" (UID: \"3e782267-e161-40fd-90c3-2974b1fc3fd7\") " pod="odh-ai-gateway-infra/maas-api-5c45978f8c-zkfkw"
Jul 18 02:06:19.678314 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:19.678035    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c45978f8c-zkfkw"
Jul 18 02:06:19.684701 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:19.684633    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-mwlv7" event={"ID":"65c0faa6-4db2-4c9b-a49c-001bbfe83956","Type":"ContainerStarted","Data":"32c1cbcd30f726da3ea25a0d9fda64561b907e7762247584bf68c0a295b9d91a"}
Jul 18 02:06:19.845704 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:19.845664    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c45978f8c-zkfkw"]
Jul 18 02:06:19.847517 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:06:19.847485    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e782267_e161_40fd_90c3_2974b1fc3fd7.slice/crio-a1b0e8a11ac0403e1a2bfe443e4c383eb81d8e676f185beeaa4fb7b94171fde9 WatchSource:0}: Error finding container a1b0e8a11ac0403e1a2bfe443e4c383eb81d8e676f185beeaa4fb7b94171fde9: Status 404 returned error can't find the container with id a1b0e8a11ac0403e1a2bfe443e4c383eb81d8e676f185beeaa4fb7b94171fde9
Jul 18 02:06:20.690885 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:20.690796    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c45978f8c-zkfkw" event={"ID":"3e782267-e161-40fd-90c3-2974b1fc3fd7","Type":"ContainerStarted","Data":"a1b0e8a11ac0403e1a2bfe443e4c383eb81d8e676f185beeaa4fb7b94171fde9"}
Jul 18 02:06:20.692181 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:20.692155    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-mwlv7" event={"ID":"65c0faa6-4db2-4c9b-a49c-001bbfe83956","Type":"ContainerStarted","Data":"99cc44d12845069f64b86a412b3f8147b9cfd044e8deb3f88e2c51a42abd923d"}
Jul 18 02:06:20.692312 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:20.692269    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-mwlv7"
Jul 18 02:06:20.708074 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:20.708022    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-mwlv7" podStartSLOduration=1.187585346 podStartE2EDuration="2.708007356s" totalImagesPullingTime="1.52042201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:06:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:06:20.707423076 +0000 UTC m=+575.332459734" watchObservedRunningTime="2026-07-18 02:06:20.708007356 +0000 UTC m=+575.333044024"
Jul 18 02:06:21.699161 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:21.699119    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c45978f8c-zkfkw" event={"ID":"3e782267-e161-40fd-90c3-2974b1fc3fd7","Type":"ContainerStarted","Data":"08ac93f5f7552e36d242edef136e5a8a238343191e5441af8c4a0e3767d98cc6"}
Jul 18 02:06:21.716300 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:21.716245    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5c45978f8c-zkfkw" podStartSLOduration=3.245161954 podStartE2EDuration="4.716228819s" totalImagesPullingTime="1.471066865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:06:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:06:21.71496083 +0000 UTC m=+576.339997500" watchObservedRunningTime="2026-07-18 02:06:21.716228819 +0000 UTC m=+576.341265485"
Jul 18 02:06:22.705041 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:22.705011    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5c45978f8c-zkfkw"
Jul 18 02:06:28.715324 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:28.715293    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5c45978f8c-zkfkw"
Jul 18 02:06:31.701757 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:31.701726    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-mwlv7"
Jul 18 02:06:41.611151 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:41.611114    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5"]
Jul 18 02:06:41.615455 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:41.615415    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5"
Jul 18 02:06:41.618955 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:41.618895    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-7gdmh\""
Jul 18 02:06:41.618955 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:41.618906    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 18 02:06:41.618955 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:41.618908    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 18 02:06:41.619232 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:41.619217    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 18 02:06:41.626461 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:41.626420    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5"]
Jul 18 02:06:41.739168 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:41.739125    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1eb761e4-f4f1-4cfe-a0aa-9e7c8a0db609-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5\" (UID: \"1eb761e4-f4f1-4cfe-a0aa-9e7c8a0db609\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5"
Jul 18 02:06:41.739349 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:41.739172    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4mj8\" (UniqueName: \"kubernetes.io/projected/1eb761e4-f4f1-4cfe-a0aa-9e7c8a0db609-kube-api-access-p4mj8\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5\" (UID: \"1eb761e4-f4f1-4cfe-a0aa-9e7c8a0db609\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5"
Jul 18 02:06:41.739349 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:41.739256    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1eb761e4-f4f1-4cfe-a0aa-9e7c8a0db609-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5\" (UID: \"1eb761e4-f4f1-4cfe-a0aa-9e7c8a0db609\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5"
Jul 18 02:06:41.739349 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:41.739326    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1eb761e4-f4f1-4cfe-a0aa-9e7c8a0db609-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5\" (UID: \"1eb761e4-f4f1-4cfe-a0aa-9e7c8a0db609\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5"
Jul 18 02:06:41.739349 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:41.739347    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1eb761e4-f4f1-4cfe-a0aa-9e7c8a0db609-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5\" (UID: \"1eb761e4-f4f1-4cfe-a0aa-9e7c8a0db609\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5"
Jul 18 02:06:41.739544 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:41.739389    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1eb761e4-f4f1-4cfe-a0aa-9e7c8a0db609-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5\" (UID: \"1eb761e4-f4f1-4cfe-a0aa-9e7c8a0db609\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5"
Jul 18 02:06:41.840055 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:41.840015    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1eb761e4-f4f1-4cfe-a0aa-9e7c8a0db609-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5\" (UID: \"1eb761e4-f4f1-4cfe-a0aa-9e7c8a0db609\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5"
Jul 18 02:06:41.840055 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:41.840057    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4mj8\" (UniqueName: \"kubernetes.io/projected/1eb761e4-f4f1-4cfe-a0aa-9e7c8a0db609-kube-api-access-p4mj8\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5\" (UID: \"1eb761e4-f4f1-4cfe-a0aa-9e7c8a0db609\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5"
Jul 18 02:06:41.840272 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:41.840183    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1eb761e4-f4f1-4cfe-a0aa-9e7c8a0db609-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5\" (UID: \"1eb761e4-f4f1-4cfe-a0aa-9e7c8a0db609\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5"
Jul 18 02:06:41.840314 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:41.840275    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1eb761e4-f4f1-4cfe-a0aa-9e7c8a0db609-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5\" (UID: \"1eb761e4-f4f1-4cfe-a0aa-9e7c8a0db609\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5"
Jul 18 02:06:41.840314 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:41.840305    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1eb761e4-f4f1-4cfe-a0aa-9e7c8a0db609-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5\" (UID: \"1eb761e4-f4f1-4cfe-a0aa-9e7c8a0db609\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5"
Jul 18 02:06:41.840387 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:41.840364    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1eb761e4-f4f1-4cfe-a0aa-9e7c8a0db609-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5\" (UID: \"1eb761e4-f4f1-4cfe-a0aa-9e7c8a0db609\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5"
Jul 18 02:06:41.840504 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:41.840486    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1eb761e4-f4f1-4cfe-a0aa-9e7c8a0db609-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5\" (UID: \"1eb761e4-f4f1-4cfe-a0aa-9e7c8a0db609\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5"
Jul 18 02:06:41.840578 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:41.840536    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1eb761e4-f4f1-4cfe-a0aa-9e7c8a0db609-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5\" (UID: \"1eb761e4-f4f1-4cfe-a0aa-9e7c8a0db609\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5"
Jul 18 02:06:41.840646 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:41.840626    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1eb761e4-f4f1-4cfe-a0aa-9e7c8a0db609-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5\" (UID: \"1eb761e4-f4f1-4cfe-a0aa-9e7c8a0db609\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5"
Jul 18 02:06:41.842610 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:41.842588    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1eb761e4-f4f1-4cfe-a0aa-9e7c8a0db609-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5\" (UID: \"1eb761e4-f4f1-4cfe-a0aa-9e7c8a0db609\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5"
Jul 18 02:06:41.842805 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:41.842787    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1eb761e4-f4f1-4cfe-a0aa-9e7c8a0db609-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5\" (UID: \"1eb761e4-f4f1-4cfe-a0aa-9e7c8a0db609\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5"
Jul 18 02:06:41.847566 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:41.847544    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4mj8\" (UniqueName: \"kubernetes.io/projected/1eb761e4-f4f1-4cfe-a0aa-9e7c8a0db609-kube-api-access-p4mj8\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5\" (UID: \"1eb761e4-f4f1-4cfe-a0aa-9e7c8a0db609\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5"
Jul 18 02:06:41.928358 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:41.928251    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5"
Jul 18 02:06:42.069391 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:42.069362    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5"]
Jul 18 02:06:42.072464 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:06:42.072400    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1eb761e4_f4f1_4cfe_a0aa_9e7c8a0db609.slice/crio-1edb39706e7e427de1f246935d59d6a494f85dc14ad961af593cb231395364fa WatchSource:0}: Error finding container 1edb39706e7e427de1f246935d59d6a494f85dc14ad961af593cb231395364fa: Status 404 returned error can't find the container with id 1edb39706e7e427de1f246935d59d6a494f85dc14ad961af593cb231395364fa
Jul 18 02:06:42.805408 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:42.805365    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5" event={"ID":"1eb761e4-f4f1-4cfe-a0aa-9e7c8a0db609","Type":"ContainerStarted","Data":"1edb39706e7e427de1f246935d59d6a494f85dc14ad961af593cb231395364fa"}
Jul 18 02:06:43.816316 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:43.816278    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5" event={"ID":"1eb761e4-f4f1-4cfe-a0aa-9e7c8a0db609","Type":"ContainerStarted","Data":"8eae6c67fdb8f5384016bca7594dcbe503c0b7f29947e74d66e608cd8968813d"}
Jul 18 02:06:43.832090 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:43.832032    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5" podStartSLOduration=1.694203607 podStartE2EDuration="2.832015523s" totalImagesPullingTime="1.137811916s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:06:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:06:43.831227192 +0000 UTC m=+598.456263861" watchObservedRunningTime="2026-07-18 02:06:43.832015523 +0000 UTC m=+598.457052189"
Jul 18 02:06:45.826092 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:45.826063    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-bfrdg_9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777/console-operator/1.log"
Jul 18 02:06:45.835617 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:45.835589    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-bfrdg_9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777/console-operator/1.log"
Jul 18 02:06:51.928531 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:51.928490    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5"
Jul 18 02:06:51.931020 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:51.928564    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5"
Jul 18 02:06:51.941384 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:51.941357    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5"
Jul 18 02:06:52.870946 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:06:52.870904    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5"
Jul 18 02:10:11.112222 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:10:11.112135    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-86c7b54d87-6v7bz"]
Jul 18 02:10:11.112826 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:10:11.112466    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-86c7b54d87-6v7bz" podUID="d9956f87-7b14-4d1e-a576-e62060a9065e" containerName="manager" containerID="cri-o://a27feec072150da01b79a243721647f214b32743aad0bd991e7c6b20870d585f" gracePeriod=30
Jul 18 02:10:11.374771 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:10:11.374746    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86c7b54d87-6v7bz"
Jul 18 02:10:11.511117 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:10:11.511078    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvk6s\" (UniqueName: \"kubernetes.io/projected/d9956f87-7b14-4d1e-a576-e62060a9065e-kube-api-access-nvk6s\") pod \"d9956f87-7b14-4d1e-a576-e62060a9065e\" (UID: \"d9956f87-7b14-4d1e-a576-e62060a9065e\") "
Jul 18 02:10:11.511282 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:10:11.511138    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d9956f87-7b14-4d1e-a576-e62060a9065e-webhook-cert\") pod \"d9956f87-7b14-4d1e-a576-e62060a9065e\" (UID: \"d9956f87-7b14-4d1e-a576-e62060a9065e\") "
Jul 18 02:10:11.513252 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:10:11.513210    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9956f87-7b14-4d1e-a576-e62060a9065e-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d9956f87-7b14-4d1e-a576-e62060a9065e" (UID: "d9956f87-7b14-4d1e-a576-e62060a9065e"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:10:11.513252 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:10:11.513237    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9956f87-7b14-4d1e-a576-e62060a9065e-kube-api-access-nvk6s" (OuterVolumeSpecName: "kube-api-access-nvk6s") pod "d9956f87-7b14-4d1e-a576-e62060a9065e" (UID: "d9956f87-7b14-4d1e-a576-e62060a9065e"). InnerVolumeSpecName "kube-api-access-nvk6s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:10:11.611978 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:10:11.611945    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvk6s\" (UniqueName: \"kubernetes.io/projected/d9956f87-7b14-4d1e-a576-e62060a9065e-kube-api-access-nvk6s\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:10:11.611978 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:10:11.611976    2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d9956f87-7b14-4d1e-a576-e62060a9065e-webhook-cert\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:10:11.837054 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:10:11.837021    2576 generic.go:356] "Generic (PLEG): container finished" podID="d9956f87-7b14-4d1e-a576-e62060a9065e" containerID="a27feec072150da01b79a243721647f214b32743aad0bd991e7c6b20870d585f" exitCode=0
Jul 18 02:10:11.837257 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:10:11.837065    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86c7b54d87-6v7bz"
Jul 18 02:10:11.837257 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:10:11.837110    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86c7b54d87-6v7bz" event={"ID":"d9956f87-7b14-4d1e-a576-e62060a9065e","Type":"ContainerDied","Data":"a27feec072150da01b79a243721647f214b32743aad0bd991e7c6b20870d585f"}
Jul 18 02:10:11.837257 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:10:11.837167    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86c7b54d87-6v7bz" event={"ID":"d9956f87-7b14-4d1e-a576-e62060a9065e","Type":"ContainerDied","Data":"32db286f0efcc71adc9ce4d157ea65e629fe8c83536a460d894777d255680374"}
Jul 18 02:10:11.837257 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:10:11.837199    2576 scope.go:117] "RemoveContainer" containerID="a27feec072150da01b79a243721647f214b32743aad0bd991e7c6b20870d585f"
Jul 18 02:10:11.846527 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:10:11.846490    2576 scope.go:117] "RemoveContainer" containerID="a27feec072150da01b79a243721647f214b32743aad0bd991e7c6b20870d585f"
Jul 18 02:10:11.846789 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:10:11.846763    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a27feec072150da01b79a243721647f214b32743aad0bd991e7c6b20870d585f\": container with ID starting with a27feec072150da01b79a243721647f214b32743aad0bd991e7c6b20870d585f not found: ID does not exist" containerID="a27feec072150da01b79a243721647f214b32743aad0bd991e7c6b20870d585f"
Jul 18 02:10:11.846842 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:10:11.846798    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a27feec072150da01b79a243721647f214b32743aad0bd991e7c6b20870d585f"} err="failed to get container status \"a27feec072150da01b79a243721647f214b32743aad0bd991e7c6b20870d585f\": rpc error: code = NotFound desc = could not find container \"a27feec072150da01b79a243721647f214b32743aad0bd991e7c6b20870d585f\": container with ID starting with a27feec072150da01b79a243721647f214b32743aad0bd991e7c6b20870d585f not found: ID does not exist"
Jul 18 02:10:11.859697 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:10:11.859670    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-86c7b54d87-6v7bz"]
Jul 18 02:10:11.863248 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:10:11.863224    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-86c7b54d87-6v7bz"]
Jul 18 02:10:11.970225 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:10:11.970186    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9956f87-7b14-4d1e-a576-e62060a9065e" path="/var/lib/kubelet/pods/d9956f87-7b14-4d1e-a576-e62060a9065e/volumes"
Jul 18 02:11:12.648885 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:12.648834    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-86c7b54d87-cxpfr"]
Jul 18 02:11:12.649364 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:12.649243    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9956f87-7b14-4d1e-a576-e62060a9065e" containerName="manager"
Jul 18 02:11:12.649364 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:12.649257    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9956f87-7b14-4d1e-a576-e62060a9065e" containerName="manager"
Jul 18 02:11:12.649364 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:12.649321    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9956f87-7b14-4d1e-a576-e62060a9065e" containerName="manager"
Jul 18 02:11:12.653776 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:12.653758    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86c7b54d87-cxpfr"
Jul 18 02:11:12.656938 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:12.656907    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 18 02:11:12.657060 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:12.656909    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 18 02:11:12.657060 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:12.656949    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-n7v75\""
Jul 18 02:11:12.661192 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:12.660856    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-86c7b54d87-cxpfr"]
Jul 18 02:11:12.743017 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:12.742975    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8bf64cab-65d0-4d6b-a86e-28444b00cc9e-webhook-cert\") pod \"maas-controller-86c7b54d87-cxpfr\" (UID: \"8bf64cab-65d0-4d6b-a86e-28444b00cc9e\") " pod="opendatahub/maas-controller-86c7b54d87-cxpfr"
Jul 18 02:11:12.743194 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:12.743045    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vqjx\" (UniqueName: \"kubernetes.io/projected/8bf64cab-65d0-4d6b-a86e-28444b00cc9e-kube-api-access-5vqjx\") pod \"maas-controller-86c7b54d87-cxpfr\" (UID: \"8bf64cab-65d0-4d6b-a86e-28444b00cc9e\") " pod="opendatahub/maas-controller-86c7b54d87-cxpfr"
Jul 18 02:11:12.843691 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:12.843642    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vqjx\" (UniqueName: \"kubernetes.io/projected/8bf64cab-65d0-4d6b-a86e-28444b00cc9e-kube-api-access-5vqjx\") pod \"maas-controller-86c7b54d87-cxpfr\" (UID: \"8bf64cab-65d0-4d6b-a86e-28444b00cc9e\") " pod="opendatahub/maas-controller-86c7b54d87-cxpfr"
Jul 18 02:11:12.843875 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:12.843763    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8bf64cab-65d0-4d6b-a86e-28444b00cc9e-webhook-cert\") pod \"maas-controller-86c7b54d87-cxpfr\" (UID: \"8bf64cab-65d0-4d6b-a86e-28444b00cc9e\") " pod="opendatahub/maas-controller-86c7b54d87-cxpfr"
Jul 18 02:11:12.846263 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:12.846238    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8bf64cab-65d0-4d6b-a86e-28444b00cc9e-webhook-cert\") pod \"maas-controller-86c7b54d87-cxpfr\" (UID: \"8bf64cab-65d0-4d6b-a86e-28444b00cc9e\") " pod="opendatahub/maas-controller-86c7b54d87-cxpfr"
Jul 18 02:11:12.851477 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:12.851423    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vqjx\" (UniqueName: \"kubernetes.io/projected/8bf64cab-65d0-4d6b-a86e-28444b00cc9e-kube-api-access-5vqjx\") pod \"maas-controller-86c7b54d87-cxpfr\" (UID: \"8bf64cab-65d0-4d6b-a86e-28444b00cc9e\") " pod="opendatahub/maas-controller-86c7b54d87-cxpfr"
Jul 18 02:11:12.964500 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:12.964403    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86c7b54d87-cxpfr"
Jul 18 02:11:13.101860 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:13.101831    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-86c7b54d87-cxpfr"]
Jul 18 02:11:13.103858 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:11:13.103829    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bf64cab_65d0_4d6b_a86e_28444b00cc9e.slice/crio-ddb41f53a57751df4a210263c3e965b07a8185649523519c06cb457aaeddc232 WatchSource:0}: Error finding container ddb41f53a57751df4a210263c3e965b07a8185649523519c06cb457aaeddc232: Status 404 returned error can't find the container with id ddb41f53a57751df4a210263c3e965b07a8185649523519c06cb457aaeddc232
Jul 18 02:11:13.105135 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:13.105118    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 02:11:13.136528 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:13.136489    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86c7b54d87-cxpfr" event={"ID":"8bf64cab-65d0-4d6b-a86e-28444b00cc9e","Type":"ContainerStarted","Data":"ddb41f53a57751df4a210263c3e965b07a8185649523519c06cb457aaeddc232"}
Jul 18 02:11:14.142777 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:14.142735    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86c7b54d87-cxpfr" event={"ID":"8bf64cab-65d0-4d6b-a86e-28444b00cc9e","Type":"ContainerStarted","Data":"ab39575b4d29647bcf979303f82624b0fea3a37003c89e7556280424425b0ca8"}
Jul 18 02:11:14.143171 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:14.142931    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-86c7b54d87-cxpfr"
Jul 18 02:11:14.158919 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:14.158870    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-86c7b54d87-cxpfr" podStartSLOduration=1.587945474 podStartE2EDuration="2.1588545s" totalImagesPullingTime="570.909026ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:11:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:11:14.156822849 +0000 UTC m=+868.781859516" watchObservedRunningTime="2026-07-18 02:11:14.1588545 +0000 UTC m=+868.783891250"
Jul 18 02:11:25.149637 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:25.149599    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-86c7b54d87-cxpfr"
Jul 18 02:11:29.797884 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:29.797849    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-86c7b54d87-cxpfr"]
Jul 18 02:11:29.798277 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:29.798082    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-86c7b54d87-cxpfr" podUID="8bf64cab-65d0-4d6b-a86e-28444b00cc9e" containerName="manager" containerID="cri-o://ab39575b4d29647bcf979303f82624b0fea3a37003c89e7556280424425b0ca8" gracePeriod=30
Jul 18 02:11:30.059107 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:30.059047    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86c7b54d87-cxpfr"
Jul 18 02:11:30.098147 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:30.098110    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8bf64cab-65d0-4d6b-a86e-28444b00cc9e-webhook-cert\") pod \"8bf64cab-65d0-4d6b-a86e-28444b00cc9e\" (UID: \"8bf64cab-65d0-4d6b-a86e-28444b00cc9e\") "
Jul 18 02:11:30.098331 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:30.098237    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5vqjx\" (UniqueName: \"kubernetes.io/projected/8bf64cab-65d0-4d6b-a86e-28444b00cc9e-kube-api-access-5vqjx\") pod \"8bf64cab-65d0-4d6b-a86e-28444b00cc9e\" (UID: \"8bf64cab-65d0-4d6b-a86e-28444b00cc9e\") "
Jul 18 02:11:30.100675 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:30.100639    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bf64cab-65d0-4d6b-a86e-28444b00cc9e-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "8bf64cab-65d0-4d6b-a86e-28444b00cc9e" (UID: "8bf64cab-65d0-4d6b-a86e-28444b00cc9e"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:11:30.100821 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:30.100763    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bf64cab-65d0-4d6b-a86e-28444b00cc9e-kube-api-access-5vqjx" (OuterVolumeSpecName: "kube-api-access-5vqjx") pod "8bf64cab-65d0-4d6b-a86e-28444b00cc9e" (UID: "8bf64cab-65d0-4d6b-a86e-28444b00cc9e"). InnerVolumeSpecName "kube-api-access-5vqjx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:11:30.199477 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:30.199421    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vqjx\" (UniqueName: \"kubernetes.io/projected/8bf64cab-65d0-4d6b-a86e-28444b00cc9e-kube-api-access-5vqjx\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:11:30.199477 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:30.199471    2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8bf64cab-65d0-4d6b-a86e-28444b00cc9e-webhook-cert\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:11:30.225803 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:30.225767    2576 generic.go:356] "Generic (PLEG): container finished" podID="8bf64cab-65d0-4d6b-a86e-28444b00cc9e" containerID="ab39575b4d29647bcf979303f82624b0fea3a37003c89e7556280424425b0ca8" exitCode=0
Jul 18 02:11:30.225943 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:30.225828    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86c7b54d87-cxpfr"
Jul 18 02:11:30.225943 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:30.225860    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86c7b54d87-cxpfr" event={"ID":"8bf64cab-65d0-4d6b-a86e-28444b00cc9e","Type":"ContainerDied","Data":"ab39575b4d29647bcf979303f82624b0fea3a37003c89e7556280424425b0ca8"}
Jul 18 02:11:30.225943 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:30.225920    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86c7b54d87-cxpfr" event={"ID":"8bf64cab-65d0-4d6b-a86e-28444b00cc9e","Type":"ContainerDied","Data":"ddb41f53a57751df4a210263c3e965b07a8185649523519c06cb457aaeddc232"}
Jul 18 02:11:30.226053 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:30.225944    2576 scope.go:117] "RemoveContainer" containerID="ab39575b4d29647bcf979303f82624b0fea3a37003c89e7556280424425b0ca8"
Jul 18 02:11:30.234856 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:30.234837    2576 scope.go:117] "RemoveContainer" containerID="ab39575b4d29647bcf979303f82624b0fea3a37003c89e7556280424425b0ca8"
Jul 18 02:11:30.235127 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:11:30.235108    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab39575b4d29647bcf979303f82624b0fea3a37003c89e7556280424425b0ca8\": container with ID starting with ab39575b4d29647bcf979303f82624b0fea3a37003c89e7556280424425b0ca8 not found: ID does not exist" containerID="ab39575b4d29647bcf979303f82624b0fea3a37003c89e7556280424425b0ca8"
Jul 18 02:11:30.235215 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:30.235132    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab39575b4d29647bcf979303f82624b0fea3a37003c89e7556280424425b0ca8"} err="failed to get container status \"ab39575b4d29647bcf979303f82624b0fea3a37003c89e7556280424425b0ca8\": rpc error: code = NotFound desc = could not find container \"ab39575b4d29647bcf979303f82624b0fea3a37003c89e7556280424425b0ca8\": container with ID starting with ab39575b4d29647bcf979303f82624b0fea3a37003c89e7556280424425b0ca8 not found: ID does not exist"
Jul 18 02:11:30.247928 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:30.247899    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-86c7b54d87-cxpfr"]
Jul 18 02:11:30.251359 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:30.251338    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-86c7b54d87-cxpfr"]
Jul 18 02:11:31.056707 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:31.056673    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-86c7b54d87-ks9zb"]
Jul 18 02:11:31.057099 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:31.057049    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bf64cab-65d0-4d6b-a86e-28444b00cc9e" containerName="manager"
Jul 18 02:11:31.057099 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:31.057059    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bf64cab-65d0-4d6b-a86e-28444b00cc9e" containerName="manager"
Jul 18 02:11:31.057166 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:31.057129    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bf64cab-65d0-4d6b-a86e-28444b00cc9e" containerName="manager"
Jul 18 02:11:31.061488 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:31.061466    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86c7b54d87-ks9zb"
Jul 18 02:11:31.063711 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:31.063687    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 18 02:11:31.063853 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:31.063697    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 18 02:11:31.064520 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:31.064499    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-n7v75\""
Jul 18 02:11:31.069048 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:31.069027    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-86c7b54d87-ks9zb"]
Jul 18 02:11:31.106156 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:31.106123    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/de375459-cd35-459b-bffd-1a82e0ece9a3-webhook-cert\") pod \"maas-controller-86c7b54d87-ks9zb\" (UID: \"de375459-cd35-459b-bffd-1a82e0ece9a3\") " pod="opendatahub/maas-controller-86c7b54d87-ks9zb"
Jul 18 02:11:31.106341 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:31.106169    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxhrs\" (UniqueName: \"kubernetes.io/projected/de375459-cd35-459b-bffd-1a82e0ece9a3-kube-api-access-zxhrs\") pod \"maas-controller-86c7b54d87-ks9zb\" (UID: \"de375459-cd35-459b-bffd-1a82e0ece9a3\") " pod="opendatahub/maas-controller-86c7b54d87-ks9zb"
Jul 18 02:11:31.207082 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:31.207043    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/de375459-cd35-459b-bffd-1a82e0ece9a3-webhook-cert\") pod \"maas-controller-86c7b54d87-ks9zb\" (UID: \"de375459-cd35-459b-bffd-1a82e0ece9a3\") " pod="opendatahub/maas-controller-86c7b54d87-ks9zb"
Jul 18 02:11:31.207326 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:31.207105    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxhrs\" (UniqueName: \"kubernetes.io/projected/de375459-cd35-459b-bffd-1a82e0ece9a3-kube-api-access-zxhrs\") pod \"maas-controller-86c7b54d87-ks9zb\" (UID: \"de375459-cd35-459b-bffd-1a82e0ece9a3\") " pod="opendatahub/maas-controller-86c7b54d87-ks9zb"
Jul 18 02:11:31.209485 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:31.209464    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/de375459-cd35-459b-bffd-1a82e0ece9a3-webhook-cert\") pod \"maas-controller-86c7b54d87-ks9zb\" (UID: \"de375459-cd35-459b-bffd-1a82e0ece9a3\") " pod="opendatahub/maas-controller-86c7b54d87-ks9zb"
Jul 18 02:11:31.214873 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:31.214852    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxhrs\" (UniqueName: \"kubernetes.io/projected/de375459-cd35-459b-bffd-1a82e0ece9a3-kube-api-access-zxhrs\") pod \"maas-controller-86c7b54d87-ks9zb\" (UID: \"de375459-cd35-459b-bffd-1a82e0ece9a3\") " pod="opendatahub/maas-controller-86c7b54d87-ks9zb"
Jul 18 02:11:31.373018 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:31.372924    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86c7b54d87-ks9zb"
Jul 18 02:11:31.512338 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:31.512308    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-86c7b54d87-ks9zb"]
Jul 18 02:11:31.514572 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:11:31.514541    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde375459_cd35_459b_bffd_1a82e0ece9a3.slice/crio-c7d9d5ded6a5150080ce673865f2e173d41450f227dba40d1bd30d00c8b1bf28 WatchSource:0}: Error finding container c7d9d5ded6a5150080ce673865f2e173d41450f227dba40d1bd30d00c8b1bf28: Status 404 returned error can't find the container with id c7d9d5ded6a5150080ce673865f2e173d41450f227dba40d1bd30d00c8b1bf28
Jul 18 02:11:31.970265 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:31.970227    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bf64cab-65d0-4d6b-a86e-28444b00cc9e" path="/var/lib/kubelet/pods/8bf64cab-65d0-4d6b-a86e-28444b00cc9e/volumes"
Jul 18 02:11:32.239596 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:32.239561    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86c7b54d87-ks9zb" event={"ID":"de375459-cd35-459b-bffd-1a82e0ece9a3","Type":"ContainerStarted","Data":"f7106d38f57e210ea48d47f2edb718160ee8cee14a15f5bb0260840db44efb56"}
Jul 18 02:11:32.240022 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:32.239613    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86c7b54d87-ks9zb" event={"ID":"de375459-cd35-459b-bffd-1a82e0ece9a3","Type":"ContainerStarted","Data":"c7d9d5ded6a5150080ce673865f2e173d41450f227dba40d1bd30d00c8b1bf28"}
Jul 18 02:11:32.240022 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:32.239639    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-86c7b54d87-ks9zb"
Jul 18 02:11:32.257335 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:32.257282    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-86c7b54d87-ks9zb" podStartSLOduration=0.92582187 podStartE2EDuration="1.257266181s" totalImagesPullingTime="331.444311ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:11:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:11:32.254799665 +0000 UTC m=+886.879836332" watchObservedRunningTime="2026-07-18 02:11:32.257266181 +0000 UTC m=+886.882302848"
Jul 18 02:11:43.246839 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:43.246762    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-86c7b54d87-ks9zb"
Jul 18 02:11:45.857033 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:45.857002    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-bfrdg_9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777/console-operator/1.log"
Jul 18 02:11:45.863582 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:11:45.863559    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-bfrdg_9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777/console-operator/1.log"
Jul 18 02:15:00.137397 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:15:00.137359    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29739015-b8zw8"]
Jul 18 02:15:00.140873 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:15:00.140856    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739015-b8zw8"
Jul 18 02:15:00.143491 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:15:00.143471    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g69gp\""
Jul 18 02:15:00.150207 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:15:00.150190    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739015-b8zw8"
Jul 18 02:15:00.156978 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:15:00.156955    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29739015-b8zw8"]
Jul 18 02:15:00.292404 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:15:00.292371    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29739015-b8zw8"]
Jul 18 02:15:00.295138 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:15:00.295111    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6b0d47f_f2db_4d94_aee7_952dfcbd5f42.slice/crio-dba58b370f29aa5b1414a6ae80a143ba95cba2345a289d9f09afe2d768ae25e2 WatchSource:0}: Error finding container dba58b370f29aa5b1414a6ae80a143ba95cba2345a289d9f09afe2d768ae25e2: Status 404 returned error can't find the container with id dba58b370f29aa5b1414a6ae80a143ba95cba2345a289d9f09afe2d768ae25e2
Jul 18 02:15:01.277679 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:15:01.277632    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739015-b8zw8" event={"ID":"c6b0d47f-f2db-4d94-aee7-952dfcbd5f42","Type":"ContainerStarted","Data":"dba58b370f29aa5b1414a6ae80a143ba95cba2345a289d9f09afe2d768ae25e2"}
Jul 18 02:15:03.292577 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:15:03.292539    2576 generic.go:356] "Generic (PLEG): container finished" podID="c6b0d47f-f2db-4d94-aee7-952dfcbd5f42" containerID="650d8515edf71713a632a347a890f6ee9c6685b54232ca95239f2b4905a6e66b" exitCode=0
Jul 18 02:15:03.292962 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:15:03.292619    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739015-b8zw8" event={"ID":"c6b0d47f-f2db-4d94-aee7-952dfcbd5f42","Type":"ContainerDied","Data":"650d8515edf71713a632a347a890f6ee9c6685b54232ca95239f2b4905a6e66b"}
Jul 18 02:15:04.440990 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:15:04.440963    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739015-b8zw8"
Jul 18 02:15:05.304080 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:15:05.304050    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739015-b8zw8"
Jul 18 02:15:05.304257 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:15:05.304047    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739015-b8zw8" event={"ID":"c6b0d47f-f2db-4d94-aee7-952dfcbd5f42","Type":"ContainerDied","Data":"dba58b370f29aa5b1414a6ae80a143ba95cba2345a289d9f09afe2d768ae25e2"}
Jul 18 02:15:05.304257 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:15:05.304177    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dba58b370f29aa5b1414a6ae80a143ba95cba2345a289d9f09afe2d768ae25e2"
Jul 18 02:16:19.748049 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:16:19.747957    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7tj9r"]
Jul 18 02:16:19.748659 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:16:19.748482    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7tj9r" podUID="5214dfeb-feb7-41d4-9041-e8ee71ff099b" containerName="manager" containerID="cri-o://28d9682945bcb4b65365e87df8c8895cb27fa0816567b38f7aae7930b941b1b5" gracePeriod=10
Jul 18 02:16:22.609832 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:16:22.609804    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7tj9r"
Jul 18 02:16:22.610650 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:16:22.610632    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rhr8k\" (UniqueName: \"kubernetes.io/projected/5214dfeb-feb7-41d4-9041-e8ee71ff099b-kube-api-access-rhr8k\") pod \"5214dfeb-feb7-41d4-9041-e8ee71ff099b\" (UID: \"5214dfeb-feb7-41d4-9041-e8ee71ff099b\") "
Jul 18 02:16:22.610719 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:16:22.610675    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5214dfeb-feb7-41d4-9041-e8ee71ff099b-extensions-socket-volume\") pod \"5214dfeb-feb7-41d4-9041-e8ee71ff099b\" (UID: \"5214dfeb-feb7-41d4-9041-e8ee71ff099b\") "
Jul 18 02:16:22.611089 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:16:22.611059    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5214dfeb-feb7-41d4-9041-e8ee71ff099b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5214dfeb-feb7-41d4-9041-e8ee71ff099b" (UID: "5214dfeb-feb7-41d4-9041-e8ee71ff099b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:16:22.612686 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:16:22.612659    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5214dfeb-feb7-41d4-9041-e8ee71ff099b-kube-api-access-rhr8k" (OuterVolumeSpecName: "kube-api-access-rhr8k") pod "5214dfeb-feb7-41d4-9041-e8ee71ff099b" (UID: "5214dfeb-feb7-41d4-9041-e8ee71ff099b"). InnerVolumeSpecName "kube-api-access-rhr8k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:16:22.691065 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:16:22.691030    2576 generic.go:356] "Generic (PLEG): container finished" podID="5214dfeb-feb7-41d4-9041-e8ee71ff099b" containerID="28d9682945bcb4b65365e87df8c8895cb27fa0816567b38f7aae7930b941b1b5" exitCode=0
Jul 18 02:16:22.691232 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:16:22.691076    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7tj9r"
Jul 18 02:16:22.691232 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:16:22.691109    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7tj9r" event={"ID":"5214dfeb-feb7-41d4-9041-e8ee71ff099b","Type":"ContainerDied","Data":"28d9682945bcb4b65365e87df8c8895cb27fa0816567b38f7aae7930b941b1b5"}
Jul 18 02:16:22.691232 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:16:22.691167    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7tj9r" event={"ID":"5214dfeb-feb7-41d4-9041-e8ee71ff099b","Type":"ContainerDied","Data":"ae17aa4cfa5a13b3292e0ff2902ae455a48d2b695b0e4b6af38a8ac26ea58ff4"}
Jul 18 02:16:22.691232 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:16:22.691191    2576 scope.go:117] "RemoveContainer" containerID="28d9682945bcb4b65365e87df8c8895cb27fa0816567b38f7aae7930b941b1b5"
Jul 18 02:16:22.701525 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:16:22.701409    2576 scope.go:117] "RemoveContainer" containerID="28d9682945bcb4b65365e87df8c8895cb27fa0816567b38f7aae7930b941b1b5"
Jul 18 02:16:22.701810 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:16:22.701790    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28d9682945bcb4b65365e87df8c8895cb27fa0816567b38f7aae7930b941b1b5\": container with ID starting with 28d9682945bcb4b65365e87df8c8895cb27fa0816567b38f7aae7930b941b1b5 not found: ID does not exist" containerID="28d9682945bcb4b65365e87df8c8895cb27fa0816567b38f7aae7930b941b1b5"
Jul 18 02:16:22.701854 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:16:22.701822    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28d9682945bcb4b65365e87df8c8895cb27fa0816567b38f7aae7930b941b1b5"} err="failed to get container status \"28d9682945bcb4b65365e87df8c8895cb27fa0816567b38f7aae7930b941b1b5\": rpc error: code = NotFound desc = could not find container \"28d9682945bcb4b65365e87df8c8895cb27fa0816567b38f7aae7930b941b1b5\": container with ID starting with 28d9682945bcb4b65365e87df8c8895cb27fa0816567b38f7aae7930b941b1b5 not found: ID does not exist"
Jul 18 02:16:22.711526 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:16:22.711498    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rhr8k\" (UniqueName: \"kubernetes.io/projected/5214dfeb-feb7-41d4-9041-e8ee71ff099b-kube-api-access-rhr8k\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:16:22.711651 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:16:22.711529    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5214dfeb-feb7-41d4-9041-e8ee71ff099b-extensions-socket-volume\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:16:22.712299 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:16:22.712280    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7tj9r"]
Jul 18 02:16:22.715624 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:16:22.715598    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7tj9r"]
Jul 18 02:16:23.970960 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:16:23.970926    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5214dfeb-feb7-41d4-9041-e8ee71ff099b" path="/var/lib/kubelet/pods/5214dfeb-feb7-41d4-9041-e8ee71ff099b/volumes"
Jul 18 02:16:45.888474 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:16:45.888414    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-bfrdg_9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777/console-operator/1.log"
Jul 18 02:16:45.890760 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:16:45.890738    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-bfrdg_9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777/console-operator/1.log"
Jul 18 02:17:20.917103 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:17:20.917067    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7cw4l"]
Jul 18 02:17:20.917597 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:17:20.917490    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6b0d47f-f2db-4d94-aee7-952dfcbd5f42" containerName="cleanup"
Jul 18 02:17:20.917597 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:17:20.917502    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6b0d47f-f2db-4d94-aee7-952dfcbd5f42" containerName="cleanup"
Jul 18 02:17:20.917597 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:17:20.917513    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5214dfeb-feb7-41d4-9041-e8ee71ff099b" containerName="manager"
Jul 18 02:17:20.917597 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:17:20.917518    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5214dfeb-feb7-41d4-9041-e8ee71ff099b" containerName="manager"
Jul 18 02:17:20.917597 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:17:20.917598    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6b0d47f-f2db-4d94-aee7-952dfcbd5f42" containerName="cleanup"
Jul 18 02:17:20.917765 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:17:20.917606    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5214dfeb-feb7-41d4-9041-e8ee71ff099b" containerName="manager"
Jul 18 02:17:20.920664 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:17:20.920644    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7cw4l"
Jul 18 02:17:20.922981 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:17:20.922959    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-8xdm8\""
Jul 18 02:17:20.931222 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:17:20.931198    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7cw4l"]
Jul 18 02:17:21.027266 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:17:21.027178    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xb7j\" (UniqueName: \"kubernetes.io/projected/9710b342-09a4-4302-b8f2-3fb1f380c7fd-kube-api-access-5xb7j\") pod \"kuadrant-operator-controller-manager-86878c769f-7cw4l\" (UID: \"9710b342-09a4-4302-b8f2-3fb1f380c7fd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7cw4l"
Jul 18 02:17:21.027495 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:17:21.027300    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9710b342-09a4-4302-b8f2-3fb1f380c7fd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7cw4l\" (UID: \"9710b342-09a4-4302-b8f2-3fb1f380c7fd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7cw4l"
Jul 18 02:17:21.128041 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:17:21.128006    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xb7j\" (UniqueName: \"kubernetes.io/projected/9710b342-09a4-4302-b8f2-3fb1f380c7fd-kube-api-access-5xb7j\") pod \"kuadrant-operator-controller-manager-86878c769f-7cw4l\" (UID: \"9710b342-09a4-4302-b8f2-3fb1f380c7fd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7cw4l"
Jul 18 02:17:21.128195 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:17:21.128053    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9710b342-09a4-4302-b8f2-3fb1f380c7fd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7cw4l\" (UID: \"9710b342-09a4-4302-b8f2-3fb1f380c7fd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7cw4l"
Jul 18 02:17:21.128378 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:17:21.128357    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9710b342-09a4-4302-b8f2-3fb1f380c7fd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7cw4l\" (UID: \"9710b342-09a4-4302-b8f2-3fb1f380c7fd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7cw4l"
Jul 18 02:17:21.141898 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:17:21.141863    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xb7j\" (UniqueName: \"kubernetes.io/projected/9710b342-09a4-4302-b8f2-3fb1f380c7fd-kube-api-access-5xb7j\") pod \"kuadrant-operator-controller-manager-86878c769f-7cw4l\" (UID: \"9710b342-09a4-4302-b8f2-3fb1f380c7fd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7cw4l"
Jul 18 02:17:21.232126 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:17:21.232034    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7cw4l"
Jul 18 02:17:21.374327 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:17:21.374300    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7cw4l"]
Jul 18 02:17:21.376811 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:17:21.376785    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9710b342_09a4_4302_b8f2_3fb1f380c7fd.slice/crio-e86e530ea19b434d8b085806ffd65ae0c5d1ff5f2b65095494d21b71aed8dce7 WatchSource:0}: Error finding container e86e530ea19b434d8b085806ffd65ae0c5d1ff5f2b65095494d21b71aed8dce7: Status 404 returned error can't find the container with id e86e530ea19b434d8b085806ffd65ae0c5d1ff5f2b65095494d21b71aed8dce7
Jul 18 02:17:21.986200 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:17:21.986162    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7cw4l" event={"ID":"9710b342-09a4-4302-b8f2-3fb1f380c7fd","Type":"ContainerStarted","Data":"5508a1604d5293989940244251293b870db418090d3cc17090696822d02a07af"}
Jul 18 02:17:21.986614 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:17:21.986225    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7cw4l" event={"ID":"9710b342-09a4-4302-b8f2-3fb1f380c7fd","Type":"ContainerStarted","Data":"e86e530ea19b434d8b085806ffd65ae0c5d1ff5f2b65095494d21b71aed8dce7"}
Jul 18 02:17:21.986614 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:17:21.986248    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7cw4l"
Jul 18 02:17:22.005695 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:17:22.005640    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7cw4l" podStartSLOduration=2.005623558 podStartE2EDuration="2.005623558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:17:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:17:22.003553099 +0000 UTC m=+1236.628589767" watchObservedRunningTime="2026-07-18 02:17:22.005623558 +0000 UTC m=+1236.630660227"
Jul 18 02:17:32.994916 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:17:32.994883    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7cw4l"
Jul 18 02:21:45.921703 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:21:45.921673    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-bfrdg_9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777/console-operator/1.log"
Jul 18 02:21:45.922265 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:21:45.922051    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-bfrdg_9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777/console-operator/1.log"
Jul 18 02:23:38.935217 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:23:38.935129    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-914a7c26-8886cf99f-pn8z6"]
Jul 18 02:23:38.938880 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:23:38.938858    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-914a7c26-8886cf99f-pn8z6"
Jul 18 02:23:38.941258 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:23:38.941235    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-914a7c26-serving-cert\""
Jul 18 02:23:38.950523 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:23:38.950498    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-914a7c26-8886cf99f-pn8z6"]
Jul 18 02:23:39.026447 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:23:39.026395    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/780f6759-01a0-468f-adc9-686d5d64aa3d-maas-api-tls\") pod \"maas-api-e2e-ait-914a7c26-8886cf99f-pn8z6\" (UID: \"780f6759-01a0-468f-adc9-686d5d64aa3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-914a7c26-8886cf99f-pn8z6"
Jul 18 02:23:39.026698 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:23:39.026553    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/780f6759-01a0-468f-adc9-686d5d64aa3d-openshift-service-ca\") pod \"maas-api-e2e-ait-914a7c26-8886cf99f-pn8z6\" (UID: \"780f6759-01a0-468f-adc9-686d5d64aa3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-914a7c26-8886cf99f-pn8z6"
Jul 18 02:23:39.026698 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:23:39.026641    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ct2mr\" (UniqueName: \"kubernetes.io/projected/780f6759-01a0-468f-adc9-686d5d64aa3d-kube-api-access-ct2mr\") pod \"maas-api-e2e-ait-914a7c26-8886cf99f-pn8z6\" (UID: \"780f6759-01a0-468f-adc9-686d5d64aa3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-914a7c26-8886cf99f-pn8z6"
Jul 18 02:23:39.127994 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:23:39.127949    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ct2mr\" (UniqueName: \"kubernetes.io/projected/780f6759-01a0-468f-adc9-686d5d64aa3d-kube-api-access-ct2mr\") pod \"maas-api-e2e-ait-914a7c26-8886cf99f-pn8z6\" (UID: \"780f6759-01a0-468f-adc9-686d5d64aa3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-914a7c26-8886cf99f-pn8z6"
Jul 18 02:23:39.128200 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:23:39.128066    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/780f6759-01a0-468f-adc9-686d5d64aa3d-maas-api-tls\") pod \"maas-api-e2e-ait-914a7c26-8886cf99f-pn8z6\" (UID: \"780f6759-01a0-468f-adc9-686d5d64aa3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-914a7c26-8886cf99f-pn8z6"
Jul 18 02:23:39.128200 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:23:39.128115    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/780f6759-01a0-468f-adc9-686d5d64aa3d-openshift-service-ca\") pod \"maas-api-e2e-ait-914a7c26-8886cf99f-pn8z6\" (UID: \"780f6759-01a0-468f-adc9-686d5d64aa3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-914a7c26-8886cf99f-pn8z6"
Jul 18 02:23:39.128815 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:23:39.128789    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/780f6759-01a0-468f-adc9-686d5d64aa3d-openshift-service-ca\") pod \"maas-api-e2e-ait-914a7c26-8886cf99f-pn8z6\" (UID: \"780f6759-01a0-468f-adc9-686d5d64aa3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-914a7c26-8886cf99f-pn8z6"
Jul 18 02:23:39.130598 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:23:39.130579    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/780f6759-01a0-468f-adc9-686d5d64aa3d-maas-api-tls\") pod \"maas-api-e2e-ait-914a7c26-8886cf99f-pn8z6\" (UID: \"780f6759-01a0-468f-adc9-686d5d64aa3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-914a7c26-8886cf99f-pn8z6"
Jul 18 02:23:39.137007 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:23:39.136982    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ct2mr\" (UniqueName: \"kubernetes.io/projected/780f6759-01a0-468f-adc9-686d5d64aa3d-kube-api-access-ct2mr\") pod \"maas-api-e2e-ait-914a7c26-8886cf99f-pn8z6\" (UID: \"780f6759-01a0-468f-adc9-686d5d64aa3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-914a7c26-8886cf99f-pn8z6"
Jul 18 02:23:39.248907 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:23:39.248811    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-914a7c26-8886cf99f-pn8z6"
Jul 18 02:23:39.397213 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:23:39.397186    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-914a7c26-8886cf99f-pn8z6"]
Jul 18 02:23:39.398789 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:23:39.398758    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod780f6759_01a0_468f_adc9_686d5d64aa3d.slice/crio-9392520040bf54661276d0b7d536575f86d286123d92314371f1e3387a4ebd23 WatchSource:0}: Error finding container 9392520040bf54661276d0b7d536575f86d286123d92314371f1e3387a4ebd23: Status 404 returned error can't find the container with id 9392520040bf54661276d0b7d536575f86d286123d92314371f1e3387a4ebd23
Jul 18 02:23:39.400212 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:23:39.400195    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 02:23:39.901777 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:23:39.901742    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-914a7c26-8886cf99f-pn8z6" event={"ID":"780f6759-01a0-468f-adc9-686d5d64aa3d","Type":"ContainerStarted","Data":"9392520040bf54661276d0b7d536575f86d286123d92314371f1e3387a4ebd23"}
Jul 18 02:23:40.908486 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:23:40.908451    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-914a7c26-8886cf99f-pn8z6" event={"ID":"780f6759-01a0-468f-adc9-686d5d64aa3d","Type":"ContainerStarted","Data":"380a13b9d9225ec4551cd278ffca3cdec712cf77f183f4038c06f12cb2b6ef52"}
Jul 18 02:23:40.908984 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:23:40.908572    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-914a7c26-8886cf99f-pn8z6"
Jul 18 02:23:40.930498 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:23:40.930447    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-914a7c26-8886cf99f-pn8z6" podStartSLOduration=2.353659995 podStartE2EDuration="2.93041542s" totalImagesPullingTime="576.755425ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:23:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:23:40.928152679 +0000 UTC m=+1615.553189346" watchObservedRunningTime="2026-07-18 02:23:40.93041542 +0000 UTC m=+1615.555452086"
Jul 18 02:23:46.918600 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:23:46.918571    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-914a7c26-8886cf99f-pn8z6"
Jul 18 02:24:04.256700 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:04.256661    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-914a7c26-mh6w8"]
Jul 18 02:24:04.260261 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:04.260244    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-914a7c26-mh6w8"
Jul 18 02:24:04.262770 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:04.262750    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g69gp\""
Jul 18 02:24:04.269320 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:04.269294    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-914a7c26-mh6w8"]
Jul 18 02:24:04.356514 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:04.356478    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53f0ac56-4839-42f4-98b1-a5d342e5b2d7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-914a7c26-mh6w8\" (UID: \"53f0ac56-4839-42f4-98b1-a5d342e5b2d7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-914a7c26-mh6w8"
Jul 18 02:24:04.457969 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:04.457936    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53f0ac56-4839-42f4-98b1-a5d342e5b2d7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-914a7c26-mh6w8\" (UID: \"53f0ac56-4839-42f4-98b1-a5d342e5b2d7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-914a7c26-mh6w8"
Jul 18 02:24:04.458629 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:04.458611    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53f0ac56-4839-42f4-98b1-a5d342e5b2d7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-914a7c26-mh6w8\" (UID: \"53f0ac56-4839-42f4-98b1-a5d342e5b2d7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-914a7c26-mh6w8"
Jul 18 02:24:04.571996 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:04.571905    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-914a7c26-mh6w8"
Jul 18 02:24:04.709727 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:04.709698    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-914a7c26-mh6w8"]
Jul 18 02:24:04.712763 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:24:04.712732    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53f0ac56_4839_42f4_98b1_a5d342e5b2d7.slice/crio-6899d0f02231074d54b26268a4fd7b98073e5733b2163fa1805680c1200c4386 WatchSource:0}: Error finding container 6899d0f02231074d54b26268a4fd7b98073e5733b2163fa1805680c1200c4386: Status 404 returned error can't find the container with id 6899d0f02231074d54b26268a4fd7b98073e5733b2163fa1805680c1200c4386
Jul 18 02:24:05.029780 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:05.029740    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-914a7c26-mh6w8" event={"ID":"53f0ac56-4839-42f4-98b1-a5d342e5b2d7","Type":"ContainerStarted","Data":"24db48338db9354f14f06fe5b9816dfa947c9dfb6fcb0ce8e8aa3a85e82de30c"}
Jul 18 02:24:05.029954 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:05.029800    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-914a7c26-mh6w8" event={"ID":"53f0ac56-4839-42f4-98b1-a5d342e5b2d7","Type":"ContainerStarted","Data":"6899d0f02231074d54b26268a4fd7b98073e5733b2163fa1805680c1200c4386"}
Jul 18 02:24:05.054683 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:05.054628    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-914a7c26-mh6w8" podStartSLOduration=1.054612521 podStartE2EDuration="1.054612521s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:24:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:24:05.05186818 +0000 UTC m=+1639.676904848" watchObservedRunningTime="2026-07-18 02:24:05.054612521 +0000 UTC m=+1639.679649217"
Jul 18 02:24:06.036773 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:06.036738    2576 generic.go:356] "Generic (PLEG): container finished" podID="53f0ac56-4839-42f4-98b1-a5d342e5b2d7" containerID="24db48338db9354f14f06fe5b9816dfa947c9dfb6fcb0ce8e8aa3a85e82de30c" exitCode=0
Jul 18 02:24:06.037243 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:06.036826    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-914a7c26-mh6w8" event={"ID":"53f0ac56-4839-42f4-98b1-a5d342e5b2d7","Type":"ContainerDied","Data":"24db48338db9354f14f06fe5b9816dfa947c9dfb6fcb0ce8e8aa3a85e82de30c"}
Jul 18 02:24:07.183730 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:07.183701    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-914a7c26-mh6w8"
Jul 18 02:24:07.285219 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:07.285183    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53f0ac56-4839-42f4-98b1-a5d342e5b2d7-maas-api-service-ca\") pod \"53f0ac56-4839-42f4-98b1-a5d342e5b2d7\" (UID: \"53f0ac56-4839-42f4-98b1-a5d342e5b2d7\") "
Jul 18 02:24:07.285606 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:07.285582    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53f0ac56-4839-42f4-98b1-a5d342e5b2d7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "53f0ac56-4839-42f4-98b1-a5d342e5b2d7" (UID: "53f0ac56-4839-42f4-98b1-a5d342e5b2d7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:24:07.386346 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:07.386314    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53f0ac56-4839-42f4-98b1-a5d342e5b2d7-maas-api-service-ca\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:24:08.047871 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:08.047833    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-914a7c26-mh6w8"
Jul 18 02:24:08.047871 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:08.047851    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-914a7c26-mh6w8" event={"ID":"53f0ac56-4839-42f4-98b1-a5d342e5b2d7","Type":"ContainerDied","Data":"6899d0f02231074d54b26268a4fd7b98073e5733b2163fa1805680c1200c4386"}
Jul 18 02:24:08.048115 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:08.047896    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6899d0f02231074d54b26268a4fd7b98073e5733b2163fa1805680c1200c4386"
Jul 18 02:24:14.711981 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:14.711936    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-914a7c26-8886cf99f-pn8z6"]
Jul 18 02:24:14.712560 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:14.712213    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-914a7c26-8886cf99f-pn8z6" podUID="780f6759-01a0-468f-adc9-686d5d64aa3d" containerName="maas-api" containerID="cri-o://380a13b9d9225ec4551cd278ffca3cdec712cf77f183f4038c06f12cb2b6ef52" gracePeriod=30
Jul 18 02:24:14.966964 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:14.966898    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-914a7c26-8886cf99f-pn8z6"
Jul 18 02:24:15.059801 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:15.059770    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ct2mr\" (UniqueName: \"kubernetes.io/projected/780f6759-01a0-468f-adc9-686d5d64aa3d-kube-api-access-ct2mr\") pod \"780f6759-01a0-468f-adc9-686d5d64aa3d\" (UID: \"780f6759-01a0-468f-adc9-686d5d64aa3d\") "
Jul 18 02:24:15.059997 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:15.059817    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/780f6759-01a0-468f-adc9-686d5d64aa3d-maas-api-tls\") pod \"780f6759-01a0-468f-adc9-686d5d64aa3d\" (UID: \"780f6759-01a0-468f-adc9-686d5d64aa3d\") "
Jul 18 02:24:15.059997 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:15.059868    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/780f6759-01a0-468f-adc9-686d5d64aa3d-openshift-service-ca\") pod \"780f6759-01a0-468f-adc9-686d5d64aa3d\" (UID: \"780f6759-01a0-468f-adc9-686d5d64aa3d\") "
Jul 18 02:24:15.060299 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:15.060273    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/780f6759-01a0-468f-adc9-686d5d64aa3d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "780f6759-01a0-468f-adc9-686d5d64aa3d" (UID: "780f6759-01a0-468f-adc9-686d5d64aa3d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:24:15.062213 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:15.062184    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/780f6759-01a0-468f-adc9-686d5d64aa3d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "780f6759-01a0-468f-adc9-686d5d64aa3d" (UID: "780f6759-01a0-468f-adc9-686d5d64aa3d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:24:15.062213 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:15.062192    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/780f6759-01a0-468f-adc9-686d5d64aa3d-kube-api-access-ct2mr" (OuterVolumeSpecName: "kube-api-access-ct2mr") pod "780f6759-01a0-468f-adc9-686d5d64aa3d" (UID: "780f6759-01a0-468f-adc9-686d5d64aa3d"). InnerVolumeSpecName "kube-api-access-ct2mr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:24:15.083680 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:15.083644    2576 generic.go:356] "Generic (PLEG): container finished" podID="780f6759-01a0-468f-adc9-686d5d64aa3d" containerID="380a13b9d9225ec4551cd278ffca3cdec712cf77f183f4038c06f12cb2b6ef52" exitCode=0
Jul 18 02:24:15.083831 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:15.083694    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-914a7c26-8886cf99f-pn8z6"
Jul 18 02:24:15.083831 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:15.083720    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-914a7c26-8886cf99f-pn8z6" event={"ID":"780f6759-01a0-468f-adc9-686d5d64aa3d","Type":"ContainerDied","Data":"380a13b9d9225ec4551cd278ffca3cdec712cf77f183f4038c06f12cb2b6ef52"}
Jul 18 02:24:15.083831 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:15.083781    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-914a7c26-8886cf99f-pn8z6" event={"ID":"780f6759-01a0-468f-adc9-686d5d64aa3d","Type":"ContainerDied","Data":"9392520040bf54661276d0b7d536575f86d286123d92314371f1e3387a4ebd23"}
Jul 18 02:24:15.083831 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:15.083806    2576 scope.go:117] "RemoveContainer" containerID="380a13b9d9225ec4551cd278ffca3cdec712cf77f183f4038c06f12cb2b6ef52"
Jul 18 02:24:15.096727 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:15.096702    2576 scope.go:117] "RemoveContainer" containerID="380a13b9d9225ec4551cd278ffca3cdec712cf77f183f4038c06f12cb2b6ef52"
Jul 18 02:24:15.097013 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:24:15.096994    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"380a13b9d9225ec4551cd278ffca3cdec712cf77f183f4038c06f12cb2b6ef52\": container with ID starting with 380a13b9d9225ec4551cd278ffca3cdec712cf77f183f4038c06f12cb2b6ef52 not found: ID does not exist" containerID="380a13b9d9225ec4551cd278ffca3cdec712cf77f183f4038c06f12cb2b6ef52"
Jul 18 02:24:15.097081 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:15.097027    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"380a13b9d9225ec4551cd278ffca3cdec712cf77f183f4038c06f12cb2b6ef52"} err="failed to get container status \"380a13b9d9225ec4551cd278ffca3cdec712cf77f183f4038c06f12cb2b6ef52\": rpc error: code = NotFound desc = could not find container \"380a13b9d9225ec4551cd278ffca3cdec712cf77f183f4038c06f12cb2b6ef52\": container with ID starting with 380a13b9d9225ec4551cd278ffca3cdec712cf77f183f4038c06f12cb2b6ef52 not found: ID does not exist"
Jul 18 02:24:15.106132 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:15.106105    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-914a7c26-8886cf99f-pn8z6"]
Jul 18 02:24:15.110088 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:15.110062    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-914a7c26-8886cf99f-pn8z6"]
Jul 18 02:24:15.161525 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:15.161487    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/780f6759-01a0-468f-adc9-686d5d64aa3d-openshift-service-ca\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:24:15.161525 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:15.161518    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ct2mr\" (UniqueName: \"kubernetes.io/projected/780f6759-01a0-468f-adc9-686d5d64aa3d-kube-api-access-ct2mr\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:24:15.161525 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:15.161531    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/780f6759-01a0-468f-adc9-686d5d64aa3d-maas-api-tls\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:24:15.969390 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:15.969348    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="780f6759-01a0-468f-adc9-686d5d64aa3d" path="/var/lib/kubelet/pods/780f6759-01a0-468f-adc9-686d5d64aa3d/volumes"
Jul 18 02:24:19.338611 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:19.338573    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-914a7c26-mh6w8"]
Jul 18 02:24:19.341050 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:19.341021    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-914a7c26-mh6w8"]
Jul 18 02:24:19.969217 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:19.969185    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53f0ac56-4839-42f4-98b1-a5d342e5b2d7" path="/var/lib/kubelet/pods/53f0ac56-4839-42f4-98b1-a5d342e5b2d7/volumes"
Jul 18 02:24:23.208953 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:23.208920    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-5e522e36-6687489bb6-8l45r"]
Jul 18 02:24:23.209308 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:23.209294    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53f0ac56-4839-42f4-98b1-a5d342e5b2d7" containerName="revoke-keys"
Jul 18 02:24:23.209308 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:23.209305    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="53f0ac56-4839-42f4-98b1-a5d342e5b2d7" containerName="revoke-keys"
Jul 18 02:24:23.209383 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:23.209327    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="780f6759-01a0-468f-adc9-686d5d64aa3d" containerName="maas-api"
Jul 18 02:24:23.209383 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:23.209333    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="780f6759-01a0-468f-adc9-686d5d64aa3d" containerName="maas-api"
Jul 18 02:24:23.209474 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:23.209390    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="780f6759-01a0-468f-adc9-686d5d64aa3d" containerName="maas-api"
Jul 18 02:24:23.209474 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:23.209399    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="53f0ac56-4839-42f4-98b1-a5d342e5b2d7" containerName="revoke-keys"
Jul 18 02:24:23.212483 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:23.212464    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5e522e36-6687489bb6-8l45r"
Jul 18 02:24:23.219196 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:23.219172    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-5e522e36-serving-cert\""
Jul 18 02:24:23.237657 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:23.237629    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-5e522e36-6687489bb6-8l45r"]
Jul 18 02:24:23.337583 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:23.337550    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zp22\" (UniqueName: \"kubernetes.io/projected/e2c16138-96b9-440d-a3ff-7a20792a0c5f-kube-api-access-2zp22\") pod \"maas-api-e2e-migrate-5e522e36-6687489bb6-8l45r\" (UID: \"e2c16138-96b9-440d-a3ff-7a20792a0c5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5e522e36-6687489bb6-8l45r"
Jul 18 02:24:23.337799 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:23.337618    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2c16138-96b9-440d-a3ff-7a20792a0c5f-maas-api-tls\") pod \"maas-api-e2e-migrate-5e522e36-6687489bb6-8l45r\" (UID: \"e2c16138-96b9-440d-a3ff-7a20792a0c5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5e522e36-6687489bb6-8l45r"
Jul 18 02:24:23.337799 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:23.337727    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2c16138-96b9-440d-a3ff-7a20792a0c5f-openshift-service-ca\") pod \"maas-api-e2e-migrate-5e522e36-6687489bb6-8l45r\" (UID: \"e2c16138-96b9-440d-a3ff-7a20792a0c5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5e522e36-6687489bb6-8l45r"
Jul 18 02:24:23.438983 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:23.438940    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zp22\" (UniqueName: \"kubernetes.io/projected/e2c16138-96b9-440d-a3ff-7a20792a0c5f-kube-api-access-2zp22\") pod \"maas-api-e2e-migrate-5e522e36-6687489bb6-8l45r\" (UID: \"e2c16138-96b9-440d-a3ff-7a20792a0c5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5e522e36-6687489bb6-8l45r"
Jul 18 02:24:23.439187 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:23.438998    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2c16138-96b9-440d-a3ff-7a20792a0c5f-maas-api-tls\") pod \"maas-api-e2e-migrate-5e522e36-6687489bb6-8l45r\" (UID: \"e2c16138-96b9-440d-a3ff-7a20792a0c5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5e522e36-6687489bb6-8l45r"
Jul 18 02:24:23.439187 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:23.439082    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2c16138-96b9-440d-a3ff-7a20792a0c5f-openshift-service-ca\") pod \"maas-api-e2e-migrate-5e522e36-6687489bb6-8l45r\" (UID: \"e2c16138-96b9-440d-a3ff-7a20792a0c5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5e522e36-6687489bb6-8l45r"
Jul 18 02:24:23.439187 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:24:23.439109    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-5e522e36-serving-cert: secret "maas-api-e2e-migrate-5e522e36-serving-cert" not found
Jul 18 02:24:23.439187 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:24:23.439179    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2c16138-96b9-440d-a3ff-7a20792a0c5f-maas-api-tls podName:e2c16138-96b9-440d-a3ff-7a20792a0c5f nodeName:}" failed. No retries permitted until 2026-07-18 02:24:23.939161801 +0000 UTC m=+1658.564198446 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e2c16138-96b9-440d-a3ff-7a20792a0c5f-maas-api-tls") pod "maas-api-e2e-migrate-5e522e36-6687489bb6-8l45r" (UID: "e2c16138-96b9-440d-a3ff-7a20792a0c5f") : secret "maas-api-e2e-migrate-5e522e36-serving-cert" not found
Jul 18 02:24:23.439839 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:23.439820    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2c16138-96b9-440d-a3ff-7a20792a0c5f-openshift-service-ca\") pod \"maas-api-e2e-migrate-5e522e36-6687489bb6-8l45r\" (UID: \"e2c16138-96b9-440d-a3ff-7a20792a0c5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5e522e36-6687489bb6-8l45r"
Jul 18 02:24:23.447736 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:23.447713    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zp22\" (UniqueName: \"kubernetes.io/projected/e2c16138-96b9-440d-a3ff-7a20792a0c5f-kube-api-access-2zp22\") pod \"maas-api-e2e-migrate-5e522e36-6687489bb6-8l45r\" (UID: \"e2c16138-96b9-440d-a3ff-7a20792a0c5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5e522e36-6687489bb6-8l45r"
Jul 18 02:24:23.943745 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:23.943712    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2c16138-96b9-440d-a3ff-7a20792a0c5f-maas-api-tls\") pod \"maas-api-e2e-migrate-5e522e36-6687489bb6-8l45r\" (UID: \"e2c16138-96b9-440d-a3ff-7a20792a0c5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5e522e36-6687489bb6-8l45r"
Jul 18 02:24:23.946166 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:23.946138    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2c16138-96b9-440d-a3ff-7a20792a0c5f-maas-api-tls\") pod \"maas-api-e2e-migrate-5e522e36-6687489bb6-8l45r\" (UID: \"e2c16138-96b9-440d-a3ff-7a20792a0c5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5e522e36-6687489bb6-8l45r"
Jul 18 02:24:24.122140 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:24.122096    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5e522e36-6687489bb6-8l45r"
Jul 18 02:24:24.264791 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:24.264764    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-5e522e36-6687489bb6-8l45r"]
Jul 18 02:24:24.266972 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:24:24.266942    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2c16138_96b9_440d_a3ff_7a20792a0c5f.slice/crio-7c04e3ea7b82b3cee1dc963b4aa9be823fb7d1aab6b84249bd523a1c770f9a6e WatchSource:0}: Error finding container 7c04e3ea7b82b3cee1dc963b4aa9be823fb7d1aab6b84249bd523a1c770f9a6e: Status 404 returned error can't find the container with id 7c04e3ea7b82b3cee1dc963b4aa9be823fb7d1aab6b84249bd523a1c770f9a6e
Jul 18 02:24:25.136132 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:25.136096    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5e522e36-6687489bb6-8l45r" event={"ID":"e2c16138-96b9-440d-a3ff-7a20792a0c5f","Type":"ContainerStarted","Data":"35df1ff79d0b4856c5ecd532cdb8d63bd4849bcdfd4ea3b67274cf94bc21a9d0"}
Jul 18 02:24:25.136322 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:25.136150    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5e522e36-6687489bb6-8l45r" event={"ID":"e2c16138-96b9-440d-a3ff-7a20792a0c5f","Type":"ContainerStarted","Data":"7c04e3ea7b82b3cee1dc963b4aa9be823fb7d1aab6b84249bd523a1c770f9a6e"}
Jul 18 02:24:25.136322 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:25.136182    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5e522e36-6687489bb6-8l45r"
Jul 18 02:24:25.155557 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:25.155488    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5e522e36-6687489bb6-8l45r" podStartSLOduration=1.7170755 podStartE2EDuration="2.155471216s" totalImagesPullingTime="438.395716ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:24:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:24:25.154132762 +0000 UTC m=+1659.779169436" watchObservedRunningTime="2026-07-18 02:24:25.155471216 +0000 UTC m=+1659.780507883"
Jul 18 02:24:31.147351 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:31.147316    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5e522e36-6687489bb6-8l45r"
Jul 18 02:24:48.112302 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:48.112258    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5e522e36-c8t7l"]
Jul 18 02:24:48.117300 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:48.117276    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5e522e36-c8t7l"
Jul 18 02:24:48.127646 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:48.127611    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g69gp\""
Jul 18 02:24:48.130254 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:48.130226    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5e522e36-c8t7l"]
Jul 18 02:24:48.148319 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:48.148285    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8035581-4b97-4ab2-8cb8-0cd0d8b095eb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5e522e36-c8t7l\" (UID: \"d8035581-4b97-4ab2-8cb8-0cd0d8b095eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5e522e36-c8t7l"
Jul 18 02:24:48.249397 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:48.249357    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8035581-4b97-4ab2-8cb8-0cd0d8b095eb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5e522e36-c8t7l\" (UID: \"d8035581-4b97-4ab2-8cb8-0cd0d8b095eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5e522e36-c8t7l"
Jul 18 02:24:48.250059 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:48.250040    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8035581-4b97-4ab2-8cb8-0cd0d8b095eb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5e522e36-c8t7l\" (UID: \"d8035581-4b97-4ab2-8cb8-0cd0d8b095eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5e522e36-c8t7l"
Jul 18 02:24:48.438029 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:48.437990    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5e522e36-c8t7l"
Jul 18 02:24:48.576157 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:48.576119    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5e522e36-c8t7l"]
Jul 18 02:24:48.577923 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:24:48.577896    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8035581_4b97_4ab2_8cb8_0cd0d8b095eb.slice/crio-c32ce0e5bb05601f0302cd796fbb383c76d2051552c0c514487083b6d95468d7 WatchSource:0}: Error finding container c32ce0e5bb05601f0302cd796fbb383c76d2051552c0c514487083b6d95468d7: Status 404 returned error can't find the container with id c32ce0e5bb05601f0302cd796fbb383c76d2051552c0c514487083b6d95468d7
Jul 18 02:24:49.262338 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:49.262300    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5e522e36-c8t7l" event={"ID":"d8035581-4b97-4ab2-8cb8-0cd0d8b095eb","Type":"ContainerStarted","Data":"5db7bf0174a71b81da4d3561c20246e92d17051f6d23f90f9234f9ee2be723ce"}
Jul 18 02:24:49.262753 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:49.262355    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5e522e36-c8t7l" event={"ID":"d8035581-4b97-4ab2-8cb8-0cd0d8b095eb","Type":"ContainerStarted","Data":"c32ce0e5bb05601f0302cd796fbb383c76d2051552c0c514487083b6d95468d7"}
Jul 18 02:24:49.279868 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:49.279825    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5e522e36-c8t7l" podStartSLOduration=1.279809717 podStartE2EDuration="1.279809717s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:24:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:24:49.277927023 +0000 UTC m=+1683.902963691" watchObservedRunningTime="2026-07-18 02:24:49.279809717 +0000 UTC m=+1683.904846383"
Jul 18 02:24:50.269022 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:50.268988    2576 generic.go:356] "Generic (PLEG): container finished" podID="d8035581-4b97-4ab2-8cb8-0cd0d8b095eb" containerID="5db7bf0174a71b81da4d3561c20246e92d17051f6d23f90f9234f9ee2be723ce" exitCode=0
Jul 18 02:24:50.269408 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:50.269048    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5e522e36-c8t7l" event={"ID":"d8035581-4b97-4ab2-8cb8-0cd0d8b095eb","Type":"ContainerDied","Data":"5db7bf0174a71b81da4d3561c20246e92d17051f6d23f90f9234f9ee2be723ce"}
Jul 18 02:24:51.413525 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:51.413495    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5e522e36-c8t7l"
Jul 18 02:24:51.479651 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:51.479608    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8035581-4b97-4ab2-8cb8-0cd0d8b095eb-maas-api-service-ca\") pod \"d8035581-4b97-4ab2-8cb8-0cd0d8b095eb\" (UID: \"d8035581-4b97-4ab2-8cb8-0cd0d8b095eb\") "
Jul 18 02:24:51.479964 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:51.479942    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8035581-4b97-4ab2-8cb8-0cd0d8b095eb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d8035581-4b97-4ab2-8cb8-0cd0d8b095eb" (UID: "d8035581-4b97-4ab2-8cb8-0cd0d8b095eb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:24:51.580930 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:51.580836    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8035581-4b97-4ab2-8cb8-0cd0d8b095eb-maas-api-service-ca\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:24:52.280918 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:52.280829    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5e522e36-c8t7l" event={"ID":"d8035581-4b97-4ab2-8cb8-0cd0d8b095eb","Type":"ContainerDied","Data":"c32ce0e5bb05601f0302cd796fbb383c76d2051552c0c514487083b6d95468d7"}
Jul 18 02:24:52.280918 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:52.280855    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5e522e36-c8t7l"
Jul 18 02:24:52.280918 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:52.280880    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c32ce0e5bb05601f0302cd796fbb383c76d2051552c0c514487083b6d95468d7"
Jul 18 02:24:58.737699 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:58.737664    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-5e522e36-6687489bb6-8l45r"]
Jul 18 02:24:58.738146 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:58.737908    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5e522e36-6687489bb6-8l45r" podUID="e2c16138-96b9-440d-a3ff-7a20792a0c5f" containerName="maas-api" containerID="cri-o://35df1ff79d0b4856c5ecd532cdb8d63bd4849bcdfd4ea3b67274cf94bc21a9d0" gracePeriod=30
Jul 18 02:24:58.998986 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:58.998924    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5e522e36-6687489bb6-8l45r"
Jul 18 02:24:59.047062 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:59.047028    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2c16138-96b9-440d-a3ff-7a20792a0c5f-openshift-service-ca\") pod \"e2c16138-96b9-440d-a3ff-7a20792a0c5f\" (UID: \"e2c16138-96b9-440d-a3ff-7a20792a0c5f\") "
Jul 18 02:24:59.047238 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:59.047100    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2zp22\" (UniqueName: \"kubernetes.io/projected/e2c16138-96b9-440d-a3ff-7a20792a0c5f-kube-api-access-2zp22\") pod \"e2c16138-96b9-440d-a3ff-7a20792a0c5f\" (UID: \"e2c16138-96b9-440d-a3ff-7a20792a0c5f\") "
Jul 18 02:24:59.047238 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:59.047190    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2c16138-96b9-440d-a3ff-7a20792a0c5f-maas-api-tls\") pod \"e2c16138-96b9-440d-a3ff-7a20792a0c5f\" (UID: \"e2c16138-96b9-440d-a3ff-7a20792a0c5f\") "
Jul 18 02:24:59.047415 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:59.047393    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2c16138-96b9-440d-a3ff-7a20792a0c5f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e2c16138-96b9-440d-a3ff-7a20792a0c5f" (UID: "e2c16138-96b9-440d-a3ff-7a20792a0c5f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:24:59.047496 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:59.047481    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2c16138-96b9-440d-a3ff-7a20792a0c5f-openshift-service-ca\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:24:59.049218 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:59.049185    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2c16138-96b9-440d-a3ff-7a20792a0c5f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e2c16138-96b9-440d-a3ff-7a20792a0c5f" (UID: "e2c16138-96b9-440d-a3ff-7a20792a0c5f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:24:59.049218 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:59.049209    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2c16138-96b9-440d-a3ff-7a20792a0c5f-kube-api-access-2zp22" (OuterVolumeSpecName: "kube-api-access-2zp22") pod "e2c16138-96b9-440d-a3ff-7a20792a0c5f" (UID: "e2c16138-96b9-440d-a3ff-7a20792a0c5f"). InnerVolumeSpecName "kube-api-access-2zp22". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:24:59.148257 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:59.148210    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2zp22\" (UniqueName: \"kubernetes.io/projected/e2c16138-96b9-440d-a3ff-7a20792a0c5f-kube-api-access-2zp22\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:24:59.148257 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:59.148248    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2c16138-96b9-440d-a3ff-7a20792a0c5f-maas-api-tls\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:24:59.322596 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:59.322496    2576 generic.go:356] "Generic (PLEG): container finished" podID="e2c16138-96b9-440d-a3ff-7a20792a0c5f" containerID="35df1ff79d0b4856c5ecd532cdb8d63bd4849bcdfd4ea3b67274cf94bc21a9d0" exitCode=0
Jul 18 02:24:59.322596 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:59.322541    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5e522e36-6687489bb6-8l45r"
Jul 18 02:24:59.322596 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:59.322557    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5e522e36-6687489bb6-8l45r" event={"ID":"e2c16138-96b9-440d-a3ff-7a20792a0c5f","Type":"ContainerDied","Data":"35df1ff79d0b4856c5ecd532cdb8d63bd4849bcdfd4ea3b67274cf94bc21a9d0"}
Jul 18 02:24:59.322596 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:59.322601    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5e522e36-6687489bb6-8l45r" event={"ID":"e2c16138-96b9-440d-a3ff-7a20792a0c5f","Type":"ContainerDied","Data":"7c04e3ea7b82b3cee1dc963b4aa9be823fb7d1aab6b84249bd523a1c770f9a6e"}
Jul 18 02:24:59.322907 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:59.322621    2576 scope.go:117] "RemoveContainer" containerID="35df1ff79d0b4856c5ecd532cdb8d63bd4849bcdfd4ea3b67274cf94bc21a9d0"
Jul 18 02:24:59.331507 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:59.331484    2576 scope.go:117] "RemoveContainer" containerID="35df1ff79d0b4856c5ecd532cdb8d63bd4849bcdfd4ea3b67274cf94bc21a9d0"
Jul 18 02:24:59.331795 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:24:59.331775    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35df1ff79d0b4856c5ecd532cdb8d63bd4849bcdfd4ea3b67274cf94bc21a9d0\": container with ID starting with 35df1ff79d0b4856c5ecd532cdb8d63bd4849bcdfd4ea3b67274cf94bc21a9d0 not found: ID does not exist" containerID="35df1ff79d0b4856c5ecd532cdb8d63bd4849bcdfd4ea3b67274cf94bc21a9d0"
Jul 18 02:24:59.331849 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:59.331803    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35df1ff79d0b4856c5ecd532cdb8d63bd4849bcdfd4ea3b67274cf94bc21a9d0"} err="failed to get container status \"35df1ff79d0b4856c5ecd532cdb8d63bd4849bcdfd4ea3b67274cf94bc21a9d0\": rpc error: code = NotFound desc = could not find container \"35df1ff79d0b4856c5ecd532cdb8d63bd4849bcdfd4ea3b67274cf94bc21a9d0\": container with ID starting with 35df1ff79d0b4856c5ecd532cdb8d63bd4849bcdfd4ea3b67274cf94bc21a9d0 not found: ID does not exist"
Jul 18 02:24:59.358640 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:59.358600    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-5e522e36-6687489bb6-8l45r"]
Jul 18 02:24:59.374923 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:59.374887    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-5e522e36-6687489bb6-8l45r"]
Jul 18 02:24:59.969792 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:24:59.969755    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2c16138-96b9-440d-a3ff-7a20792a0c5f" path="/var/lib/kubelet/pods/e2c16138-96b9-440d-a3ff-7a20792a0c5f/volumes"
Jul 18 02:25:03.215881 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:03.215836    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5e522e36-c8t7l"]
Jul 18 02:25:03.218271 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:03.218238    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5e522e36-c8t7l"]
Jul 18 02:25:03.969191 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:03.969150    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8035581-4b97-4ab2-8cb8-0cd0d8b095eb" path="/var/lib/kubelet/pods/d8035581-4b97-4ab2-8cb8-0cd0d8b095eb/volumes"
Jul 18 02:25:06.367623 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:06.367589    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-eaf8d750-688fc4b8fb-wphlt"]
Jul 18 02:25:06.368212 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:06.368192    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2c16138-96b9-440d-a3ff-7a20792a0c5f" containerName="maas-api"
Jul 18 02:25:06.368264 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:06.368217    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2c16138-96b9-440d-a3ff-7a20792a0c5f" containerName="maas-api"
Jul 18 02:25:06.368264 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:06.368235    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8035581-4b97-4ab2-8cb8-0cd0d8b095eb" containerName="revoke-keys"
Jul 18 02:25:06.368264 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:06.368244    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8035581-4b97-4ab2-8cb8-0cd0d8b095eb" containerName="revoke-keys"
Jul 18 02:25:06.368371 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:06.368359    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8035581-4b97-4ab2-8cb8-0cd0d8b095eb" containerName="revoke-keys"
Jul 18 02:25:06.368411 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:06.368372    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2c16138-96b9-440d-a3ff-7a20792a0c5f" containerName="maas-api"
Jul 18 02:25:06.373508 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:06.373486    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaf8d750-688fc4b8fb-wphlt"
Jul 18 02:25:06.376045 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:06.376021    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-eaf8d750-serving-cert\""
Jul 18 02:25:06.382926 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:06.382897    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-eaf8d750-688fc4b8fb-wphlt"]
Jul 18 02:25:06.417198 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:06.417161    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28f49ad5-802c-4c74-8691-05bec8b3e2d2-openshift-service-ca\") pod \"maas-api-e2e-ait-eaf8d750-688fc4b8fb-wphlt\" (UID: \"28f49ad5-802c-4c74-8691-05bec8b3e2d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaf8d750-688fc4b8fb-wphlt"
Jul 18 02:25:06.417371 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:06.417210    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28f49ad5-802c-4c74-8691-05bec8b3e2d2-maas-api-tls\") pod \"maas-api-e2e-ait-eaf8d750-688fc4b8fb-wphlt\" (UID: \"28f49ad5-802c-4c74-8691-05bec8b3e2d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaf8d750-688fc4b8fb-wphlt"
Jul 18 02:25:06.417371 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:06.417303    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4vzp\" (UniqueName: \"kubernetes.io/projected/28f49ad5-802c-4c74-8691-05bec8b3e2d2-kube-api-access-x4vzp\") pod \"maas-api-e2e-ait-eaf8d750-688fc4b8fb-wphlt\" (UID: \"28f49ad5-802c-4c74-8691-05bec8b3e2d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaf8d750-688fc4b8fb-wphlt"
Jul 18 02:25:06.518094 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:06.518049    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28f49ad5-802c-4c74-8691-05bec8b3e2d2-maas-api-tls\") pod \"maas-api-e2e-ait-eaf8d750-688fc4b8fb-wphlt\" (UID: \"28f49ad5-802c-4c74-8691-05bec8b3e2d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaf8d750-688fc4b8fb-wphlt"
Jul 18 02:25:06.518278 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:06.518136    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4vzp\" (UniqueName: \"kubernetes.io/projected/28f49ad5-802c-4c74-8691-05bec8b3e2d2-kube-api-access-x4vzp\") pod \"maas-api-e2e-ait-eaf8d750-688fc4b8fb-wphlt\" (UID: \"28f49ad5-802c-4c74-8691-05bec8b3e2d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaf8d750-688fc4b8fb-wphlt"
Jul 18 02:25:06.518278 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:25:06.518206    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-eaf8d750-serving-cert: secret "maas-api-e2e-ait-eaf8d750-serving-cert" not found
Jul 18 02:25:06.518278 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:06.518224    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28f49ad5-802c-4c74-8691-05bec8b3e2d2-openshift-service-ca\") pod \"maas-api-e2e-ait-eaf8d750-688fc4b8fb-wphlt\" (UID: \"28f49ad5-802c-4c74-8691-05bec8b3e2d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaf8d750-688fc4b8fb-wphlt"
Jul 18 02:25:06.518395 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:25:06.518287    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28f49ad5-802c-4c74-8691-05bec8b3e2d2-maas-api-tls podName:28f49ad5-802c-4c74-8691-05bec8b3e2d2 nodeName:}" failed. No retries permitted until 2026-07-18 02:25:07.018264989 +0000 UTC m=+1701.643301644 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/28f49ad5-802c-4c74-8691-05bec8b3e2d2-maas-api-tls") pod "maas-api-e2e-ait-eaf8d750-688fc4b8fb-wphlt" (UID: "28f49ad5-802c-4c74-8691-05bec8b3e2d2") : secret "maas-api-e2e-ait-eaf8d750-serving-cert" not found
Jul 18 02:25:06.518954 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:06.518935    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28f49ad5-802c-4c74-8691-05bec8b3e2d2-openshift-service-ca\") pod \"maas-api-e2e-ait-eaf8d750-688fc4b8fb-wphlt\" (UID: \"28f49ad5-802c-4c74-8691-05bec8b3e2d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaf8d750-688fc4b8fb-wphlt"
Jul 18 02:25:06.529301 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:06.529270    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4vzp\" (UniqueName: \"kubernetes.io/projected/28f49ad5-802c-4c74-8691-05bec8b3e2d2-kube-api-access-x4vzp\") pod \"maas-api-e2e-ait-eaf8d750-688fc4b8fb-wphlt\" (UID: \"28f49ad5-802c-4c74-8691-05bec8b3e2d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaf8d750-688fc4b8fb-wphlt"
Jul 18 02:25:07.022530 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:07.022488    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28f49ad5-802c-4c74-8691-05bec8b3e2d2-maas-api-tls\") pod \"maas-api-e2e-ait-eaf8d750-688fc4b8fb-wphlt\" (UID: \"28f49ad5-802c-4c74-8691-05bec8b3e2d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaf8d750-688fc4b8fb-wphlt"
Jul 18 02:25:07.024948 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:07.024919    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28f49ad5-802c-4c74-8691-05bec8b3e2d2-maas-api-tls\") pod \"maas-api-e2e-ait-eaf8d750-688fc4b8fb-wphlt\" (UID: \"28f49ad5-802c-4c74-8691-05bec8b3e2d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaf8d750-688fc4b8fb-wphlt"
Jul 18 02:25:07.285654 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:07.285556    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaf8d750-688fc4b8fb-wphlt"
Jul 18 02:25:07.434315 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:07.434243    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-eaf8d750-688fc4b8fb-wphlt"]
Jul 18 02:25:07.436841 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:25:07.436812    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28f49ad5_802c_4c74_8691_05bec8b3e2d2.slice/crio-1fcf1dfd70da1e54af06a69a065f6bd34f7393dcaff99ca768e5b53ab84ca064 WatchSource:0}: Error finding container 1fcf1dfd70da1e54af06a69a065f6bd34f7393dcaff99ca768e5b53ab84ca064: Status 404 returned error can't find the container with id 1fcf1dfd70da1e54af06a69a065f6bd34f7393dcaff99ca768e5b53ab84ca064
Jul 18 02:25:08.381766 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:08.381714    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaf8d750-688fc4b8fb-wphlt" event={"ID":"28f49ad5-802c-4c74-8691-05bec8b3e2d2","Type":"ContainerStarted","Data":"878af143de6f0dae686632df1c0fca02b79002a65c967d6598f1b7cc3c8303ad"}
Jul 18 02:25:08.381953 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:08.381784    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaf8d750-688fc4b8fb-wphlt" event={"ID":"28f49ad5-802c-4c74-8691-05bec8b3e2d2","Type":"ContainerStarted","Data":"1fcf1dfd70da1e54af06a69a065f6bd34f7393dcaff99ca768e5b53ab84ca064"}
Jul 18 02:25:08.381953 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:08.381813    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaf8d750-688fc4b8fb-wphlt"
Jul 18 02:25:08.399319 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:08.399263    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaf8d750-688fc4b8fb-wphlt" podStartSLOduration=1.748463227 podStartE2EDuration="2.399243203s" totalImagesPullingTime="650.779976ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:25:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:25:08.398228738 +0000 UTC m=+1703.023265407" watchObservedRunningTime="2026-07-18 02:25:08.399243203 +0000 UTC m=+1703.024279872"
Jul 18 02:25:14.393074 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:14.393042    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaf8d750-688fc4b8fb-wphlt"
Jul 18 02:25:32.171811 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:32.171759    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eaf8d750-s8vvb"]
Jul 18 02:25:32.175267 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:32.175250    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eaf8d750-s8vvb"
Jul 18 02:25:32.177571 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:32.177547    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g69gp\""
Jul 18 02:25:32.183572 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:32.183546    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eaf8d750-s8vvb"]
Jul 18 02:25:32.347258 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:32.347224    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5447c216-3d2d-48d7-8d24-0cd89ecf6523-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-eaf8d750-s8vvb\" (UID: \"5447c216-3d2d-48d7-8d24-0cd89ecf6523\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eaf8d750-s8vvb"
Jul 18 02:25:32.447885 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:32.447785    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5447c216-3d2d-48d7-8d24-0cd89ecf6523-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-eaf8d750-s8vvb\" (UID: \"5447c216-3d2d-48d7-8d24-0cd89ecf6523\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eaf8d750-s8vvb"
Jul 18 02:25:32.448544 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:32.448520    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5447c216-3d2d-48d7-8d24-0cd89ecf6523-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-eaf8d750-s8vvb\" (UID: \"5447c216-3d2d-48d7-8d24-0cd89ecf6523\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eaf8d750-s8vvb"
Jul 18 02:25:32.487801 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:32.487766    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eaf8d750-s8vvb"
Jul 18 02:25:32.622497 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:32.622467    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eaf8d750-s8vvb"]
Jul 18 02:25:32.624903 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:25:32.624868    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5447c216_3d2d_48d7_8d24_0cd89ecf6523.slice/crio-535fc7c231cbcc033c7cff3901e16c3ae6bd4e151e7fc4554587e689a97cf56c WatchSource:0}: Error finding container 535fc7c231cbcc033c7cff3901e16c3ae6bd4e151e7fc4554587e689a97cf56c: Status 404 returned error can't find the container with id 535fc7c231cbcc033c7cff3901e16c3ae6bd4e151e7fc4554587e689a97cf56c
Jul 18 02:25:33.510265 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:33.510226    2576 generic.go:356] "Generic (PLEG): container finished" podID="5447c216-3d2d-48d7-8d24-0cd89ecf6523" containerID="0e664bca8f60ca0c1e25e226b5c27efe49de34a4b0cfe4c7bb216b7159372d24" exitCode=0
Jul 18 02:25:33.510699 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:33.510316    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eaf8d750-s8vvb" event={"ID":"5447c216-3d2d-48d7-8d24-0cd89ecf6523","Type":"ContainerDied","Data":"0e664bca8f60ca0c1e25e226b5c27efe49de34a4b0cfe4c7bb216b7159372d24"}
Jul 18 02:25:33.510699 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:33.510420    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eaf8d750-s8vvb" event={"ID":"5447c216-3d2d-48d7-8d24-0cd89ecf6523","Type":"ContainerStarted","Data":"535fc7c231cbcc033c7cff3901e16c3ae6bd4e151e7fc4554587e689a97cf56c"}
Jul 18 02:25:34.647116 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:34.647083    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eaf8d750-s8vvb"
Jul 18 02:25:34.768171 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:34.768072    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5447c216-3d2d-48d7-8d24-0cd89ecf6523-maas-api-service-ca\") pod \"5447c216-3d2d-48d7-8d24-0cd89ecf6523\" (UID: \"5447c216-3d2d-48d7-8d24-0cd89ecf6523\") "
Jul 18 02:25:34.768529 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:34.768502    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5447c216-3d2d-48d7-8d24-0cd89ecf6523-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5447c216-3d2d-48d7-8d24-0cd89ecf6523" (UID: "5447c216-3d2d-48d7-8d24-0cd89ecf6523"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:25:34.869087 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:34.869045    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5447c216-3d2d-48d7-8d24-0cd89ecf6523-maas-api-service-ca\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:25:35.522375 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:35.522345    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eaf8d750-s8vvb"
Jul 18 02:25:35.522566 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:35.522384    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eaf8d750-s8vvb" event={"ID":"5447c216-3d2d-48d7-8d24-0cd89ecf6523","Type":"ContainerDied","Data":"535fc7c231cbcc033c7cff3901e16c3ae6bd4e151e7fc4554587e689a97cf56c"}
Jul 18 02:25:35.522566 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:35.522444    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="535fc7c231cbcc033c7cff3901e16c3ae6bd4e151e7fc4554587e689a97cf56c"
Jul 18 02:25:42.934611 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:42.934574    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-eaf8d750-688fc4b8fb-wphlt"]
Jul 18 02:25:42.935171 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:42.935043    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaf8d750-688fc4b8fb-wphlt" podUID="28f49ad5-802c-4c74-8691-05bec8b3e2d2" containerName="maas-api" containerID="cri-o://878af143de6f0dae686632df1c0fca02b79002a65c967d6598f1b7cc3c8303ad" gracePeriod=30
Jul 18 02:25:43.197617 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:43.197550    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaf8d750-688fc4b8fb-wphlt"
Jul 18 02:25:43.236050 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:43.236018    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28f49ad5-802c-4c74-8691-05bec8b3e2d2-maas-api-tls\") pod \"28f49ad5-802c-4c74-8691-05bec8b3e2d2\" (UID: \"28f49ad5-802c-4c74-8691-05bec8b3e2d2\") "
Jul 18 02:25:43.236214 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:43.236077    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x4vzp\" (UniqueName: \"kubernetes.io/projected/28f49ad5-802c-4c74-8691-05bec8b3e2d2-kube-api-access-x4vzp\") pod \"28f49ad5-802c-4c74-8691-05bec8b3e2d2\" (UID: \"28f49ad5-802c-4c74-8691-05bec8b3e2d2\") "
Jul 18 02:25:43.236214 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:43.236131    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28f49ad5-802c-4c74-8691-05bec8b3e2d2-openshift-service-ca\") pod \"28f49ad5-802c-4c74-8691-05bec8b3e2d2\" (UID: \"28f49ad5-802c-4c74-8691-05bec8b3e2d2\") "
Jul 18 02:25:43.236639 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:43.236535    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28f49ad5-802c-4c74-8691-05bec8b3e2d2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "28f49ad5-802c-4c74-8691-05bec8b3e2d2" (UID: "28f49ad5-802c-4c74-8691-05bec8b3e2d2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:25:43.238123 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:43.238093    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28f49ad5-802c-4c74-8691-05bec8b3e2d2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "28f49ad5-802c-4c74-8691-05bec8b3e2d2" (UID: "28f49ad5-802c-4c74-8691-05bec8b3e2d2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:25:43.238233 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:43.238152    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28f49ad5-802c-4c74-8691-05bec8b3e2d2-kube-api-access-x4vzp" (OuterVolumeSpecName: "kube-api-access-x4vzp") pod "28f49ad5-802c-4c74-8691-05bec8b3e2d2" (UID: "28f49ad5-802c-4c74-8691-05bec8b3e2d2"). InnerVolumeSpecName "kube-api-access-x4vzp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:25:43.337603 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:43.337566    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28f49ad5-802c-4c74-8691-05bec8b3e2d2-openshift-service-ca\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:25:43.337603 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:43.337602    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28f49ad5-802c-4c74-8691-05bec8b3e2d2-maas-api-tls\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:25:43.337823 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:43.337617    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4vzp\" (UniqueName: \"kubernetes.io/projected/28f49ad5-802c-4c74-8691-05bec8b3e2d2-kube-api-access-x4vzp\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:25:43.567513 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:43.567408    2576 generic.go:356] "Generic (PLEG): container finished" podID="28f49ad5-802c-4c74-8691-05bec8b3e2d2" containerID="878af143de6f0dae686632df1c0fca02b79002a65c967d6598f1b7cc3c8303ad" exitCode=0
Jul 18 02:25:43.567513 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:43.567475    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaf8d750-688fc4b8fb-wphlt" event={"ID":"28f49ad5-802c-4c74-8691-05bec8b3e2d2","Type":"ContainerDied","Data":"878af143de6f0dae686632df1c0fca02b79002a65c967d6598f1b7cc3c8303ad"}
Jul 18 02:25:43.567513 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:43.567484    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaf8d750-688fc4b8fb-wphlt"
Jul 18 02:25:43.567752 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:43.567528    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eaf8d750-688fc4b8fb-wphlt" event={"ID":"28f49ad5-802c-4c74-8691-05bec8b3e2d2","Type":"ContainerDied","Data":"1fcf1dfd70da1e54af06a69a065f6bd34f7393dcaff99ca768e5b53ab84ca064"}
Jul 18 02:25:43.567752 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:43.567560    2576 scope.go:117] "RemoveContainer" containerID="878af143de6f0dae686632df1c0fca02b79002a65c967d6598f1b7cc3c8303ad"
Jul 18 02:25:43.577248 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:43.577227    2576 scope.go:117] "RemoveContainer" containerID="878af143de6f0dae686632df1c0fca02b79002a65c967d6598f1b7cc3c8303ad"
Jul 18 02:25:43.577572 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:25:43.577543    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"878af143de6f0dae686632df1c0fca02b79002a65c967d6598f1b7cc3c8303ad\": container with ID starting with 878af143de6f0dae686632df1c0fca02b79002a65c967d6598f1b7cc3c8303ad not found: ID does not exist" containerID="878af143de6f0dae686632df1c0fca02b79002a65c967d6598f1b7cc3c8303ad"
Jul 18 02:25:43.577636 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:43.577584    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"878af143de6f0dae686632df1c0fca02b79002a65c967d6598f1b7cc3c8303ad"} err="failed to get container status \"878af143de6f0dae686632df1c0fca02b79002a65c967d6598f1b7cc3c8303ad\": rpc error: code = NotFound desc = could not find container \"878af143de6f0dae686632df1c0fca02b79002a65c967d6598f1b7cc3c8303ad\": container with ID starting with 878af143de6f0dae686632df1c0fca02b79002a65c967d6598f1b7cc3c8303ad not found: ID does not exist"
Jul 18 02:25:43.590108 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:43.590071    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-eaf8d750-688fc4b8fb-wphlt"]
Jul 18 02:25:43.594336 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:43.594300    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-eaf8d750-688fc4b8fb-wphlt"]
Jul 18 02:25:43.969402 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:43.969369    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28f49ad5-802c-4c74-8691-05bec8b3e2d2" path="/var/lib/kubelet/pods/28f49ad5-802c-4c74-8691-05bec8b3e2d2/volumes"
Jul 18 02:25:47.260217 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:47.260184    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eaf8d750-s8vvb"]
Jul 18 02:25:47.262130 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:47.262106    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eaf8d750-s8vvb"]
Jul 18 02:25:47.970466 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:47.970411    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5447c216-3d2d-48d7-8d24-0cd89ecf6523" path="/var/lib/kubelet/pods/5447c216-3d2d-48d7-8d24-0cd89ecf6523/volumes"
Jul 18 02:25:51.527975 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:51.527940    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-be63187d-758dcc4597-4njht"]
Jul 18 02:25:51.528458 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:51.528315    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5447c216-3d2d-48d7-8d24-0cd89ecf6523" containerName="revoke-keys"
Jul 18 02:25:51.528458 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:51.528327    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5447c216-3d2d-48d7-8d24-0cd89ecf6523" containerName="revoke-keys"
Jul 18 02:25:51.528458 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:51.528341    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28f49ad5-802c-4c74-8691-05bec8b3e2d2" containerName="maas-api"
Jul 18 02:25:51.528458 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:51.528347    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="28f49ad5-802c-4c74-8691-05bec8b3e2d2" containerName="maas-api"
Jul 18 02:25:51.528458 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:51.528417    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5447c216-3d2d-48d7-8d24-0cd89ecf6523" containerName="revoke-keys"
Jul 18 02:25:51.528458 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:51.528444    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="28f49ad5-802c-4c74-8691-05bec8b3e2d2" containerName="maas-api"
Jul 18 02:25:51.531404 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:51.531383    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-be63187d-758dcc4597-4njht"
Jul 18 02:25:51.535755 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:51.535732    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-be63187d-serving-cert\""
Jul 18 02:25:51.543764 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:51.543737    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-be63187d-758dcc4597-4njht"]
Jul 18 02:25:51.617655 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:51.617621    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02883ddc-f071-4a02-a5ef-6f643a45b20d-maas-api-tls\") pod \"maas-api-e2e-derive-be63187d-758dcc4597-4njht\" (UID: \"02883ddc-f071-4a02-a5ef-6f643a45b20d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-be63187d-758dcc4597-4njht"
Jul 18 02:25:51.617655 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:51.617657    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02883ddc-f071-4a02-a5ef-6f643a45b20d-openshift-service-ca\") pod \"maas-api-e2e-derive-be63187d-758dcc4597-4njht\" (UID: \"02883ddc-f071-4a02-a5ef-6f643a45b20d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-be63187d-758dcc4597-4njht"
Jul 18 02:25:51.617885 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:51.617676    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqz8t\" (UniqueName: \"kubernetes.io/projected/02883ddc-f071-4a02-a5ef-6f643a45b20d-kube-api-access-sqz8t\") pod \"maas-api-e2e-derive-be63187d-758dcc4597-4njht\" (UID: \"02883ddc-f071-4a02-a5ef-6f643a45b20d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-be63187d-758dcc4597-4njht"
Jul 18 02:25:51.718608 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:51.718566    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqz8t\" (UniqueName: \"kubernetes.io/projected/02883ddc-f071-4a02-a5ef-6f643a45b20d-kube-api-access-sqz8t\") pod \"maas-api-e2e-derive-be63187d-758dcc4597-4njht\" (UID: \"02883ddc-f071-4a02-a5ef-6f643a45b20d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-be63187d-758dcc4597-4njht"
Jul 18 02:25:51.718787 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:51.718692    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02883ddc-f071-4a02-a5ef-6f643a45b20d-maas-api-tls\") pod \"maas-api-e2e-derive-be63187d-758dcc4597-4njht\" (UID: \"02883ddc-f071-4a02-a5ef-6f643a45b20d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-be63187d-758dcc4597-4njht"
Jul 18 02:25:51.718787 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:51.718711    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02883ddc-f071-4a02-a5ef-6f643a45b20d-openshift-service-ca\") pod \"maas-api-e2e-derive-be63187d-758dcc4597-4njht\" (UID: \"02883ddc-f071-4a02-a5ef-6f643a45b20d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-be63187d-758dcc4597-4njht"
Jul 18 02:25:51.719309 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:51.719290    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02883ddc-f071-4a02-a5ef-6f643a45b20d-openshift-service-ca\") pod \"maas-api-e2e-derive-be63187d-758dcc4597-4njht\" (UID: \"02883ddc-f071-4a02-a5ef-6f643a45b20d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-be63187d-758dcc4597-4njht"
Jul 18 02:25:51.721380 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:51.721354    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02883ddc-f071-4a02-a5ef-6f643a45b20d-maas-api-tls\") pod \"maas-api-e2e-derive-be63187d-758dcc4597-4njht\" (UID: \"02883ddc-f071-4a02-a5ef-6f643a45b20d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-be63187d-758dcc4597-4njht"
Jul 18 02:25:51.726990 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:51.726962    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqz8t\" (UniqueName: \"kubernetes.io/projected/02883ddc-f071-4a02-a5ef-6f643a45b20d-kube-api-access-sqz8t\") pod \"maas-api-e2e-derive-be63187d-758dcc4597-4njht\" (UID: \"02883ddc-f071-4a02-a5ef-6f643a45b20d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-be63187d-758dcc4597-4njht"
Jul 18 02:25:51.844254 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:51.844154    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-be63187d-758dcc4597-4njht"
Jul 18 02:25:52.006785 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:25:52.006753    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02883ddc_f071_4a02_a5ef_6f643a45b20d.slice/crio-17abcd207378bb0000cf986e4eebd1e6e7ba13e301ca3411ae24a61eb0c496c8 WatchSource:0}: Error finding container 17abcd207378bb0000cf986e4eebd1e6e7ba13e301ca3411ae24a61eb0c496c8: Status 404 returned error can't find the container with id 17abcd207378bb0000cf986e4eebd1e6e7ba13e301ca3411ae24a61eb0c496c8
Jul 18 02:25:52.007058 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:52.007037    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-be63187d-758dcc4597-4njht"]
Jul 18 02:25:52.623945 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:52.623858    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-be63187d-758dcc4597-4njht" event={"ID":"02883ddc-f071-4a02-a5ef-6f643a45b20d","Type":"ContainerStarted","Data":"d31c095d4f85f2be711dd812ccb07a9bf1add55d18c22e86fab99e43e003ab47"}
Jul 18 02:25:52.623945 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:52.623914    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-be63187d-758dcc4597-4njht" event={"ID":"02883ddc-f071-4a02-a5ef-6f643a45b20d","Type":"ContainerStarted","Data":"17abcd207378bb0000cf986e4eebd1e6e7ba13e301ca3411ae24a61eb0c496c8"}
Jul 18 02:25:52.623945 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:52.623942    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-be63187d-758dcc4597-4njht"
Jul 18 02:25:52.643174 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:52.643111    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-be63187d-758dcc4597-4njht" podStartSLOduration=1.281004072 podStartE2EDuration="1.643095336s" totalImagesPullingTime="362.091264ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:25:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:25:52.642546015 +0000 UTC m=+1747.267582706" watchObservedRunningTime="2026-07-18 02:25:52.643095336 +0000 UTC m=+1747.268132033"
Jul 18 02:25:58.635251 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:25:58.635222    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-be63187d-758dcc4597-4njht"
Jul 18 02:26:16.328529 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:16.328494    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-be63187d-kfx7b"]
Jul 18 02:26:16.332267 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:16.332249    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-be63187d-kfx7b"
Jul 18 02:26:16.334719 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:16.334700    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g69gp\""
Jul 18 02:26:16.339398 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:16.339360    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-be63187d-kfx7b"]
Jul 18 02:26:16.442995 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:16.442962    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/143cfbba-05d0-42c6-9341-0724335c4b18-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-be63187d-kfx7b\" (UID: \"143cfbba-05d0-42c6-9341-0724335c4b18\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-be63187d-kfx7b"
Jul 18 02:26:16.544021 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:16.543984    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/143cfbba-05d0-42c6-9341-0724335c4b18-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-be63187d-kfx7b\" (UID: \"143cfbba-05d0-42c6-9341-0724335c4b18\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-be63187d-kfx7b"
Jul 18 02:26:16.544700 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:16.544675    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/143cfbba-05d0-42c6-9341-0724335c4b18-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-be63187d-kfx7b\" (UID: \"143cfbba-05d0-42c6-9341-0724335c4b18\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-be63187d-kfx7b"
Jul 18 02:26:16.645197 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:16.645160    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-be63187d-kfx7b"
Jul 18 02:26:16.791374 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:16.791338    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-be63187d-kfx7b"]
Jul 18 02:26:16.792280 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:26:16.792256    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod143cfbba_05d0_42c6_9341_0724335c4b18.slice/crio-f29dbf1f3cced64e3b5f62ec72db6120b0ab20b2d88be9117ffd5576b662bded WatchSource:0}: Error finding container f29dbf1f3cced64e3b5f62ec72db6120b0ab20b2d88be9117ffd5576b662bded: Status 404 returned error can't find the container with id f29dbf1f3cced64e3b5f62ec72db6120b0ab20b2d88be9117ffd5576b662bded
Jul 18 02:26:17.769609 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:17.769516    2576 generic.go:356] "Generic (PLEG): container finished" podID="143cfbba-05d0-42c6-9341-0724335c4b18" containerID="bc40242cecdca9dece0c596189a6480e05480de7e688dc3cb6e9962f88cf51b3" exitCode=0
Jul 18 02:26:17.770021 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:17.769606    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-be63187d-kfx7b" event={"ID":"143cfbba-05d0-42c6-9341-0724335c4b18","Type":"ContainerDied","Data":"bc40242cecdca9dece0c596189a6480e05480de7e688dc3cb6e9962f88cf51b3"}
Jul 18 02:26:17.770021 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:17.769654    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-be63187d-kfx7b" event={"ID":"143cfbba-05d0-42c6-9341-0724335c4b18","Type":"ContainerStarted","Data":"f29dbf1f3cced64e3b5f62ec72db6120b0ab20b2d88be9117ffd5576b662bded"}
Jul 18 02:26:18.912812 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:18.912783    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-be63187d-kfx7b"
Jul 18 02:26:19.067624 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:19.067530    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/143cfbba-05d0-42c6-9341-0724335c4b18-maas-api-service-ca\") pod \"143cfbba-05d0-42c6-9341-0724335c4b18\" (UID: \"143cfbba-05d0-42c6-9341-0724335c4b18\") "
Jul 18 02:26:19.067888 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:19.067867    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/143cfbba-05d0-42c6-9341-0724335c4b18-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "143cfbba-05d0-42c6-9341-0724335c4b18" (UID: "143cfbba-05d0-42c6-9341-0724335c4b18"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:26:19.169063 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:19.169027    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/143cfbba-05d0-42c6-9341-0724335c4b18-maas-api-service-ca\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:26:19.780961 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:19.780936    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-be63187d-kfx7b"
Jul 18 02:26:19.781145 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:19.780972    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-be63187d-kfx7b" event={"ID":"143cfbba-05d0-42c6-9341-0724335c4b18","Type":"ContainerDied","Data":"f29dbf1f3cced64e3b5f62ec72db6120b0ab20b2d88be9117ffd5576b662bded"}
Jul 18 02:26:19.781145 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:19.781018    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f29dbf1f3cced64e3b5f62ec72db6120b0ab20b2d88be9117ffd5576b662bded"
Jul 18 02:26:26.902985 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:26.902945    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-be63187d-758dcc4597-4njht"]
Jul 18 02:26:26.903479 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:26.903247    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-be63187d-758dcc4597-4njht" podUID="02883ddc-f071-4a02-a5ef-6f643a45b20d" containerName="maas-api" containerID="cri-o://d31c095d4f85f2be711dd812ccb07a9bf1add55d18c22e86fab99e43e003ab47" gracePeriod=30
Jul 18 02:26:27.165854 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:27.165789    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-be63187d-758dcc4597-4njht"
Jul 18 02:26:27.246042 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:27.246003    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02883ddc-f071-4a02-a5ef-6f643a45b20d-maas-api-tls\") pod \"02883ddc-f071-4a02-a5ef-6f643a45b20d\" (UID: \"02883ddc-f071-4a02-a5ef-6f643a45b20d\") "
Jul 18 02:26:27.246247 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:27.246068    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02883ddc-f071-4a02-a5ef-6f643a45b20d-openshift-service-ca\") pod \"02883ddc-f071-4a02-a5ef-6f643a45b20d\" (UID: \"02883ddc-f071-4a02-a5ef-6f643a45b20d\") "
Jul 18 02:26:27.246247 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:27.246089    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqz8t\" (UniqueName: \"kubernetes.io/projected/02883ddc-f071-4a02-a5ef-6f643a45b20d-kube-api-access-sqz8t\") pod \"02883ddc-f071-4a02-a5ef-6f643a45b20d\" (UID: \"02883ddc-f071-4a02-a5ef-6f643a45b20d\") "
Jul 18 02:26:27.246488 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:27.246421    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02883ddc-f071-4a02-a5ef-6f643a45b20d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "02883ddc-f071-4a02-a5ef-6f643a45b20d" (UID: "02883ddc-f071-4a02-a5ef-6f643a45b20d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:26:27.248229 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:27.248192    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02883ddc-f071-4a02-a5ef-6f643a45b20d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "02883ddc-f071-4a02-a5ef-6f643a45b20d" (UID: "02883ddc-f071-4a02-a5ef-6f643a45b20d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:26:27.248327 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:27.248231    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02883ddc-f071-4a02-a5ef-6f643a45b20d-kube-api-access-sqz8t" (OuterVolumeSpecName: "kube-api-access-sqz8t") pod "02883ddc-f071-4a02-a5ef-6f643a45b20d" (UID: "02883ddc-f071-4a02-a5ef-6f643a45b20d"). InnerVolumeSpecName "kube-api-access-sqz8t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:26:27.347177 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:27.347142    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02883ddc-f071-4a02-a5ef-6f643a45b20d-openshift-service-ca\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:26:27.347177 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:27.347173    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqz8t\" (UniqueName: \"kubernetes.io/projected/02883ddc-f071-4a02-a5ef-6f643a45b20d-kube-api-access-sqz8t\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:26:27.347177 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:27.347184    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02883ddc-f071-4a02-a5ef-6f643a45b20d-maas-api-tls\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:26:27.831866 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:27.831827    2576 generic.go:356] "Generic (PLEG): container finished" podID="02883ddc-f071-4a02-a5ef-6f643a45b20d" containerID="d31c095d4f85f2be711dd812ccb07a9bf1add55d18c22e86fab99e43e003ab47" exitCode=0
Jul 18 02:26:27.832084 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:27.831877    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-be63187d-758dcc4597-4njht"
Jul 18 02:26:27.832084 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:27.831917    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-be63187d-758dcc4597-4njht" event={"ID":"02883ddc-f071-4a02-a5ef-6f643a45b20d","Type":"ContainerDied","Data":"d31c095d4f85f2be711dd812ccb07a9bf1add55d18c22e86fab99e43e003ab47"}
Jul 18 02:26:27.832084 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:27.831979    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-be63187d-758dcc4597-4njht" event={"ID":"02883ddc-f071-4a02-a5ef-6f643a45b20d","Type":"ContainerDied","Data":"17abcd207378bb0000cf986e4eebd1e6e7ba13e301ca3411ae24a61eb0c496c8"}
Jul 18 02:26:27.832084 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:27.832010    2576 scope.go:117] "RemoveContainer" containerID="d31c095d4f85f2be711dd812ccb07a9bf1add55d18c22e86fab99e43e003ab47"
Jul 18 02:26:27.840929 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:27.840910    2576 scope.go:117] "RemoveContainer" containerID="d31c095d4f85f2be711dd812ccb07a9bf1add55d18c22e86fab99e43e003ab47"
Jul 18 02:26:27.841207 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:26:27.841188    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d31c095d4f85f2be711dd812ccb07a9bf1add55d18c22e86fab99e43e003ab47\": container with ID starting with d31c095d4f85f2be711dd812ccb07a9bf1add55d18c22e86fab99e43e003ab47 not found: ID does not exist" containerID="d31c095d4f85f2be711dd812ccb07a9bf1add55d18c22e86fab99e43e003ab47"
Jul 18 02:26:27.841277 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:27.841221    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d31c095d4f85f2be711dd812ccb07a9bf1add55d18c22e86fab99e43e003ab47"} err="failed to get container status \"d31c095d4f85f2be711dd812ccb07a9bf1add55d18c22e86fab99e43e003ab47\": rpc error: code = NotFound desc = could not find container \"d31c095d4f85f2be711dd812ccb07a9bf1add55d18c22e86fab99e43e003ab47\": container with ID starting with d31c095d4f85f2be711dd812ccb07a9bf1add55d18c22e86fab99e43e003ab47 not found: ID does not exist"
Jul 18 02:26:27.854058 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:27.854033    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-be63187d-758dcc4597-4njht"]
Jul 18 02:26:27.857927 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:27.857904    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-be63187d-758dcc4597-4njht"]
Jul 18 02:26:27.969904 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:27.969873    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02883ddc-f071-4a02-a5ef-6f643a45b20d" path="/var/lib/kubelet/pods/02883ddc-f071-4a02-a5ef-6f643a45b20d/volumes"
Jul 18 02:26:31.446070 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:31.446031    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-be63187d-kfx7b"]
Jul 18 02:26:31.450228 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:31.450190    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-be63187d-kfx7b"]
Jul 18 02:26:31.969472 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:31.969420    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="143cfbba-05d0-42c6-9341-0724335c4b18" path="/var/lib/kubelet/pods/143cfbba-05d0-42c6-9341-0724335c4b18/volumes"
Jul 18 02:26:45.951764 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:45.951727    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-bfrdg_9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777/console-operator/1.log"
Jul 18 02:26:45.954347 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:26:45.954323    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-bfrdg_9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777/console-operator/1.log"
Jul 18 02:28:15.810789 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:15.810695    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2bf983eb-654988bd76-qjgt9"]
Jul 18 02:28:15.811362 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:15.811085    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="143cfbba-05d0-42c6-9341-0724335c4b18" containerName="revoke-keys"
Jul 18 02:28:15.811362 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:15.811096    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="143cfbba-05d0-42c6-9341-0724335c4b18" containerName="revoke-keys"
Jul 18 02:28:15.811362 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:15.811108    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02883ddc-f071-4a02-a5ef-6f643a45b20d" containerName="maas-api"
Jul 18 02:28:15.811362 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:15.811114    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="02883ddc-f071-4a02-a5ef-6f643a45b20d" containerName="maas-api"
Jul 18 02:28:15.811362 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:15.811188    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="02883ddc-f071-4a02-a5ef-6f643a45b20d" containerName="maas-api"
Jul 18 02:28:15.811362 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:15.811196    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="143cfbba-05d0-42c6-9341-0724335c4b18" containerName="revoke-keys"
Jul 18 02:28:15.814569 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:15.814537    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf983eb-654988bd76-qjgt9"
Jul 18 02:28:15.817670 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:15.817647    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-2bf983eb-serving-cert\""
Jul 18 02:28:15.818267 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:15.818223    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2bf983eb-654988bd76-qjgt9"]
Jul 18 02:28:15.828849 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:15.828827    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c-maas-api-tls\") pod \"maas-api-e2e-mt-2bf983eb-654988bd76-qjgt9\" (UID: \"b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf983eb-654988bd76-qjgt9"
Jul 18 02:28:15.828973 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:15.828883    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbz2l\" (UniqueName: \"kubernetes.io/projected/b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c-kube-api-access-dbz2l\") pod \"maas-api-e2e-mt-2bf983eb-654988bd76-qjgt9\" (UID: \"b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf983eb-654988bd76-qjgt9"
Jul 18 02:28:15.828973 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:15.828921    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c-openshift-service-ca\") pod \"maas-api-e2e-mt-2bf983eb-654988bd76-qjgt9\" (UID: \"b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf983eb-654988bd76-qjgt9"
Jul 18 02:28:15.929608 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:15.929576    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbz2l\" (UniqueName: \"kubernetes.io/projected/b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c-kube-api-access-dbz2l\") pod \"maas-api-e2e-mt-2bf983eb-654988bd76-qjgt9\" (UID: \"b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf983eb-654988bd76-qjgt9"
Jul 18 02:28:15.929814 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:15.929633    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c-openshift-service-ca\") pod \"maas-api-e2e-mt-2bf983eb-654988bd76-qjgt9\" (UID: \"b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf983eb-654988bd76-qjgt9"
Jul 18 02:28:15.929814 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:15.929669    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c-maas-api-tls\") pod \"maas-api-e2e-mt-2bf983eb-654988bd76-qjgt9\" (UID: \"b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf983eb-654988bd76-qjgt9"
Jul 18 02:28:15.929814 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:28:15.929777    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-2bf983eb-serving-cert: secret "maas-api-e2e-mt-2bf983eb-serving-cert" not found
Jul 18 02:28:15.929996 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:28:15.929855    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c-maas-api-tls podName:b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c nodeName:}" failed. No retries permitted until 2026-07-18 02:28:16.429834253 +0000 UTC m=+1891.054870899 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c-maas-api-tls") pod "maas-api-e2e-mt-2bf983eb-654988bd76-qjgt9" (UID: "b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c") : secret "maas-api-e2e-mt-2bf983eb-serving-cert" not found
Jul 18 02:28:15.930359 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:15.930337    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c-openshift-service-ca\") pod \"maas-api-e2e-mt-2bf983eb-654988bd76-qjgt9\" (UID: \"b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf983eb-654988bd76-qjgt9"
Jul 18 02:28:15.938820 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:15.938797    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbz2l\" (UniqueName: \"kubernetes.io/projected/b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c-kube-api-access-dbz2l\") pod \"maas-api-e2e-mt-2bf983eb-654988bd76-qjgt9\" (UID: \"b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf983eb-654988bd76-qjgt9"
Jul 18 02:28:16.433064 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:16.433025    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c-maas-api-tls\") pod \"maas-api-e2e-mt-2bf983eb-654988bd76-qjgt9\" (UID: \"b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf983eb-654988bd76-qjgt9"
Jul 18 02:28:16.435602 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:16.435579    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c-maas-api-tls\") pod \"maas-api-e2e-mt-2bf983eb-654988bd76-qjgt9\" (UID: \"b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf983eb-654988bd76-qjgt9"
Jul 18 02:28:16.725979 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:16.725884    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf983eb-654988bd76-qjgt9"
Jul 18 02:28:16.865410 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:16.865380    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2bf983eb-654988bd76-qjgt9"]
Jul 18 02:28:16.867954 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:28:16.867925    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0eaafd0_36ef_40fa_8c21_1dd0bef59f0c.slice/crio-609f0f29d44ed9a76315039e68c76f00d50dfd3af744ac244536f2879b630460 WatchSource:0}: Error finding container 609f0f29d44ed9a76315039e68c76f00d50dfd3af744ac244536f2879b630460: Status 404 returned error can't find the container with id 609f0f29d44ed9a76315039e68c76f00d50dfd3af744ac244536f2879b630460
Jul 18 02:28:17.402807 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:17.402770    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf983eb-654988bd76-qjgt9" event={"ID":"b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c","Type":"ContainerStarted","Data":"e5aadb8fc6820658b2d481e9f33b3830a1c9d476357023bc05f3237c8165397f"}
Jul 18 02:28:17.403005 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:17.402829    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf983eb-654988bd76-qjgt9" event={"ID":"b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c","Type":"ContainerStarted","Data":"609f0f29d44ed9a76315039e68c76f00d50dfd3af744ac244536f2879b630460"}
Jul 18 02:28:17.403005 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:17.402955    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf983eb-654988bd76-qjgt9"
Jul 18 02:28:17.420056 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:17.420007    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf983eb-654988bd76-qjgt9" podStartSLOduration=2.016708663 podStartE2EDuration="2.419991602s" totalImagesPullingTime="403.282939ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:28:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:28:17.418420033 +0000 UTC m=+1892.043456702" watchObservedRunningTime="2026-07-18 02:28:17.419991602 +0000 UTC m=+1892.045028268"
Jul 18 02:28:23.414143 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:23.414109    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf983eb-654988bd76-qjgt9"
Jul 18 02:28:27.885458 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:27.885404    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2bf983eb-m4qhn"]
Jul 18 02:28:27.889158 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:27.889137    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2bf983eb-m4qhn"
Jul 18 02:28:27.892406 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:27.892383    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g69gp\""
Jul 18 02:28:27.900790 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:27.900761    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2bf983eb-m4qhn"]
Jul 18 02:28:27.936937 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:27.936905    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33b75464-69f0-46b7-836c-314f2b2a1d7e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2bf983eb-m4qhn\" (UID: \"33b75464-69f0-46b7-836c-314f2b2a1d7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2bf983eb-m4qhn"
Jul 18 02:28:28.037911 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:28.037874    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33b75464-69f0-46b7-836c-314f2b2a1d7e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2bf983eb-m4qhn\" (UID: \"33b75464-69f0-46b7-836c-314f2b2a1d7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2bf983eb-m4qhn"
Jul 18 02:28:28.038520 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:28.038500    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33b75464-69f0-46b7-836c-314f2b2a1d7e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2bf983eb-m4qhn\" (UID: \"33b75464-69f0-46b7-836c-314f2b2a1d7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2bf983eb-m4qhn"
Jul 18 02:28:28.201138 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:28.201040    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2bf983eb-m4qhn"
Jul 18 02:28:28.339120 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:28.339082    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2bf983eb-m4qhn"]
Jul 18 02:28:28.340526 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:28:28.340491    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33b75464_69f0_46b7_836c_314f2b2a1d7e.slice/crio-26c9c40009b8066d975f0f54b6bbff8f8d54687e47a8095a647ec75d0f09052b WatchSource:0}: Error finding container 26c9c40009b8066d975f0f54b6bbff8f8d54687e47a8095a647ec75d0f09052b: Status 404 returned error can't find the container with id 26c9c40009b8066d975f0f54b6bbff8f8d54687e47a8095a647ec75d0f09052b
Jul 18 02:28:28.467164 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:28.467080    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2bf983eb-m4qhn" event={"ID":"33b75464-69f0-46b7-836c-314f2b2a1d7e","Type":"ContainerStarted","Data":"26c9c40009b8066d975f0f54b6bbff8f8d54687e47a8095a647ec75d0f09052b"}
Jul 18 02:28:29.474812 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:29.474774    2576 generic.go:356] "Generic (PLEG): container finished" podID="33b75464-69f0-46b7-836c-314f2b2a1d7e" containerID="a7af5a0191159bfc30f9d2360481e27124e1f66f596a9cd124284381a7aead7f" exitCode=0
Jul 18 02:28:29.475202 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:29.474856    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2bf983eb-m4qhn" event={"ID":"33b75464-69f0-46b7-836c-314f2b2a1d7e","Type":"ContainerDied","Data":"a7af5a0191159bfc30f9d2360481e27124e1f66f596a9cd124284381a7aead7f"}
Jul 18 02:28:30.622143 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:30.622116    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2bf983eb-m4qhn"
Jul 18 02:28:30.661375 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:30.661344    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33b75464-69f0-46b7-836c-314f2b2a1d7e-maas-api-service-ca\") pod \"33b75464-69f0-46b7-836c-314f2b2a1d7e\" (UID: \"33b75464-69f0-46b7-836c-314f2b2a1d7e\") "
Jul 18 02:28:30.661753 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:30.661723    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33b75464-69f0-46b7-836c-314f2b2a1d7e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "33b75464-69f0-46b7-836c-314f2b2a1d7e" (UID: "33b75464-69f0-46b7-836c-314f2b2a1d7e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:28:30.762162 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:30.762073    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33b75464-69f0-46b7-836c-314f2b2a1d7e-maas-api-service-ca\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:28:31.485402 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:31.485372    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2bf983eb-m4qhn"
Jul 18 02:28:31.485402 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:31.485394    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2bf983eb-m4qhn" event={"ID":"33b75464-69f0-46b7-836c-314f2b2a1d7e","Type":"ContainerDied","Data":"26c9c40009b8066d975f0f54b6bbff8f8d54687e47a8095a647ec75d0f09052b"}
Jul 18 02:28:31.485631 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:31.485455    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="26c9c40009b8066d975f0f54b6bbff8f8d54687e47a8095a647ec75d0f09052b"
Jul 18 02:28:38.533231 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:38.533194    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2bf983eb-654988bd76-qjgt9"]
Jul 18 02:28:38.533982 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:38.533949    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf983eb-654988bd76-qjgt9" podUID="b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c" containerName="maas-api" containerID="cri-o://e5aadb8fc6820658b2d481e9f33b3830a1c9d476357023bc05f3237c8165397f" gracePeriod=30
Jul 18 02:28:38.796413 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:38.796343    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf983eb-654988bd76-qjgt9"
Jul 18 02:28:38.940103 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:38.939994    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dbz2l\" (UniqueName: \"kubernetes.io/projected/b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c-kube-api-access-dbz2l\") pod \"b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c\" (UID: \"b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c\") "
Jul 18 02:28:38.940103 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:38.940071    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c-maas-api-tls\") pod \"b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c\" (UID: \"b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c\") "
Jul 18 02:28:38.940103 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:38.940102    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c-openshift-service-ca\") pod \"b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c\" (UID: \"b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c\") "
Jul 18 02:28:38.947254 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:38.940491    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c" (UID: "b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:28:38.947254 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:38.942185    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c" (UID: "b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:28:38.947254 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:38.942231    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c-kube-api-access-dbz2l" (OuterVolumeSpecName: "kube-api-access-dbz2l") pod "b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c" (UID: "b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c"). InnerVolumeSpecName "kube-api-access-dbz2l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:28:39.041164 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:39.041126    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c-maas-api-tls\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:28:39.041164 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:39.041160    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c-openshift-service-ca\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:28:39.041164 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:39.041170    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dbz2l\" (UniqueName: \"kubernetes.io/projected/b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c-kube-api-access-dbz2l\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:28:39.530652 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:39.530619    2576 generic.go:356] "Generic (PLEG): container finished" podID="b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c" containerID="e5aadb8fc6820658b2d481e9f33b3830a1c9d476357023bc05f3237c8165397f" exitCode=0
Jul 18 02:28:39.530842 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:39.530665    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf983eb-654988bd76-qjgt9"
Jul 18 02:28:39.530842 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:39.530718    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf983eb-654988bd76-qjgt9" event={"ID":"b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c","Type":"ContainerDied","Data":"e5aadb8fc6820658b2d481e9f33b3830a1c9d476357023bc05f3237c8165397f"}
Jul 18 02:28:39.530842 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:39.530773    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf983eb-654988bd76-qjgt9" event={"ID":"b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c","Type":"ContainerDied","Data":"609f0f29d44ed9a76315039e68c76f00d50dfd3af744ac244536f2879b630460"}
Jul 18 02:28:39.530842 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:39.530803    2576 scope.go:117] "RemoveContainer" containerID="e5aadb8fc6820658b2d481e9f33b3830a1c9d476357023bc05f3237c8165397f"
Jul 18 02:28:39.547179 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:39.547040    2576 scope.go:117] "RemoveContainer" containerID="e5aadb8fc6820658b2d481e9f33b3830a1c9d476357023bc05f3237c8165397f"
Jul 18 02:28:39.547395 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:28:39.547299    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5aadb8fc6820658b2d481e9f33b3830a1c9d476357023bc05f3237c8165397f\": container with ID starting with e5aadb8fc6820658b2d481e9f33b3830a1c9d476357023bc05f3237c8165397f not found: ID does not exist" containerID="e5aadb8fc6820658b2d481e9f33b3830a1c9d476357023bc05f3237c8165397f"
Jul 18 02:28:39.547395 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:39.547326    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5aadb8fc6820658b2d481e9f33b3830a1c9d476357023bc05f3237c8165397f"} err="failed to get container status \"e5aadb8fc6820658b2d481e9f33b3830a1c9d476357023bc05f3237c8165397f\": rpc error: code = NotFound desc = could not find container \"e5aadb8fc6820658b2d481e9f33b3830a1c9d476357023bc05f3237c8165397f\": container with ID starting with e5aadb8fc6820658b2d481e9f33b3830a1c9d476357023bc05f3237c8165397f not found: ID does not exist"
Jul 18 02:28:39.555273 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:39.555249    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2bf983eb-654988bd76-qjgt9"]
Jul 18 02:28:39.558959 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:39.558937    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2bf983eb-654988bd76-qjgt9"]
Jul 18 02:28:39.969976 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:39.969938    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c" path="/var/lib/kubelet/pods/b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c/volumes"
Jul 18 02:28:42.993675 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:42.993634    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2bf983eb-m4qhn"]
Jul 18 02:28:42.996158 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:42.996134    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2bf983eb-m4qhn"]
Jul 18 02:28:43.969327 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:28:43.969294    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33b75464-69f0-46b7-836c-314f2b2a1d7e" path="/var/lib/kubelet/pods/33b75464-69f0-46b7-836c-314f2b2a1d7e/volumes"
Jul 18 02:29:13.100365 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:13.100327    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-327441da-6484b7dc5c-8zrv8"]
Jul 18 02:29:13.100952 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:13.100919    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c" containerName="maas-api"
Jul 18 02:29:13.100952 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:13.100939    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c" containerName="maas-api"
Jul 18 02:29:13.100952 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:13.100953    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33b75464-69f0-46b7-836c-314f2b2a1d7e" containerName="revoke-keys"
Jul 18 02:29:13.101124 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:13.100962    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="33b75464-69f0-46b7-836c-314f2b2a1d7e" containerName="revoke-keys"
Jul 18 02:29:13.101124 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:13.101076    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="33b75464-69f0-46b7-836c-314f2b2a1d7e" containerName="revoke-keys"
Jul 18 02:29:13.101124 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:13.101094    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0eaafd0-36ef-40fa-8c21-1dd0bef59f0c" containerName="maas-api"
Jul 18 02:29:13.104443 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:13.104410    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-327441da-6484b7dc5c-8zrv8"
Jul 18 02:29:13.106955 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:13.106928    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-327441da-serving-cert\""
Jul 18 02:29:13.116218 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:13.116194    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-327441da-6484b7dc5c-8zrv8"]
Jul 18 02:29:13.154923 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:13.154886    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdbb4e74-0178-44d9-be48-4497359d6d67-maas-api-tls\") pod \"maas-api-e2e-mt-327441da-6484b7dc5c-8zrv8\" (UID: \"cdbb4e74-0178-44d9-be48-4497359d6d67\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-327441da-6484b7dc5c-8zrv8"
Jul 18 02:29:13.155097 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:13.154952    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdbb4e74-0178-44d9-be48-4497359d6d67-openshift-service-ca\") pod \"maas-api-e2e-mt-327441da-6484b7dc5c-8zrv8\" (UID: \"cdbb4e74-0178-44d9-be48-4497359d6d67\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-327441da-6484b7dc5c-8zrv8"
Jul 18 02:29:13.155097 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:13.155033    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9xnz\" (UniqueName: \"kubernetes.io/projected/cdbb4e74-0178-44d9-be48-4497359d6d67-kube-api-access-q9xnz\") pod \"maas-api-e2e-mt-327441da-6484b7dc5c-8zrv8\" (UID: \"cdbb4e74-0178-44d9-be48-4497359d6d67\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-327441da-6484b7dc5c-8zrv8"
Jul 18 02:29:13.255872 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:13.255828    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdbb4e74-0178-44d9-be48-4497359d6d67-maas-api-tls\") pod \"maas-api-e2e-mt-327441da-6484b7dc5c-8zrv8\" (UID: \"cdbb4e74-0178-44d9-be48-4497359d6d67\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-327441da-6484b7dc5c-8zrv8"
Jul 18 02:29:13.256051 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:13.255891    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdbb4e74-0178-44d9-be48-4497359d6d67-openshift-service-ca\") pod \"maas-api-e2e-mt-327441da-6484b7dc5c-8zrv8\" (UID: \"cdbb4e74-0178-44d9-be48-4497359d6d67\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-327441da-6484b7dc5c-8zrv8"
Jul 18 02:29:13.256051 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:13.255927    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9xnz\" (UniqueName: \"kubernetes.io/projected/cdbb4e74-0178-44d9-be48-4497359d6d67-kube-api-access-q9xnz\") pod \"maas-api-e2e-mt-327441da-6484b7dc5c-8zrv8\" (UID: \"cdbb4e74-0178-44d9-be48-4497359d6d67\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-327441da-6484b7dc5c-8zrv8"
Jul 18 02:29:13.256051 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:29:13.255986    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-327441da-serving-cert: secret "maas-api-e2e-mt-327441da-serving-cert" not found
Jul 18 02:29:13.256250 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:29:13.256055    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdbb4e74-0178-44d9-be48-4497359d6d67-maas-api-tls podName:cdbb4e74-0178-44d9-be48-4497359d6d67 nodeName:}" failed. No retries permitted until 2026-07-18 02:29:13.756037405 +0000 UTC m=+1948.381074067 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cdbb4e74-0178-44d9-be48-4497359d6d67-maas-api-tls") pod "maas-api-e2e-mt-327441da-6484b7dc5c-8zrv8" (UID: "cdbb4e74-0178-44d9-be48-4497359d6d67") : secret "maas-api-e2e-mt-327441da-serving-cert" not found
Jul 18 02:29:13.256664 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:13.256641    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdbb4e74-0178-44d9-be48-4497359d6d67-openshift-service-ca\") pod \"maas-api-e2e-mt-327441da-6484b7dc5c-8zrv8\" (UID: \"cdbb4e74-0178-44d9-be48-4497359d6d67\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-327441da-6484b7dc5c-8zrv8"
Jul 18 02:29:13.265290 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:13.265262    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9xnz\" (UniqueName: \"kubernetes.io/projected/cdbb4e74-0178-44d9-be48-4497359d6d67-kube-api-access-q9xnz\") pod \"maas-api-e2e-mt-327441da-6484b7dc5c-8zrv8\" (UID: \"cdbb4e74-0178-44d9-be48-4497359d6d67\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-327441da-6484b7dc5c-8zrv8"
Jul 18 02:29:13.760224 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:13.760190    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdbb4e74-0178-44d9-be48-4497359d6d67-maas-api-tls\") pod \"maas-api-e2e-mt-327441da-6484b7dc5c-8zrv8\" (UID: \"cdbb4e74-0178-44d9-be48-4497359d6d67\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-327441da-6484b7dc5c-8zrv8"
Jul 18 02:29:13.762802 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:13.762776    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdbb4e74-0178-44d9-be48-4497359d6d67-maas-api-tls\") pod \"maas-api-e2e-mt-327441da-6484b7dc5c-8zrv8\" (UID: \"cdbb4e74-0178-44d9-be48-4497359d6d67\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-327441da-6484b7dc5c-8zrv8"
Jul 18 02:29:14.015750 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:14.015661    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-327441da-6484b7dc5c-8zrv8"
Jul 18 02:29:14.158583 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:14.158556    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-327441da-6484b7dc5c-8zrv8"]
Jul 18 02:29:14.161600 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:29:14.161568    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdbb4e74_0178_44d9_be48_4497359d6d67.slice/crio-9aeb9ebe9b975dcd926120a110b47c5db92ec92d60a4194499b6ea65c0448069 WatchSource:0}: Error finding container 9aeb9ebe9b975dcd926120a110b47c5db92ec92d60a4194499b6ea65c0448069: Status 404 returned error can't find the container with id 9aeb9ebe9b975dcd926120a110b47c5db92ec92d60a4194499b6ea65c0448069
Jul 18 02:29:14.162939 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:14.162921    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 02:29:14.719742 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:14.719653    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-327441da-6484b7dc5c-8zrv8" event={"ID":"cdbb4e74-0178-44d9-be48-4497359d6d67","Type":"ContainerStarted","Data":"35616a6d46f198afd20237b9e3f67e28558183039e6887d8c032323622da9ca4"}
Jul 18 02:29:14.719742 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:14.719707    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-327441da-6484b7dc5c-8zrv8" event={"ID":"cdbb4e74-0178-44d9-be48-4497359d6d67","Type":"ContainerStarted","Data":"9aeb9ebe9b975dcd926120a110b47c5db92ec92d60a4194499b6ea65c0448069"}
Jul 18 02:29:14.719934 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:14.719769    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-327441da-6484b7dc5c-8zrv8"
Jul 18 02:29:14.736407 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:14.736346    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-327441da-6484b7dc5c-8zrv8" podStartSLOduration=1.437029434 podStartE2EDuration="1.736327326s" totalImagesPullingTime="299.297892ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:29:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:29:14.73544903 +0000 UTC m=+1949.360485688" watchObservedRunningTime="2026-07-18 02:29:14.736327326 +0000 UTC m=+1949.361363993"
Jul 18 02:29:20.730154 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:20.730124    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-327441da-6484b7dc5c-8zrv8"
Jul 18 02:29:56.228946 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:56.228870    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-327441da-5cp9b"]
Jul 18 02:29:56.232345 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:56.232326    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-327441da-5cp9b"
Jul 18 02:29:56.235340 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:56.235320    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g69gp\""
Jul 18 02:29:56.240505 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:56.240479    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-327441da-5cp9b"]
Jul 18 02:29:56.347928 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:56.347887    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3a6b993-e54c-4960-a65f-c947a86ead8f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-327441da-5cp9b\" (UID: \"c3a6b993-e54c-4960-a65f-c947a86ead8f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-327441da-5cp9b"
Jul 18 02:29:56.449198 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:56.449158    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3a6b993-e54c-4960-a65f-c947a86ead8f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-327441da-5cp9b\" (UID: \"c3a6b993-e54c-4960-a65f-c947a86ead8f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-327441da-5cp9b"
Jul 18 02:29:56.449931 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:56.449910    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3a6b993-e54c-4960-a65f-c947a86ead8f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-327441da-5cp9b\" (UID: \"c3a6b993-e54c-4960-a65f-c947a86ead8f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-327441da-5cp9b"
Jul 18 02:29:56.544261 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:56.544172    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-327441da-5cp9b"
Jul 18 02:29:56.682488 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:56.682335    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-327441da-5cp9b"]
Jul 18 02:29:56.685061 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:29:56.685031    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3a6b993_e54c_4960_a65f_c947a86ead8f.slice/crio-43eaef4b715228849f2db869e5cccb505ce14aee5f104b99b6e573915a6238f6 WatchSource:0}: Error finding container 43eaef4b715228849f2db869e5cccb505ce14aee5f104b99b6e573915a6238f6: Status 404 returned error can't find the container with id 43eaef4b715228849f2db869e5cccb505ce14aee5f104b99b6e573915a6238f6
Jul 18 02:29:56.956151 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:56.956116    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-327441da-5cp9b" event={"ID":"c3a6b993-e54c-4960-a65f-c947a86ead8f","Type":"ContainerStarted","Data":"43eaef4b715228849f2db869e5cccb505ce14aee5f104b99b6e573915a6238f6"}
Jul 18 02:29:57.962676 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:57.962641    2576 generic.go:356] "Generic (PLEG): container finished" podID="c3a6b993-e54c-4960-a65f-c947a86ead8f" containerID="d0b6341dece90a80d21138f4bb806ed2cec50b8ebbc548efcd0a81f896d892f3" exitCode=0
Jul 18 02:29:57.968694 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:57.968658    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-327441da-5cp9b" event={"ID":"c3a6b993-e54c-4960-a65f-c947a86ead8f","Type":"ContainerDied","Data":"d0b6341dece90a80d21138f4bb806ed2cec50b8ebbc548efcd0a81f896d892f3"}
Jul 18 02:29:59.104076 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:59.104047    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-327441da-5cp9b"
Jul 18 02:29:59.169585 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:59.169557    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3a6b993-e54c-4960-a65f-c947a86ead8f-maas-api-service-ca\") pod \"c3a6b993-e54c-4960-a65f-c947a86ead8f\" (UID: \"c3a6b993-e54c-4960-a65f-c947a86ead8f\") "
Jul 18 02:29:59.169937 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:59.169914    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3a6b993-e54c-4960-a65f-c947a86ead8f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c3a6b993-e54c-4960-a65f-c947a86ead8f" (UID: "c3a6b993-e54c-4960-a65f-c947a86ead8f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:29:59.270369 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:59.270274    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3a6b993-e54c-4960-a65f-c947a86ead8f-maas-api-service-ca\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:29:59.975551 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:59.975519    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-327441da-5cp9b"
Jul 18 02:29:59.975755 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:59.975575    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-327441da-5cp9b" event={"ID":"c3a6b993-e54c-4960-a65f-c947a86ead8f","Type":"ContainerDied","Data":"43eaef4b715228849f2db869e5cccb505ce14aee5f104b99b6e573915a6238f6"}
Jul 18 02:29:59.975755 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:29:59.975630    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="43eaef4b715228849f2db869e5cccb505ce14aee5f104b99b6e573915a6238f6"
Jul 18 02:30:00.131872 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:00.131836    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-327441da-29739030-4m2fg"]
Jul 18 02:30:00.132403 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:00.132382    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3a6b993-e54c-4960-a65f-c947a86ead8f" containerName="revoke-keys"
Jul 18 02:30:00.132553 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:00.132405    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3a6b993-e54c-4960-a65f-c947a86ead8f" containerName="revoke-keys"
Jul 18 02:30:00.132553 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:00.132544    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3a6b993-e54c-4960-a65f-c947a86ead8f" containerName="revoke-keys"
Jul 18 02:30:00.135741 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:00.135718    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29739030-vfp6h"]
Jul 18 02:30:00.135859 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:00.135824    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-327441da-29739030-4m2fg"
Jul 18 02:30:00.138057 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:00.138036    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g69gp\""
Jul 18 02:30:00.138916 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:00.138899    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739030-vfp6h"
Jul 18 02:30:00.144618 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:00.143971    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-327441da-29739030-4m2fg"]
Jul 18 02:30:00.146655 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:00.146632    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29739030-vfp6h"]
Jul 18 02:30:00.148627 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:00.148609    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-327441da-29739030-4m2fg"
Jul 18 02:30:00.158148 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:00.158128    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739030-vfp6h"
Jul 18 02:30:00.308688 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:00.308658    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-327441da-29739030-4m2fg"]
Jul 18 02:30:00.312232 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:30:00.311509    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4786fd73_5d2c_4ed1_a501_04cf822ecab1.slice/crio-69837487dd54202144285eea2ca9fd12fe034762440122cec13e08fff3ae194b WatchSource:0}: Error finding container 69837487dd54202144285eea2ca9fd12fe034762440122cec13e08fff3ae194b: Status 404 returned error can't find the container with id 69837487dd54202144285eea2ca9fd12fe034762440122cec13e08fff3ae194b
Jul 18 02:30:00.340174 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:00.340138    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29739030-vfp6h"]
Jul 18 02:30:00.983209 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:00.983171    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739030-vfp6h" event={"ID":"22302086-7e30-4a77-b188-0da393608dd5","Type":"ContainerStarted","Data":"294e6640e0f5468b85900bb5050ab9a9613ea3c819d77d2e586f1350bc48cb35"}
Jul 18 02:30:00.983385 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:00.983241    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739030-vfp6h" event={"ID":"22302086-7e30-4a77-b188-0da393608dd5","Type":"ContainerStarted","Data":"2ada9c74d4e61a3932815d4e30d41c72cea2537395fdda922b383930d87fb09d"}
Jul 18 02:30:00.984770 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:00.984740    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-327441da-29739030-4m2fg" event={"ID":"4786fd73-5d2c-4ed1-a501-04cf822ecab1","Type":"ContainerStarted","Data":"ee50688b7fc931d16863170db7b682de962278a1464862bb32eaa213c757bc80"}
Jul 18 02:30:00.984876 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:00.984794    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-327441da-29739030-4m2fg" event={"ID":"4786fd73-5d2c-4ed1-a501-04cf822ecab1","Type":"ContainerStarted","Data":"69837487dd54202144285eea2ca9fd12fe034762440122cec13e08fff3ae194b"}
Jul 18 02:30:01.000863 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:01.000817    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739030-vfp6h" podStartSLOduration=1.000801474 podStartE2EDuration="1.000801474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:30:00.999031163 +0000 UTC m=+1995.624067829" watchObservedRunningTime="2026-07-18 02:30:01.000801474 +0000 UTC m=+1995.625838136"
Jul 18 02:30:01.016861 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:01.016812    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-327441da-29739030-4m2fg" podStartSLOduration=1.016798131 podStartE2EDuration="1.016798131s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:30:01.015534435 +0000 UTC m=+1995.640571103" watchObservedRunningTime="2026-07-18 02:30:01.016798131 +0000 UTC m=+1995.641834817"
Jul 18 02:30:01.991413 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:01.991377    2576 generic.go:356] "Generic (PLEG): container finished" podID="22302086-7e30-4a77-b188-0da393608dd5" containerID="294e6640e0f5468b85900bb5050ab9a9613ea3c819d77d2e586f1350bc48cb35" exitCode=0
Jul 18 02:30:01.991839 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:01.991453    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739030-vfp6h" event={"ID":"22302086-7e30-4a77-b188-0da393608dd5","Type":"ContainerDied","Data":"294e6640e0f5468b85900bb5050ab9a9613ea3c819d77d2e586f1350bc48cb35"}
Jul 18 02:30:01.992976 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:01.992949    2576 generic.go:356] "Generic (PLEG): container finished" podID="4786fd73-5d2c-4ed1-a501-04cf822ecab1" containerID="ee50688b7fc931d16863170db7b682de962278a1464862bb32eaa213c757bc80" exitCode=0
Jul 18 02:30:01.993099 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:01.993025    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-327441da-29739030-4m2fg" event={"ID":"4786fd73-5d2c-4ed1-a501-04cf822ecab1","Type":"ContainerDied","Data":"ee50688b7fc931d16863170db7b682de962278a1464862bb32eaa213c757bc80"}
Jul 18 02:30:03.142571 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:03.142544    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-327441da-29739030-4m2fg"
Jul 18 02:30:03.170992 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:03.170960    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739030-vfp6h"
Jul 18 02:30:04.004340 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:04.004312    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739030-vfp6h"
Jul 18 02:30:04.004340 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:04.004318    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739030-vfp6h" event={"ID":"22302086-7e30-4a77-b188-0da393608dd5","Type":"ContainerDied","Data":"2ada9c74d4e61a3932815d4e30d41c72cea2537395fdda922b383930d87fb09d"}
Jul 18 02:30:04.004592 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:04.004369    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2ada9c74d4e61a3932815d4e30d41c72cea2537395fdda922b383930d87fb09d"
Jul 18 02:30:04.005569 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:04.005548    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-327441da-29739030-4m2fg"
Jul 18 02:30:04.005678 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:04.005552    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-327441da-29739030-4m2fg" event={"ID":"4786fd73-5d2c-4ed1-a501-04cf822ecab1","Type":"ContainerDied","Data":"69837487dd54202144285eea2ca9fd12fe034762440122cec13e08fff3ae194b"}
Jul 18 02:30:04.005678 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:04.005666    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="69837487dd54202144285eea2ca9fd12fe034762440122cec13e08fff3ae194b"
Jul 18 02:30:11.742877 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:11.742838    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-327441da-6484b7dc5c-8zrv8"]
Jul 18 02:30:11.743349 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:11.743167    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-327441da-6484b7dc5c-8zrv8" podUID="cdbb4e74-0178-44d9-be48-4497359d6d67" containerName="maas-api" containerID="cri-o://35616a6d46f198afd20237b9e3f67e28558183039e6887d8c032323622da9ca4" gracePeriod=30
Jul 18 02:30:11.774289 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:11.774252    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-327441da-29739030-4m2fg"]
Jul 18 02:30:11.778104 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:11.778075    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-327441da-29739030-4m2fg"]
Jul 18 02:30:11.970125 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:11.970092    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4786fd73-5d2c-4ed1-a501-04cf822ecab1" path="/var/lib/kubelet/pods/4786fd73-5d2c-4ed1-a501-04cf822ecab1/volumes"
Jul 18 02:30:12.002257 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:12.002186    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-327441da-6484b7dc5c-8zrv8"
Jul 18 02:30:12.050382 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:12.050341    2576 generic.go:356] "Generic (PLEG): container finished" podID="cdbb4e74-0178-44d9-be48-4497359d6d67" containerID="35616a6d46f198afd20237b9e3f67e28558183039e6887d8c032323622da9ca4" exitCode=0
Jul 18 02:30:12.050590 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:12.050395    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-327441da-6484b7dc5c-8zrv8"
Jul 18 02:30:12.050590 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:12.050415    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-327441da-6484b7dc5c-8zrv8" event={"ID":"cdbb4e74-0178-44d9-be48-4497359d6d67","Type":"ContainerDied","Data":"35616a6d46f198afd20237b9e3f67e28558183039e6887d8c032323622da9ca4"}
Jul 18 02:30:12.050590 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:12.050494    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-327441da-6484b7dc5c-8zrv8" event={"ID":"cdbb4e74-0178-44d9-be48-4497359d6d67","Type":"ContainerDied","Data":"9aeb9ebe9b975dcd926120a110b47c5db92ec92d60a4194499b6ea65c0448069"}
Jul 18 02:30:12.050590 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:12.050518    2576 scope.go:117] "RemoveContainer" containerID="35616a6d46f198afd20237b9e3f67e28558183039e6887d8c032323622da9ca4"
Jul 18 02:30:12.059545 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:12.059525    2576 scope.go:117] "RemoveContainer" containerID="35616a6d46f198afd20237b9e3f67e28558183039e6887d8c032323622da9ca4"
Jul 18 02:30:12.059820 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:30:12.059801    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35616a6d46f198afd20237b9e3f67e28558183039e6887d8c032323622da9ca4\": container with ID starting with 35616a6d46f198afd20237b9e3f67e28558183039e6887d8c032323622da9ca4 not found: ID does not exist" containerID="35616a6d46f198afd20237b9e3f67e28558183039e6887d8c032323622da9ca4"
Jul 18 02:30:12.059897 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:12.059827    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35616a6d46f198afd20237b9e3f67e28558183039e6887d8c032323622da9ca4"} err="failed to get container status \"35616a6d46f198afd20237b9e3f67e28558183039e6887d8c032323622da9ca4\": rpc error: code = NotFound desc = could not find container \"35616a6d46f198afd20237b9e3f67e28558183039e6887d8c032323622da9ca4\": container with ID starting with 35616a6d46f198afd20237b9e3f67e28558183039e6887d8c032323622da9ca4 not found: ID does not exist"
Jul 18 02:30:12.083170 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:12.083139    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9xnz\" (UniqueName: \"kubernetes.io/projected/cdbb4e74-0178-44d9-be48-4497359d6d67-kube-api-access-q9xnz\") pod \"cdbb4e74-0178-44d9-be48-4497359d6d67\" (UID: \"cdbb4e74-0178-44d9-be48-4497359d6d67\") "
Jul 18 02:30:12.083322 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:12.083237    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdbb4e74-0178-44d9-be48-4497359d6d67-maas-api-tls\") pod \"cdbb4e74-0178-44d9-be48-4497359d6d67\" (UID: \"cdbb4e74-0178-44d9-be48-4497359d6d67\") "
Jul 18 02:30:12.083322 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:12.083258    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdbb4e74-0178-44d9-be48-4497359d6d67-openshift-service-ca\") pod \"cdbb4e74-0178-44d9-be48-4497359d6d67\" (UID: \"cdbb4e74-0178-44d9-be48-4497359d6d67\") "
Jul 18 02:30:12.083767 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:12.083729    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdbb4e74-0178-44d9-be48-4497359d6d67-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cdbb4e74-0178-44d9-be48-4497359d6d67" (UID: "cdbb4e74-0178-44d9-be48-4497359d6d67"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:30:12.085830 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:12.085802    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdbb4e74-0178-44d9-be48-4497359d6d67-kube-api-access-q9xnz" (OuterVolumeSpecName: "kube-api-access-q9xnz") pod "cdbb4e74-0178-44d9-be48-4497359d6d67" (UID: "cdbb4e74-0178-44d9-be48-4497359d6d67"). InnerVolumeSpecName "kube-api-access-q9xnz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:30:12.085830 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:12.085805    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdbb4e74-0178-44d9-be48-4497359d6d67-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cdbb4e74-0178-44d9-be48-4497359d6d67" (UID: "cdbb4e74-0178-44d9-be48-4497359d6d67"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:30:12.184964 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:12.184922    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdbb4e74-0178-44d9-be48-4497359d6d67-maas-api-tls\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:30:12.184964 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:12.184959    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdbb4e74-0178-44d9-be48-4497359d6d67-openshift-service-ca\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:30:12.184964 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:12.184972    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9xnz\" (UniqueName: \"kubernetes.io/projected/cdbb4e74-0178-44d9-be48-4497359d6d67-kube-api-access-q9xnz\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:30:12.373781 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:12.373748    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-327441da-6484b7dc5c-8zrv8"]
Jul 18 02:30:12.381053 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:12.381026    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-327441da-6484b7dc5c-8zrv8"]
Jul 18 02:30:13.970674 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:13.970627    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdbb4e74-0178-44d9-be48-4497359d6d67" path="/var/lib/kubelet/pods/cdbb4e74-0178-44d9-be48-4497359d6d67/volumes"
Jul 18 02:30:16.326858 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:16.326819    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-327441da-5cp9b"]
Jul 18 02:30:16.329714 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:16.329686    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-327441da-5cp9b"]
Jul 18 02:30:17.969604 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:17.969570    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3a6b993-e54c-4960-a65f-c947a86ead8f" path="/var/lib/kubelet/pods/c3a6b993-e54c-4960-a65f-c947a86ead8f/volumes"
Jul 18 02:30:46.070097 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:30:46.070066    2576 scope.go:117] "RemoveContainer" containerID="24db48338db9354f14f06fe5b9816dfa947c9dfb6fcb0ce8e8aa3a85e82de30c"
Jul 18 02:31:45.982673 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:31:45.982642    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-bfrdg_9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777/console-operator/1.log"
Jul 18 02:31:45.986906 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:31:45.986880    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-bfrdg_9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777/console-operator/1.log"
Jul 18 02:31:46.097343 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:31:46.097311    2576 scope.go:117] "RemoveContainer" containerID="0e664bca8f60ca0c1e25e226b5c27efe49de34a4b0cfe4c7bb216b7159372d24"
Jul 18 02:31:46.105796 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:31:46.105769    2576 scope.go:117] "RemoveContainer" containerID="5db7bf0174a71b81da4d3561c20246e92d17051f6d23f90f9234f9ee2be723ce"
Jul 18 02:32:16.505376 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.505334    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"]
Jul 18 02:32:16.505907 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.505731    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4786fd73-5d2c-4ed1-a501-04cf822ecab1" containerName="cleanup"
Jul 18 02:32:16.505907 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.505744    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4786fd73-5d2c-4ed1-a501-04cf822ecab1" containerName="cleanup"
Jul 18 02:32:16.505907 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.505757    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdbb4e74-0178-44d9-be48-4497359d6d67" containerName="maas-api"
Jul 18 02:32:16.505907 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.505763    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdbb4e74-0178-44d9-be48-4497359d6d67" containerName="maas-api"
Jul 18 02:32:16.505907 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.505772    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22302086-7e30-4a77-b188-0da393608dd5" containerName="cleanup"
Jul 18 02:32:16.505907 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.505778    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="22302086-7e30-4a77-b188-0da393608dd5" containerName="cleanup"
Jul 18 02:32:16.505907 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.505883    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4786fd73-5d2c-4ed1-a501-04cf822ecab1" containerName="cleanup"
Jul 18 02:32:16.505907 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.505893    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdbb4e74-0178-44d9-be48-4497359d6d67" containerName="maas-api"
Jul 18 02:32:16.505907 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.505904    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="22302086-7e30-4a77-b188-0da393608dd5" containerName="cleanup"
Jul 18 02:32:16.509100 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.509080    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"
Jul 18 02:32:16.512072 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.512048    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-9e2f67-openshift-default-dockercfg-fsbj4\""
Jul 18 02:32:16.534684 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.534649    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"]
Jul 18 02:32:16.682700 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.682667    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23b913aa-93c9-41f1-8f29-feb27620f34e-istio-data\") pod \"e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd\" (UID: \"23b913aa-93c9-41f1-8f29-feb27620f34e\") " pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"
Jul 18 02:32:16.682700 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.682703    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7f5p2\" (UniqueName: \"kubernetes.io/projected/23b913aa-93c9-41f1-8f29-feb27620f34e-kube-api-access-7f5p2\") pod \"e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd\" (UID: \"23b913aa-93c9-41f1-8f29-feb27620f34e\") " pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"
Jul 18 02:32:16.682993 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.682729    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23b913aa-93c9-41f1-8f29-feb27620f34e-workload-socket\") pod \"e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd\" (UID: \"23b913aa-93c9-41f1-8f29-feb27620f34e\") " pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"
Jul 18 02:32:16.682993 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.682825    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23b913aa-93c9-41f1-8f29-feb27620f34e-istiod-ca-cert\") pod \"e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd\" (UID: \"23b913aa-93c9-41f1-8f29-feb27620f34e\") " pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"
Jul 18 02:32:16.682993 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.682898    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23b913aa-93c9-41f1-8f29-feb27620f34e-istio-envoy\") pod \"e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd\" (UID: \"23b913aa-93c9-41f1-8f29-feb27620f34e\") " pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"
Jul 18 02:32:16.682993 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.682931    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23b913aa-93c9-41f1-8f29-feb27620f34e-istio-token\") pod \"e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd\" (UID: \"23b913aa-93c9-41f1-8f29-feb27620f34e\") " pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"
Jul 18 02:32:16.682993 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.682959    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23b913aa-93c9-41f1-8f29-feb27620f34e-credential-socket\") pod \"e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd\" (UID: \"23b913aa-93c9-41f1-8f29-feb27620f34e\") " pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"
Jul 18 02:32:16.683190 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.683025    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23b913aa-93c9-41f1-8f29-feb27620f34e-workload-certs\") pod \"e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd\" (UID: \"23b913aa-93c9-41f1-8f29-feb27620f34e\") " pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"
Jul 18 02:32:16.683190 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.683099    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23b913aa-93c9-41f1-8f29-feb27620f34e-istio-podinfo\") pod \"e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd\" (UID: \"23b913aa-93c9-41f1-8f29-feb27620f34e\") " pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"
Jul 18 02:32:16.783774 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.783688    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23b913aa-93c9-41f1-8f29-feb27620f34e-istio-podinfo\") pod \"e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd\" (UID: \"23b913aa-93c9-41f1-8f29-feb27620f34e\") " pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"
Jul 18 02:32:16.783774 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.783734    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23b913aa-93c9-41f1-8f29-feb27620f34e-istio-data\") pod \"e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd\" (UID: \"23b913aa-93c9-41f1-8f29-feb27620f34e\") " pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"
Jul 18 02:32:16.783774 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.783758    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7f5p2\" (UniqueName: \"kubernetes.io/projected/23b913aa-93c9-41f1-8f29-feb27620f34e-kube-api-access-7f5p2\") pod \"e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd\" (UID: \"23b913aa-93c9-41f1-8f29-feb27620f34e\") " pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"
Jul 18 02:32:16.784068 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.783784    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23b913aa-93c9-41f1-8f29-feb27620f34e-workload-socket\") pod \"e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd\" (UID: \"23b913aa-93c9-41f1-8f29-feb27620f34e\") " pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"
Jul 18 02:32:16.784068 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.783820    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23b913aa-93c9-41f1-8f29-feb27620f34e-istiod-ca-cert\") pod \"e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd\" (UID: \"23b913aa-93c9-41f1-8f29-feb27620f34e\") " pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"
Jul 18 02:32:16.784068 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.783865    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23b913aa-93c9-41f1-8f29-feb27620f34e-istio-envoy\") pod \"e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd\" (UID: \"23b913aa-93c9-41f1-8f29-feb27620f34e\") " pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"
Jul 18 02:32:16.784068 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.783887    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23b913aa-93c9-41f1-8f29-feb27620f34e-istio-token\") pod \"e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd\" (UID: \"23b913aa-93c9-41f1-8f29-feb27620f34e\") " pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"
Jul 18 02:32:16.784068 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.783916    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23b913aa-93c9-41f1-8f29-feb27620f34e-credential-socket\") pod \"e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd\" (UID: \"23b913aa-93c9-41f1-8f29-feb27620f34e\") " pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"
Jul 18 02:32:16.784068 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.783962    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23b913aa-93c9-41f1-8f29-feb27620f34e-workload-certs\") pod \"e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd\" (UID: \"23b913aa-93c9-41f1-8f29-feb27620f34e\") " pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"
Jul 18 02:32:16.784418 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.784159    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23b913aa-93c9-41f1-8f29-feb27620f34e-istio-data\") pod \"e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd\" (UID: \"23b913aa-93c9-41f1-8f29-feb27620f34e\") " pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"
Jul 18 02:32:16.784418 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.784257    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23b913aa-93c9-41f1-8f29-feb27620f34e-workload-socket\") pod \"e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd\" (UID: \"23b913aa-93c9-41f1-8f29-feb27620f34e\") " pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"
Jul 18 02:32:16.784418 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.784336    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23b913aa-93c9-41f1-8f29-feb27620f34e-credential-socket\") pod \"e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd\" (UID: \"23b913aa-93c9-41f1-8f29-feb27620f34e\") " pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"
Jul 18 02:32:16.784625 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.784420    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23b913aa-93c9-41f1-8f29-feb27620f34e-workload-certs\") pod \"e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd\" (UID: \"23b913aa-93c9-41f1-8f29-feb27620f34e\") " pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"
Jul 18 02:32:16.784750 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.784730    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23b913aa-93c9-41f1-8f29-feb27620f34e-istiod-ca-cert\") pod \"e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd\" (UID: \"23b913aa-93c9-41f1-8f29-feb27620f34e\") " pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"
Jul 18 02:32:16.786142 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.786118    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23b913aa-93c9-41f1-8f29-feb27620f34e-istio-envoy\") pod \"e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd\" (UID: \"23b913aa-93c9-41f1-8f29-feb27620f34e\") " pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"
Jul 18 02:32:16.786271 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.786243    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23b913aa-93c9-41f1-8f29-feb27620f34e-istio-podinfo\") pod \"e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd\" (UID: \"23b913aa-93c9-41f1-8f29-feb27620f34e\") " pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"
Jul 18 02:32:16.792146 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.792122    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23b913aa-93c9-41f1-8f29-feb27620f34e-istio-token\") pod \"e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd\" (UID: \"23b913aa-93c9-41f1-8f29-feb27620f34e\") " pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"
Jul 18 02:32:16.792292 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.792272    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7f5p2\" (UniqueName: \"kubernetes.io/projected/23b913aa-93c9-41f1-8f29-feb27620f34e-kube-api-access-7f5p2\") pod \"e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd\" (UID: \"23b913aa-93c9-41f1-8f29-feb27620f34e\") " pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"
Jul 18 02:32:16.819030 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.819000    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"
Jul 18 02:32:16.963572 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:16.963546    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"]
Jul 18 02:32:16.965182 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:32:16.965155    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23b913aa_93c9_41f1_8f29_feb27620f34e.slice/crio-211b67d4edb93060290824aec9fc2265cfe12a377250f2bbefcc85386b99ddf9 WatchSource:0}: Error finding container 211b67d4edb93060290824aec9fc2265cfe12a377250f2bbefcc85386b99ddf9: Status 404 returned error can't find the container with id 211b67d4edb93060290824aec9fc2265cfe12a377250f2bbefcc85386b99ddf9
Jul 18 02:32:17.725685 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:17.725650    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd" event={"ID":"23b913aa-93c9-41f1-8f29-feb27620f34e","Type":"ContainerStarted","Data":"211b67d4edb93060290824aec9fc2265cfe12a377250f2bbefcc85386b99ddf9"}
Jul 18 02:32:19.641243 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:19.641208    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 18 02:32:19.641496 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:19.641278    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 18 02:32:19.641496 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:19.641310    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 18 02:32:19.739125 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:19.739041    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd" event={"ID":"23b913aa-93c9-41f1-8f29-feb27620f34e","Type":"ContainerStarted","Data":"ade644167048f2c4ccc683b9ff7028975f73c57dde8fc6ec414ea57b08d2b997"}
Jul 18 02:32:19.760878 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:19.760816    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd" podStartSLOduration=1.087187464 podStartE2EDuration="3.760794603s" totalImagesPullingTime="2.673607139s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:32:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:32:19.757882567 +0000 UTC m=+2134.382919236" watchObservedRunningTime="2026-07-18 02:32:19.760794603 +0000 UTC m=+2134.385831272"
Jul 18 02:32:19.819361 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:19.819323    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"
Jul 18 02:32:20.825255 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:20.825216    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"
Jul 18 02:32:21.750596 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:21.750563    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"
Jul 18 02:32:21.751486 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:21.751466    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"
Jul 18 02:32:21.778863 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:21.778826    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9e2f67-989df7548-vkwsc"]
Jul 18 02:32:21.783172 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:21.783148    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9e2f67-989df7548-vkwsc"
Jul 18 02:32:21.786496 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:21.786379    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-9e2f67-serving-cert\""
Jul 18 02:32:21.796690 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:21.796663    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9e2f67-989df7548-vkwsc"]
Jul 18 02:32:21.938295 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:21.938258    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqr2q\" (UniqueName: \"kubernetes.io/projected/ad5b3cc5-0294-4a4b-8ebf-4090d62322dd-kube-api-access-pqr2q\") pod \"maas-api-e2e-inf-a-9e2f67-989df7548-vkwsc\" (UID: \"ad5b3cc5-0294-4a4b-8ebf-4090d62322dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9e2f67-989df7548-vkwsc"
Jul 18 02:32:21.938796 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:21.938324    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad5b3cc5-0294-4a4b-8ebf-4090d62322dd-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9e2f67-989df7548-vkwsc\" (UID: \"ad5b3cc5-0294-4a4b-8ebf-4090d62322dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9e2f67-989df7548-vkwsc"
Jul 18 02:32:21.938796 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:21.938473    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad5b3cc5-0294-4a4b-8ebf-4090d62322dd-maas-api-tls\") pod \"maas-api-e2e-inf-a-9e2f67-989df7548-vkwsc\" (UID: \"ad5b3cc5-0294-4a4b-8ebf-4090d62322dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9e2f67-989df7548-vkwsc"
Jul 18 02:32:22.039779 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:22.039701    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqr2q\" (UniqueName: \"kubernetes.io/projected/ad5b3cc5-0294-4a4b-8ebf-4090d62322dd-kube-api-access-pqr2q\") pod \"maas-api-e2e-inf-a-9e2f67-989df7548-vkwsc\" (UID: \"ad5b3cc5-0294-4a4b-8ebf-4090d62322dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9e2f67-989df7548-vkwsc"
Jul 18 02:32:22.039779 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:22.039764    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad5b3cc5-0294-4a4b-8ebf-4090d62322dd-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9e2f67-989df7548-vkwsc\" (UID: \"ad5b3cc5-0294-4a4b-8ebf-4090d62322dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9e2f67-989df7548-vkwsc"
Jul 18 02:32:22.039982 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:22.039809    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad5b3cc5-0294-4a4b-8ebf-4090d62322dd-maas-api-tls\") pod \"maas-api-e2e-inf-a-9e2f67-989df7548-vkwsc\" (UID: \"ad5b3cc5-0294-4a4b-8ebf-4090d62322dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9e2f67-989df7548-vkwsc"
Jul 18 02:32:22.039982 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:32:22.039931    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-9e2f67-serving-cert: secret "maas-api-e2e-inf-a-9e2f67-serving-cert" not found
Jul 18 02:32:22.040071 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:32:22.040003    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad5b3cc5-0294-4a4b-8ebf-4090d62322dd-maas-api-tls podName:ad5b3cc5-0294-4a4b-8ebf-4090d62322dd nodeName:}" failed. No retries permitted until 2026-07-18 02:32:22.539982551 +0000 UTC m=+2137.165019219 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ad5b3cc5-0294-4a4b-8ebf-4090d62322dd-maas-api-tls") pod "maas-api-e2e-inf-a-9e2f67-989df7548-vkwsc" (UID: "ad5b3cc5-0294-4a4b-8ebf-4090d62322dd") : secret "maas-api-e2e-inf-a-9e2f67-serving-cert" not found
Jul 18 02:32:22.040623 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:22.040595    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad5b3cc5-0294-4a4b-8ebf-4090d62322dd-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9e2f67-989df7548-vkwsc\" (UID: \"ad5b3cc5-0294-4a4b-8ebf-4090d62322dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9e2f67-989df7548-vkwsc"
Jul 18 02:32:22.048306 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:22.048277    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqr2q\" (UniqueName: \"kubernetes.io/projected/ad5b3cc5-0294-4a4b-8ebf-4090d62322dd-kube-api-access-pqr2q\") pod \"maas-api-e2e-inf-a-9e2f67-989df7548-vkwsc\" (UID: \"ad5b3cc5-0294-4a4b-8ebf-4090d62322dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9e2f67-989df7548-vkwsc"
Jul 18 02:32:22.544042 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:22.544008    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad5b3cc5-0294-4a4b-8ebf-4090d62322dd-maas-api-tls\") pod \"maas-api-e2e-inf-a-9e2f67-989df7548-vkwsc\" (UID: \"ad5b3cc5-0294-4a4b-8ebf-4090d62322dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9e2f67-989df7548-vkwsc"
Jul 18 02:32:22.546365 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:22.546337    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad5b3cc5-0294-4a4b-8ebf-4090d62322dd-maas-api-tls\") pod \"maas-api-e2e-inf-a-9e2f67-989df7548-vkwsc\" (UID: \"ad5b3cc5-0294-4a4b-8ebf-4090d62322dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9e2f67-989df7548-vkwsc"
Jul 18 02:32:22.697824 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:22.697786    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9e2f67-989df7548-vkwsc"
Jul 18 02:32:22.842635 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:22.842597    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9e2f67-989df7548-vkwsc"]
Jul 18 02:32:22.843913 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:32:22.843883    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad5b3cc5_0294_4a4b_8ebf_4090d62322dd.slice/crio-ef3a3ba3eefb0625e0fba2a365371b6a3766ce2754cc3318731d95a8073f9f50 WatchSource:0}: Error finding container ef3a3ba3eefb0625e0fba2a365371b6a3766ce2754cc3318731d95a8073f9f50: Status 404 returned error can't find the container with id ef3a3ba3eefb0625e0fba2a365371b6a3766ce2754cc3318731d95a8073f9f50
Jul 18 02:32:23.764985 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:23.764945    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9e2f67-989df7548-vkwsc" event={"ID":"ad5b3cc5-0294-4a4b-8ebf-4090d62322dd","Type":"ContainerStarted","Data":"a3646a99635c102fbe0c34bbc0e9b84189034a4774ea5f42f61e3774f901fe7c"}
Jul 18 02:32:23.765474 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:23.765011    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9e2f67-989df7548-vkwsc" event={"ID":"ad5b3cc5-0294-4a4b-8ebf-4090d62322dd","Type":"ContainerStarted","Data":"ef3a3ba3eefb0625e0fba2a365371b6a3766ce2754cc3318731d95a8073f9f50"}
Jul 18 02:32:23.781809 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:23.781754    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9e2f67-989df7548-vkwsc" podStartSLOduration=2.299414013 podStartE2EDuration="2.781736731s" totalImagesPullingTime="482.322718ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:32:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:32:23.779866961 +0000 UTC m=+2138.404903621" watchObservedRunningTime="2026-07-18 02:32:23.781736731 +0000 UTC m=+2138.406773397"
Jul 18 02:32:24.771694 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:24.771661    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9e2f67-989df7548-vkwsc"
Jul 18 02:32:30.139633 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:30.139594    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a45ea2-8475b46474-jk4pg"]
Jul 18 02:32:30.144003 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:30.143978    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a45ea2-8475b46474-jk4pg"
Jul 18 02:32:30.146398 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:30.146373    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-a45ea2-serving-cert\""
Jul 18 02:32:30.153653 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:30.153625    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a45ea2-8475b46474-jk4pg"]
Jul 18 02:32:30.233833 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:30.233798    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d8f5c80-416b-4452-acca-74836f94162a-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a45ea2-8475b46474-jk4pg\" (UID: \"0d8f5c80-416b-4452-acca-74836f94162a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a45ea2-8475b46474-jk4pg"
Jul 18 02:32:30.234010 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:30.233874    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b54bl\" (UniqueName: \"kubernetes.io/projected/0d8f5c80-416b-4452-acca-74836f94162a-kube-api-access-b54bl\") pod \"maas-api-e2e-inf-b-a45ea2-8475b46474-jk4pg\" (UID: \"0d8f5c80-416b-4452-acca-74836f94162a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a45ea2-8475b46474-jk4pg"
Jul 18 02:32:30.234010 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:30.233900    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d8f5c80-416b-4452-acca-74836f94162a-maas-api-tls\") pod \"maas-api-e2e-inf-b-a45ea2-8475b46474-jk4pg\" (UID: \"0d8f5c80-416b-4452-acca-74836f94162a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a45ea2-8475b46474-jk4pg"
Jul 18 02:32:30.335272 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:30.335235    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d8f5c80-416b-4452-acca-74836f94162a-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a45ea2-8475b46474-jk4pg\" (UID: \"0d8f5c80-416b-4452-acca-74836f94162a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a45ea2-8475b46474-jk4pg"
Jul 18 02:32:30.335469 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:30.335305    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b54bl\" (UniqueName: \"kubernetes.io/projected/0d8f5c80-416b-4452-acca-74836f94162a-kube-api-access-b54bl\") pod \"maas-api-e2e-inf-b-a45ea2-8475b46474-jk4pg\" (UID: \"0d8f5c80-416b-4452-acca-74836f94162a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a45ea2-8475b46474-jk4pg"
Jul 18 02:32:30.335469 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:30.335327    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d8f5c80-416b-4452-acca-74836f94162a-maas-api-tls\") pod \"maas-api-e2e-inf-b-a45ea2-8475b46474-jk4pg\" (UID: \"0d8f5c80-416b-4452-acca-74836f94162a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a45ea2-8475b46474-jk4pg"
Jul 18 02:32:30.335885 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:30.335865    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d8f5c80-416b-4452-acca-74836f94162a-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a45ea2-8475b46474-jk4pg\" (UID: \"0d8f5c80-416b-4452-acca-74836f94162a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a45ea2-8475b46474-jk4pg"
Jul 18 02:32:30.337791 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:30.337768    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d8f5c80-416b-4452-acca-74836f94162a-maas-api-tls\") pod \"maas-api-e2e-inf-b-a45ea2-8475b46474-jk4pg\" (UID: \"0d8f5c80-416b-4452-acca-74836f94162a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a45ea2-8475b46474-jk4pg"
Jul 18 02:32:30.343402 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:30.343384    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b54bl\" (UniqueName: \"kubernetes.io/projected/0d8f5c80-416b-4452-acca-74836f94162a-kube-api-access-b54bl\") pod \"maas-api-e2e-inf-b-a45ea2-8475b46474-jk4pg\" (UID: \"0d8f5c80-416b-4452-acca-74836f94162a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a45ea2-8475b46474-jk4pg"
Jul 18 02:32:30.456624 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:30.456535    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a45ea2-8475b46474-jk4pg"
Jul 18 02:32:30.652941 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:30.652912    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a45ea2-8475b46474-jk4pg"]
Jul 18 02:32:30.655330 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:32:30.655303    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d8f5c80_416b_4452_acca_74836f94162a.slice/crio-c81781e91c8c29d5336035efb75eab099b132b4670aa5e7c03ba9d83e0d1fe58 WatchSource:0}: Error finding container c81781e91c8c29d5336035efb75eab099b132b4670aa5e7c03ba9d83e0d1fe58: Status 404 returned error can't find the container with id c81781e91c8c29d5336035efb75eab099b132b4670aa5e7c03ba9d83e0d1fe58
Jul 18 02:32:30.785157 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:30.785066    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9e2f67-989df7548-vkwsc"
Jul 18 02:32:30.824583 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:30.824544    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a45ea2-8475b46474-jk4pg" event={"ID":"0d8f5c80-416b-4452-acca-74836f94162a","Type":"ContainerStarted","Data":"c81781e91c8c29d5336035efb75eab099b132b4670aa5e7c03ba9d83e0d1fe58"}
Jul 18 02:32:31.832620 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:31.832579    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a45ea2-8475b46474-jk4pg" event={"ID":"0d8f5c80-416b-4452-acca-74836f94162a","Type":"ContainerStarted","Data":"31eb59e436aa12e9fcc3353253d1febeca94180b9f13078a4f9901f1e44d1c73"}
Jul 18 02:32:31.833064 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:31.832754    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a45ea2-8475b46474-jk4pg"
Jul 18 02:32:31.865067 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:31.865002    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a45ea2-8475b46474-jk4pg" podStartSLOduration=1.347013619 podStartE2EDuration="1.864978182s" totalImagesPullingTime="517.964563ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:32:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:32:31.860966402 +0000 UTC m=+2146.486003070" watchObservedRunningTime="2026-07-18 02:32:31.864978182 +0000 UTC m=+2146.490014916"
Jul 18 02:32:33.846674 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:33.846583    2576 generic.go:356] "Generic (PLEG): container finished" podID="23b913aa-93c9-41f1-8f29-feb27620f34e" containerID="ade644167048f2c4ccc683b9ff7028975f73c57dde8fc6ec414ea57b08d2b997" exitCode=0
Jul 18 02:32:33.846674 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:33.846659    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd" event={"ID":"23b913aa-93c9-41f1-8f29-feb27620f34e","Type":"ContainerDied","Data":"ade644167048f2c4ccc683b9ff7028975f73c57dde8fc6ec414ea57b08d2b997"}
Jul 18 02:32:33.847117 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:33.847101    2576 scope.go:117] "RemoveContainer" containerID="ade644167048f2c4ccc683b9ff7028975f73c57dde8fc6ec414ea57b08d2b997"
Jul 18 02:32:33.849059 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:33.849027    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 18 02:32:33.849132 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:33.849097    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 18 02:32:33.849169 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:33.849130    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 18 02:32:34.853718 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:34.853676    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd" event={"ID":"23b913aa-93c9-41f1-8f29-feb27620f34e","Type":"ContainerStarted","Data":"1b14168acb957429b792785ac844c7465bfe6e27001ff268015b84a68a47155e"}
Jul 18 02:32:35.820153 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:35.820106    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"
Jul 18 02:32:35.821415 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:35.821387    2576 patch_prober.go:28] interesting pod/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Jul 18 02:32:35.821545 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:35.821464    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd" podUID="23b913aa-93c9-41f1-8f29-feb27620f34e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Jul 18 02:32:36.751494 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:36.751457    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"
Jul 18 02:32:36.819630 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:36.819590    2576 patch_prober.go:28] interesting pod/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Jul 18 02:32:36.819820 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:36.819645    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd" podUID="23b913aa-93c9-41f1-8f29-feb27620f34e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Jul 18 02:32:37.820544 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:37.820505    2576 patch_prober.go:28] interesting pod/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Jul 18 02:32:37.820956 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:37.820579    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd" podUID="23b913aa-93c9-41f1-8f29-feb27620f34e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Jul 18 02:32:37.845089 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:37.845061    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a45ea2-8475b46474-jk4pg"
Jul 18 02:32:38.824287 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:38.824207    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"
Jul 18 02:32:38.825258 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:38.825234    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"
Jul 18 02:32:46.130656 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:32:46.130621    2576 scope.go:117] "RemoveContainer" containerID="bc40242cecdca9dece0c596189a6480e05480de7e688dc3cb6e9962f88cf51b3"
Jul 18 02:34:14.992528 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:14.992417    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9e2f67-989df7548-vkwsc"]
Jul 18 02:34:14.993097 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:14.992735    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9e2f67-989df7548-vkwsc" podUID="ad5b3cc5-0294-4a4b-8ebf-4090d62322dd" containerName="maas-api" containerID="cri-o://a3646a99635c102fbe0c34bbc0e9b84189034a4774ea5f42f61e3774f901fe7c" gracePeriod=30
Jul 18 02:34:15.252671 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:15.252609    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9e2f67-989df7548-vkwsc"
Jul 18 02:34:15.323774 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:15.323739    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad5b3cc5-0294-4a4b-8ebf-4090d62322dd-maas-api-tls\") pod \"ad5b3cc5-0294-4a4b-8ebf-4090d62322dd\" (UID: \"ad5b3cc5-0294-4a4b-8ebf-4090d62322dd\") "
Jul 18 02:34:15.323938 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:15.323887    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad5b3cc5-0294-4a4b-8ebf-4090d62322dd-openshift-service-ca\") pod \"ad5b3cc5-0294-4a4b-8ebf-4090d62322dd\" (UID: \"ad5b3cc5-0294-4a4b-8ebf-4090d62322dd\") "
Jul 18 02:34:15.323938 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:15.323919    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pqr2q\" (UniqueName: \"kubernetes.io/projected/ad5b3cc5-0294-4a4b-8ebf-4090d62322dd-kube-api-access-pqr2q\") pod \"ad5b3cc5-0294-4a4b-8ebf-4090d62322dd\" (UID: \"ad5b3cc5-0294-4a4b-8ebf-4090d62322dd\") "
Jul 18 02:34:15.324290 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:15.324263    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad5b3cc5-0294-4a4b-8ebf-4090d62322dd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ad5b3cc5-0294-4a4b-8ebf-4090d62322dd" (UID: "ad5b3cc5-0294-4a4b-8ebf-4090d62322dd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:34:15.325804 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:15.325774    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad5b3cc5-0294-4a4b-8ebf-4090d62322dd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ad5b3cc5-0294-4a4b-8ebf-4090d62322dd" (UID: "ad5b3cc5-0294-4a4b-8ebf-4090d62322dd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:34:15.325990 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:15.325974    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad5b3cc5-0294-4a4b-8ebf-4090d62322dd-kube-api-access-pqr2q" (OuterVolumeSpecName: "kube-api-access-pqr2q") pod "ad5b3cc5-0294-4a4b-8ebf-4090d62322dd" (UID: "ad5b3cc5-0294-4a4b-8ebf-4090d62322dd"). InnerVolumeSpecName "kube-api-access-pqr2q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:34:15.383638 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:15.383596    2576 generic.go:356] "Generic (PLEG): container finished" podID="ad5b3cc5-0294-4a4b-8ebf-4090d62322dd" containerID="a3646a99635c102fbe0c34bbc0e9b84189034a4774ea5f42f61e3774f901fe7c" exitCode=0
Jul 18 02:34:15.383638 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:15.383639    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9e2f67-989df7548-vkwsc"
Jul 18 02:34:15.383887 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:15.383640    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9e2f67-989df7548-vkwsc" event={"ID":"ad5b3cc5-0294-4a4b-8ebf-4090d62322dd","Type":"ContainerDied","Data":"a3646a99635c102fbe0c34bbc0e9b84189034a4774ea5f42f61e3774f901fe7c"}
Jul 18 02:34:15.383887 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:15.383771    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9e2f67-989df7548-vkwsc" event={"ID":"ad5b3cc5-0294-4a4b-8ebf-4090d62322dd","Type":"ContainerDied","Data":"ef3a3ba3eefb0625e0fba2a365371b6a3766ce2754cc3318731d95a8073f9f50"}
Jul 18 02:34:15.383887 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:15.383798    2576 scope.go:117] "RemoveContainer" containerID="a3646a99635c102fbe0c34bbc0e9b84189034a4774ea5f42f61e3774f901fe7c"
Jul 18 02:34:15.393108 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:15.393088    2576 scope.go:117] "RemoveContainer" containerID="a3646a99635c102fbe0c34bbc0e9b84189034a4774ea5f42f61e3774f901fe7c"
Jul 18 02:34:15.393352 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:34:15.393333    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3646a99635c102fbe0c34bbc0e9b84189034a4774ea5f42f61e3774f901fe7c\": container with ID starting with a3646a99635c102fbe0c34bbc0e9b84189034a4774ea5f42f61e3774f901fe7c not found: ID does not exist" containerID="a3646a99635c102fbe0c34bbc0e9b84189034a4774ea5f42f61e3774f901fe7c"
Jul 18 02:34:15.393399 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:15.393361    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3646a99635c102fbe0c34bbc0e9b84189034a4774ea5f42f61e3774f901fe7c"} err="failed to get container status \"a3646a99635c102fbe0c34bbc0e9b84189034a4774ea5f42f61e3774f901fe7c\": rpc error: code = NotFound desc = could not find container \"a3646a99635c102fbe0c34bbc0e9b84189034a4774ea5f42f61e3774f901fe7c\": container with ID starting with a3646a99635c102fbe0c34bbc0e9b84189034a4774ea5f42f61e3774f901fe7c not found: ID does not exist"
Jul 18 02:34:15.405619 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:15.405591    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9e2f67-989df7548-vkwsc"]
Jul 18 02:34:15.416263 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:15.411302    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9e2f67-989df7548-vkwsc"]
Jul 18 02:34:15.425114 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:15.425090    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad5b3cc5-0294-4a4b-8ebf-4090d62322dd-maas-api-tls\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:34:15.425253 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:15.425116    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad5b3cc5-0294-4a4b-8ebf-4090d62322dd-openshift-service-ca\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:34:15.425253 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:15.425126    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqr2q\" (UniqueName: \"kubernetes.io/projected/ad5b3cc5-0294-4a4b-8ebf-4090d62322dd-kube-api-access-pqr2q\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:34:15.970207 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:15.970167    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad5b3cc5-0294-4a4b-8ebf-4090d62322dd" path="/var/lib/kubelet/pods/ad5b3cc5-0294-4a4b-8ebf-4090d62322dd/volumes"
Jul 18 02:34:26.031513 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:26.031478    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"]
Jul 18 02:34:26.031954 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:26.031704    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd" podUID="23b913aa-93c9-41f1-8f29-feb27620f34e" containerName="istio-proxy" containerID="cri-o://1b14168acb957429b792785ac844c7465bfe6e27001ff268015b84a68a47155e" gracePeriod=30
Jul 18 02:34:31.277176 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.277152    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"
Jul 18 02:34:31.378265 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.378235    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23b913aa-93c9-41f1-8f29-feb27620f34e-workload-certs\") pod \"23b913aa-93c9-41f1-8f29-feb27620f34e\" (UID: \"23b913aa-93c9-41f1-8f29-feb27620f34e\") "
Jul 18 02:34:31.378415 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.378275    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23b913aa-93c9-41f1-8f29-feb27620f34e-istiod-ca-cert\") pod \"23b913aa-93c9-41f1-8f29-feb27620f34e\" (UID: \"23b913aa-93c9-41f1-8f29-feb27620f34e\") "
Jul 18 02:34:31.378415 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.378301    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23b913aa-93c9-41f1-8f29-feb27620f34e-credential-socket\") pod \"23b913aa-93c9-41f1-8f29-feb27620f34e\" (UID: \"23b913aa-93c9-41f1-8f29-feb27620f34e\") "
Jul 18 02:34:31.378523 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.378498    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23b913aa-93c9-41f1-8f29-feb27620f34e-istio-token\") pod \"23b913aa-93c9-41f1-8f29-feb27620f34e\" (UID: \"23b913aa-93c9-41f1-8f29-feb27620f34e\") "
Jul 18 02:34:31.378570 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.378544    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7f5p2\" (UniqueName: \"kubernetes.io/projected/23b913aa-93c9-41f1-8f29-feb27620f34e-kube-api-access-7f5p2\") pod \"23b913aa-93c9-41f1-8f29-feb27620f34e\" (UID: \"23b913aa-93c9-41f1-8f29-feb27620f34e\") "
Jul 18 02:34:31.378634 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.378578    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23b913aa-93c9-41f1-8f29-feb27620f34e-istio-envoy\") pod \"23b913aa-93c9-41f1-8f29-feb27620f34e\" (UID: \"23b913aa-93c9-41f1-8f29-feb27620f34e\") "
Jul 18 02:34:31.378634 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.378557    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23b913aa-93c9-41f1-8f29-feb27620f34e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "23b913aa-93c9-41f1-8f29-feb27620f34e" (UID: "23b913aa-93c9-41f1-8f29-feb27620f34e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:34:31.378744 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.378665    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23b913aa-93c9-41f1-8f29-feb27620f34e-istio-data\") pod \"23b913aa-93c9-41f1-8f29-feb27620f34e\" (UID: \"23b913aa-93c9-41f1-8f29-feb27620f34e\") "
Jul 18 02:34:31.378744 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.378699    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23b913aa-93c9-41f1-8f29-feb27620f34e-workload-socket\") pod \"23b913aa-93c9-41f1-8f29-feb27620f34e\" (UID: \"23b913aa-93c9-41f1-8f29-feb27620f34e\") "
Jul 18 02:34:31.378744 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.378665    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23b913aa-93c9-41f1-8f29-feb27620f34e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "23b913aa-93c9-41f1-8f29-feb27620f34e" (UID: "23b913aa-93c9-41f1-8f29-feb27620f34e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:34:31.378744 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.378731    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23b913aa-93c9-41f1-8f29-feb27620f34e-istio-podinfo\") pod \"23b913aa-93c9-41f1-8f29-feb27620f34e\" (UID: \"23b913aa-93c9-41f1-8f29-feb27620f34e\") "
Jul 18 02:34:31.378744 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.378707    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23b913aa-93c9-41f1-8f29-feb27620f34e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "23b913aa-93c9-41f1-8f29-feb27620f34e" (UID: "23b913aa-93c9-41f1-8f29-feb27620f34e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:34:31.379019 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.378994    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23b913aa-93c9-41f1-8f29-feb27620f34e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "23b913aa-93c9-41f1-8f29-feb27620f34e" (UID: "23b913aa-93c9-41f1-8f29-feb27620f34e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:34:31.379074 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.379059    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23b913aa-93c9-41f1-8f29-feb27620f34e-workload-certs\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:34:31.379074 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.379071    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23b913aa-93c9-41f1-8f29-feb27620f34e-istiod-ca-cert\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:34:31.379176 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.379084    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23b913aa-93c9-41f1-8f29-feb27620f34e-credential-socket\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:34:31.379176 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.379098    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23b913aa-93c9-41f1-8f29-feb27620f34e-workload-socket\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:34:31.379627 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.379605    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23b913aa-93c9-41f1-8f29-feb27620f34e-istio-data" (OuterVolumeSpecName: "istio-data") pod "23b913aa-93c9-41f1-8f29-feb27620f34e" (UID: "23b913aa-93c9-41f1-8f29-feb27620f34e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:34:31.380929 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.380899    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23b913aa-93c9-41f1-8f29-feb27620f34e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "23b913aa-93c9-41f1-8f29-feb27620f34e" (UID: "23b913aa-93c9-41f1-8f29-feb27620f34e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:34:31.381053 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.380954    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23b913aa-93c9-41f1-8f29-feb27620f34e-istio-token" (OuterVolumeSpecName: "istio-token") pod "23b913aa-93c9-41f1-8f29-feb27620f34e" (UID: "23b913aa-93c9-41f1-8f29-feb27620f34e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:34:31.381128 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.381087    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23b913aa-93c9-41f1-8f29-feb27620f34e-kube-api-access-7f5p2" (OuterVolumeSpecName: "kube-api-access-7f5p2") pod "23b913aa-93c9-41f1-8f29-feb27620f34e" (UID: "23b913aa-93c9-41f1-8f29-feb27620f34e"). InnerVolumeSpecName "kube-api-access-7f5p2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:34:31.381128 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.381101    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/23b913aa-93c9-41f1-8f29-feb27620f34e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "23b913aa-93c9-41f1-8f29-feb27620f34e" (UID: "23b913aa-93c9-41f1-8f29-feb27620f34e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 18 02:34:31.473077 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.473043    2576 generic.go:356] "Generic (PLEG): container finished" podID="23b913aa-93c9-41f1-8f29-feb27620f34e" containerID="1b14168acb957429b792785ac844c7465bfe6e27001ff268015b84a68a47155e" exitCode=0
Jul 18 02:34:31.473272 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.473088    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"
Jul 18 02:34:31.473272 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.473127    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd" event={"ID":"23b913aa-93c9-41f1-8f29-feb27620f34e","Type":"ContainerDied","Data":"1b14168acb957429b792785ac844c7465bfe6e27001ff268015b84a68a47155e"}
Jul 18 02:34:31.473272 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.473190    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd" event={"ID":"23b913aa-93c9-41f1-8f29-feb27620f34e","Type":"ContainerDied","Data":"211b67d4edb93060290824aec9fc2265cfe12a377250f2bbefcc85386b99ddf9"}
Jul 18 02:34:31.473272 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.473219    2576 scope.go:117] "RemoveContainer" containerID="1b14168acb957429b792785ac844c7465bfe6e27001ff268015b84a68a47155e"
Jul 18 02:34:31.479831 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.479807    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23b913aa-93c9-41f1-8f29-feb27620f34e-istio-token\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:34:31.479936 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.479835    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7f5p2\" (UniqueName: \"kubernetes.io/projected/23b913aa-93c9-41f1-8f29-feb27620f34e-kube-api-access-7f5p2\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:34:31.479936 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.479850    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23b913aa-93c9-41f1-8f29-feb27620f34e-istio-envoy\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:34:31.479936 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.479865    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23b913aa-93c9-41f1-8f29-feb27620f34e-istio-data\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:34:31.479936 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.479879    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23b913aa-93c9-41f1-8f29-feb27620f34e-istio-podinfo\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:34:31.482951 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.482935    2576 scope.go:117] "RemoveContainer" containerID="ade644167048f2c4ccc683b9ff7028975f73c57dde8fc6ec414ea57b08d2b997"
Jul 18 02:34:31.490926 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.490910    2576 scope.go:117] "RemoveContainer" containerID="1b14168acb957429b792785ac844c7465bfe6e27001ff268015b84a68a47155e"
Jul 18 02:34:31.491166 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:34:31.491145    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b14168acb957429b792785ac844c7465bfe6e27001ff268015b84a68a47155e\": container with ID starting with 1b14168acb957429b792785ac844c7465bfe6e27001ff268015b84a68a47155e not found: ID does not exist" containerID="1b14168acb957429b792785ac844c7465bfe6e27001ff268015b84a68a47155e"
Jul 18 02:34:31.491211 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.491175    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b14168acb957429b792785ac844c7465bfe6e27001ff268015b84a68a47155e"} err="failed to get container status \"1b14168acb957429b792785ac844c7465bfe6e27001ff268015b84a68a47155e\": rpc error: code = NotFound desc = could not find container \"1b14168acb957429b792785ac844c7465bfe6e27001ff268015b84a68a47155e\": container with ID starting with 1b14168acb957429b792785ac844c7465bfe6e27001ff268015b84a68a47155e not found: ID does not exist"
Jul 18 02:34:31.491211 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.491193    2576 scope.go:117] "RemoveContainer" containerID="ade644167048f2c4ccc683b9ff7028975f73c57dde8fc6ec414ea57b08d2b997"
Jul 18 02:34:31.491397 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:34:31.491382    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ade644167048f2c4ccc683b9ff7028975f73c57dde8fc6ec414ea57b08d2b997\": container with ID starting with ade644167048f2c4ccc683b9ff7028975f73c57dde8fc6ec414ea57b08d2b997 not found: ID does not exist" containerID="ade644167048f2c4ccc683b9ff7028975f73c57dde8fc6ec414ea57b08d2b997"
Jul 18 02:34:31.491448 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.491401    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ade644167048f2c4ccc683b9ff7028975f73c57dde8fc6ec414ea57b08d2b997"} err="failed to get container status \"ade644167048f2c4ccc683b9ff7028975f73c57dde8fc6ec414ea57b08d2b997\": rpc error: code = NotFound desc = could not find container \"ade644167048f2c4ccc683b9ff7028975f73c57dde8fc6ec414ea57b08d2b997\": container with ID starting with ade644167048f2c4ccc683b9ff7028975f73c57dde8fc6ec414ea57b08d2b997 not found: ID does not exist"
Jul 18 02:34:31.496860 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.496838    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"]
Jul 18 02:34:31.498698 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.498679    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-9e2f67-openshift-default-69bbf99-lnqxd"]
Jul 18 02:34:31.970257 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:31.970225    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23b913aa-93c9-41f1-8f29-feb27620f34e" path="/var/lib/kubelet/pods/23b913aa-93c9-41f1-8f29-feb27620f34e/volumes"
Jul 18 02:34:36.930800 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:36.930765    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a45ea2-8475b46474-jk4pg"]
Jul 18 02:34:36.931221 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:36.930979    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a45ea2-8475b46474-jk4pg" podUID="0d8f5c80-416b-4452-acca-74836f94162a" containerName="maas-api" containerID="cri-o://31eb59e436aa12e9fcc3353253d1febeca94180b9f13078a4f9901f1e44d1c73" gracePeriod=30
Jul 18 02:34:37.191331 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:37.191268    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a45ea2-8475b46474-jk4pg"
Jul 18 02:34:37.225991 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:37.225961    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d8f5c80-416b-4452-acca-74836f94162a-openshift-service-ca\") pod \"0d8f5c80-416b-4452-acca-74836f94162a\" (UID: \"0d8f5c80-416b-4452-acca-74836f94162a\") "
Jul 18 02:34:37.226175 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:37.226000    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d8f5c80-416b-4452-acca-74836f94162a-maas-api-tls\") pod \"0d8f5c80-416b-4452-acca-74836f94162a\" (UID: \"0d8f5c80-416b-4452-acca-74836f94162a\") "
Jul 18 02:34:37.226175 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:37.226028    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b54bl\" (UniqueName: \"kubernetes.io/projected/0d8f5c80-416b-4452-acca-74836f94162a-kube-api-access-b54bl\") pod \"0d8f5c80-416b-4452-acca-74836f94162a\" (UID: \"0d8f5c80-416b-4452-acca-74836f94162a\") "
Jul 18 02:34:37.226372 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:37.226350    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d8f5c80-416b-4452-acca-74836f94162a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0d8f5c80-416b-4452-acca-74836f94162a" (UID: "0d8f5c80-416b-4452-acca-74836f94162a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:34:37.228177 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:37.228149    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d8f5c80-416b-4452-acca-74836f94162a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0d8f5c80-416b-4452-acca-74836f94162a" (UID: "0d8f5c80-416b-4452-acca-74836f94162a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:34:37.228286 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:37.228268    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d8f5c80-416b-4452-acca-74836f94162a-kube-api-access-b54bl" (OuterVolumeSpecName: "kube-api-access-b54bl") pod "0d8f5c80-416b-4452-acca-74836f94162a" (UID: "0d8f5c80-416b-4452-acca-74836f94162a"). InnerVolumeSpecName "kube-api-access-b54bl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:34:37.326945 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:37.326901    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d8f5c80-416b-4452-acca-74836f94162a-openshift-service-ca\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:34:37.326945 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:37.326941    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d8f5c80-416b-4452-acca-74836f94162a-maas-api-tls\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:34:37.326945 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:37.326955    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b54bl\" (UniqueName: \"kubernetes.io/projected/0d8f5c80-416b-4452-acca-74836f94162a-kube-api-access-b54bl\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:34:37.508050 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:37.507955    2576 generic.go:356] "Generic (PLEG): container finished" podID="0d8f5c80-416b-4452-acca-74836f94162a" containerID="31eb59e436aa12e9fcc3353253d1febeca94180b9f13078a4f9901f1e44d1c73" exitCode=0
Jul 18 02:34:37.508050 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:37.508007    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a45ea2-8475b46474-jk4pg"
Jul 18 02:34:37.508279 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:37.508046    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a45ea2-8475b46474-jk4pg" event={"ID":"0d8f5c80-416b-4452-acca-74836f94162a","Type":"ContainerDied","Data":"31eb59e436aa12e9fcc3353253d1febeca94180b9f13078a4f9901f1e44d1c73"}
Jul 18 02:34:37.508279 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:37.508124    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a45ea2-8475b46474-jk4pg" event={"ID":"0d8f5c80-416b-4452-acca-74836f94162a","Type":"ContainerDied","Data":"c81781e91c8c29d5336035efb75eab099b132b4670aa5e7c03ba9d83e0d1fe58"}
Jul 18 02:34:37.508279 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:37.508145    2576 scope.go:117] "RemoveContainer" containerID="31eb59e436aa12e9fcc3353253d1febeca94180b9f13078a4f9901f1e44d1c73"
Jul 18 02:34:37.518644 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:37.518620    2576 scope.go:117] "RemoveContainer" containerID="31eb59e436aa12e9fcc3353253d1febeca94180b9f13078a4f9901f1e44d1c73"
Jul 18 02:34:37.518985 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:34:37.518962    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31eb59e436aa12e9fcc3353253d1febeca94180b9f13078a4f9901f1e44d1c73\": container with ID starting with 31eb59e436aa12e9fcc3353253d1febeca94180b9f13078a4f9901f1e44d1c73 not found: ID does not exist" containerID="31eb59e436aa12e9fcc3353253d1febeca94180b9f13078a4f9901f1e44d1c73"
Jul 18 02:34:37.519068 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:37.518992    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31eb59e436aa12e9fcc3353253d1febeca94180b9f13078a4f9901f1e44d1c73"} err="failed to get container status \"31eb59e436aa12e9fcc3353253d1febeca94180b9f13078a4f9901f1e44d1c73\": rpc error: code = NotFound desc = could not find container \"31eb59e436aa12e9fcc3353253d1febeca94180b9f13078a4f9901f1e44d1c73\": container with ID starting with 31eb59e436aa12e9fcc3353253d1febeca94180b9f13078a4f9901f1e44d1c73 not found: ID does not exist"
Jul 18 02:34:37.531328 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:37.531300    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a45ea2-8475b46474-jk4pg"]
Jul 18 02:34:37.533193 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:37.533175    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a45ea2-8475b46474-jk4pg"]
Jul 18 02:34:37.970439 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:37.970403    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d8f5c80-416b-4452-acca-74836f94162a" path="/var/lib/kubelet/pods/0d8f5c80-416b-4452-acca-74836f94162a/volumes"
Jul 18 02:34:46.158412 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:46.158377    2576 scope.go:117] "RemoveContainer" containerID="a7af5a0191159bfc30f9d2360481e27124e1f66f596a9cd124284381a7aead7f"
Jul 18 02:34:51.698871 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:51.698833    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6e5971-dc8779ddc-2n8kd"]
Jul 18 02:34:51.699404 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:51.699388    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad5b3cc5-0294-4a4b-8ebf-4090d62322dd" containerName="maas-api"
Jul 18 02:34:51.699473 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:51.699407    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad5b3cc5-0294-4a4b-8ebf-4090d62322dd" containerName="maas-api"
Jul 18 02:34:51.699473 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:51.699418    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d8f5c80-416b-4452-acca-74836f94162a" containerName="maas-api"
Jul 18 02:34:51.699473 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:51.699443    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d8f5c80-416b-4452-acca-74836f94162a" containerName="maas-api"
Jul 18 02:34:51.699473 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:51.699464    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23b913aa-93c9-41f1-8f29-feb27620f34e" containerName="istio-proxy"
Jul 18 02:34:51.699599 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:51.699474    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="23b913aa-93c9-41f1-8f29-feb27620f34e" containerName="istio-proxy"
Jul 18 02:34:51.699632 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:51.699598    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="23b913aa-93c9-41f1-8f29-feb27620f34e" containerName="istio-proxy"
Jul 18 02:34:51.699632 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:51.699609    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad5b3cc5-0294-4a4b-8ebf-4090d62322dd" containerName="maas-api"
Jul 18 02:34:51.699632 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:51.699621    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d8f5c80-416b-4452-acca-74836f94162a" containerName="maas-api"
Jul 18 02:34:51.699632 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:51.699629    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="23b913aa-93c9-41f1-8f29-feb27620f34e" containerName="istio-proxy"
Jul 18 02:34:51.704396 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:51.704377    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e5971-dc8779ddc-2n8kd"
Jul 18 02:34:51.706658 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:51.706639    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-6e5971-serving-cert\""
Jul 18 02:34:51.714165 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:51.714140    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6e5971-dc8779ddc-2n8kd"]
Jul 18 02:34:51.761980 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:51.761950    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66kgc\" (UniqueName: \"kubernetes.io/projected/f889d08a-7551-4544-aedb-04c1ec384e27-kube-api-access-66kgc\") pod \"maas-api-e2e-api-a-6e5971-dc8779ddc-2n8kd\" (UID: \"f889d08a-7551-4544-aedb-04c1ec384e27\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e5971-dc8779ddc-2n8kd"
Jul 18 02:34:51.762161 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:51.762068    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f889d08a-7551-4544-aedb-04c1ec384e27-maas-api-tls\") pod \"maas-api-e2e-api-a-6e5971-dc8779ddc-2n8kd\" (UID: \"f889d08a-7551-4544-aedb-04c1ec384e27\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e5971-dc8779ddc-2n8kd"
Jul 18 02:34:51.762161 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:51.762139    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f889d08a-7551-4544-aedb-04c1ec384e27-openshift-service-ca\") pod \"maas-api-e2e-api-a-6e5971-dc8779ddc-2n8kd\" (UID: \"f889d08a-7551-4544-aedb-04c1ec384e27\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e5971-dc8779ddc-2n8kd"
Jul 18 02:34:51.863003 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:51.862963    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f889d08a-7551-4544-aedb-04c1ec384e27-openshift-service-ca\") pod \"maas-api-e2e-api-a-6e5971-dc8779ddc-2n8kd\" (UID: \"f889d08a-7551-4544-aedb-04c1ec384e27\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e5971-dc8779ddc-2n8kd"
Jul 18 02:34:51.863214 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:51.863032    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66kgc\" (UniqueName: \"kubernetes.io/projected/f889d08a-7551-4544-aedb-04c1ec384e27-kube-api-access-66kgc\") pod \"maas-api-e2e-api-a-6e5971-dc8779ddc-2n8kd\" (UID: \"f889d08a-7551-4544-aedb-04c1ec384e27\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e5971-dc8779ddc-2n8kd"
Jul 18 02:34:51.863214 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:51.863115    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f889d08a-7551-4544-aedb-04c1ec384e27-maas-api-tls\") pod \"maas-api-e2e-api-a-6e5971-dc8779ddc-2n8kd\" (UID: \"f889d08a-7551-4544-aedb-04c1ec384e27\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e5971-dc8779ddc-2n8kd"
Jul 18 02:34:51.863337 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:34:51.863240    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-6e5971-serving-cert: secret "maas-api-e2e-api-a-6e5971-serving-cert" not found
Jul 18 02:34:51.863337 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:34:51.863311    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f889d08a-7551-4544-aedb-04c1ec384e27-maas-api-tls podName:f889d08a-7551-4544-aedb-04c1ec384e27 nodeName:}" failed. No retries permitted until 2026-07-18 02:34:52.363290184 +0000 UTC m=+2286.988326833 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f889d08a-7551-4544-aedb-04c1ec384e27-maas-api-tls") pod "maas-api-e2e-api-a-6e5971-dc8779ddc-2n8kd" (UID: "f889d08a-7551-4544-aedb-04c1ec384e27") : secret "maas-api-e2e-api-a-6e5971-serving-cert" not found
Jul 18 02:34:51.863781 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:51.863761    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f889d08a-7551-4544-aedb-04c1ec384e27-openshift-service-ca\") pod \"maas-api-e2e-api-a-6e5971-dc8779ddc-2n8kd\" (UID: \"f889d08a-7551-4544-aedb-04c1ec384e27\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e5971-dc8779ddc-2n8kd"
Jul 18 02:34:51.871771 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:51.871747    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66kgc\" (UniqueName: \"kubernetes.io/projected/f889d08a-7551-4544-aedb-04c1ec384e27-kube-api-access-66kgc\") pod \"maas-api-e2e-api-a-6e5971-dc8779ddc-2n8kd\" (UID: \"f889d08a-7551-4544-aedb-04c1ec384e27\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e5971-dc8779ddc-2n8kd"
Jul 18 02:34:52.366720 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:52.366681    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f889d08a-7551-4544-aedb-04c1ec384e27-maas-api-tls\") pod \"maas-api-e2e-api-a-6e5971-dc8779ddc-2n8kd\" (UID: \"f889d08a-7551-4544-aedb-04c1ec384e27\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e5971-dc8779ddc-2n8kd"
Jul 18 02:34:52.369377 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:52.369348    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f889d08a-7551-4544-aedb-04c1ec384e27-maas-api-tls\") pod \"maas-api-e2e-api-a-6e5971-dc8779ddc-2n8kd\" (UID: \"f889d08a-7551-4544-aedb-04c1ec384e27\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e5971-dc8779ddc-2n8kd"
Jul 18 02:34:52.616276 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:52.616236    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e5971-dc8779ddc-2n8kd"
Jul 18 02:34:52.771884 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:34:52.771821    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf889d08a_7551_4544_aedb_04c1ec384e27.slice/crio-2b607d31afae9996c4969208214a3dcb643c9143bb0b12a7048731f81bc728da WatchSource:0}: Error finding container 2b607d31afae9996c4969208214a3dcb643c9143bb0b12a7048731f81bc728da: Status 404 returned error can't find the container with id 2b607d31afae9996c4969208214a3dcb643c9143bb0b12a7048731f81bc728da
Jul 18 02:34:52.773701 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:52.772618    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6e5971-dc8779ddc-2n8kd"]
Jul 18 02:34:52.773837 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:52.773818    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 02:34:53.593245 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:53.593164    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e5971-dc8779ddc-2n8kd" event={"ID":"f889d08a-7551-4544-aedb-04c1ec384e27","Type":"ContainerStarted","Data":"74a44ea0654d360da6ece63eab59cbe7bd6266736809f1ce12002218c60167ad"}
Jul 18 02:34:53.593245 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:53.593219    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e5971-dc8779ddc-2n8kd" event={"ID":"f889d08a-7551-4544-aedb-04c1ec384e27","Type":"ContainerStarted","Data":"2b607d31afae9996c4969208214a3dcb643c9143bb0b12a7048731f81bc728da"}
Jul 18 02:34:53.593465 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:53.593370    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e5971-dc8779ddc-2n8kd"
Jul 18 02:34:53.617087 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:53.613580    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e5971-dc8779ddc-2n8kd" podStartSLOduration=2.159132384 podStartE2EDuration="2.613559368s" totalImagesPullingTime="454.426984ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:34:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:34:53.608982586 +0000 UTC m=+2288.234019254" watchObservedRunningTime="2026-07-18 02:34:53.613559368 +0000 UTC m=+2288.238596032"
Jul 18 02:34:56.801812 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:56.801778    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"]
Jul 18 02:34:56.802200 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:56.802188    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23b913aa-93c9-41f1-8f29-feb27620f34e" containerName="istio-proxy"
Jul 18 02:34:56.802258 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:56.802201    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="23b913aa-93c9-41f1-8f29-feb27620f34e" containerName="istio-proxy"
Jul 18 02:34:56.805630 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:56.805609    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"
Jul 18 02:34:56.808045 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:56.808019    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-e8ceff-openshift-default-dockercfg-wvh44\""
Jul 18 02:34:56.818353 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:56.818321    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"]
Jul 18 02:34:56.910559 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:56.910521    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cecbf79-c996-46bd-9dce-bd8110d10eb3-credential-socket\") pod \"e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc\" (UID: \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\") " pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"
Jul 18 02:34:56.910559 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:56.910560    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cecbf79-c996-46bd-9dce-bd8110d10eb3-workload-certs\") pod \"e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc\" (UID: \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\") " pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"
Jul 18 02:34:56.910826 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:56.910578    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cecbf79-c996-46bd-9dce-bd8110d10eb3-istio-data\") pod \"e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc\" (UID: \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\") " pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"
Jul 18 02:34:56.910826 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:56.910687    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btp2r\" (UniqueName: \"kubernetes.io/projected/7cecbf79-c996-46bd-9dce-bd8110d10eb3-kube-api-access-btp2r\") pod \"e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc\" (UID: \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\") " pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"
Jul 18 02:34:56.910826 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:56.910727    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cecbf79-c996-46bd-9dce-bd8110d10eb3-workload-socket\") pod \"e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc\" (UID: \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\") " pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"
Jul 18 02:34:56.910996 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:56.910830    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cecbf79-c996-46bd-9dce-bd8110d10eb3-istio-token\") pod \"e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc\" (UID: \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\") " pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"
Jul 18 02:34:56.910996 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:56.910860    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cecbf79-c996-46bd-9dce-bd8110d10eb3-istiod-ca-cert\") pod \"e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc\" (UID: \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\") " pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"
Jul 18 02:34:56.910996 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:56.910900    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cecbf79-c996-46bd-9dce-bd8110d10eb3-istio-podinfo\") pod \"e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc\" (UID: \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\") " pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"
Jul 18 02:34:56.910996 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:56.910939    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cecbf79-c996-46bd-9dce-bd8110d10eb3-istio-envoy\") pod \"e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc\" (UID: \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\") " pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"
Jul 18 02:34:57.011392 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:57.011353    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btp2r\" (UniqueName: \"kubernetes.io/projected/7cecbf79-c996-46bd-9dce-bd8110d10eb3-kube-api-access-btp2r\") pod \"e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc\" (UID: \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\") " pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"
Jul 18 02:34:57.011392 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:57.011396    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cecbf79-c996-46bd-9dce-bd8110d10eb3-workload-socket\") pod \"e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc\" (UID: \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\") " pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"
Jul 18 02:34:57.011668 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:57.011472    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cecbf79-c996-46bd-9dce-bd8110d10eb3-istio-token\") pod \"e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc\" (UID: \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\") " pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"
Jul 18 02:34:57.011668 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:57.011501    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cecbf79-c996-46bd-9dce-bd8110d10eb3-istiod-ca-cert\") pod \"e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc\" (UID: \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\") " pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"
Jul 18 02:34:57.011668 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:57.011538    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cecbf79-c996-46bd-9dce-bd8110d10eb3-istio-podinfo\") pod \"e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc\" (UID: \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\") " pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"
Jul 18 02:34:57.011822 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:57.011723    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cecbf79-c996-46bd-9dce-bd8110d10eb3-istio-envoy\") pod \"e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc\" (UID: \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\") " pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"
Jul 18 02:34:57.011822 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:57.011781    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cecbf79-c996-46bd-9dce-bd8110d10eb3-credential-socket\") pod \"e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc\" (UID: \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\") " pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"
Jul 18 02:34:57.011822 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:57.011803    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cecbf79-c996-46bd-9dce-bd8110d10eb3-workload-certs\") pod \"e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc\" (UID: \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\") " pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"
Jul 18 02:34:57.011973 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:57.011821    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cecbf79-c996-46bd-9dce-bd8110d10eb3-istio-data\") pod \"e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc\" (UID: \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\") " pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"
Jul 18 02:34:57.011973 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:57.011861    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cecbf79-c996-46bd-9dce-bd8110d10eb3-workload-socket\") pod \"e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc\" (UID: \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\") " pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"
Jul 18 02:34:57.012162 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:57.012140    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cecbf79-c996-46bd-9dce-bd8110d10eb3-credential-socket\") pod \"e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc\" (UID: \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\") " pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"
Jul 18 02:34:57.012230 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:57.012200    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cecbf79-c996-46bd-9dce-bd8110d10eb3-workload-certs\") pod \"e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc\" (UID: \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\") " pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"
Jul 18 02:34:57.012230 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:57.012216    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cecbf79-c996-46bd-9dce-bd8110d10eb3-istio-data\") pod \"e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc\" (UID: \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\") " pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"
Jul 18 02:34:57.012310 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:57.012234    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cecbf79-c996-46bd-9dce-bd8110d10eb3-istiod-ca-cert\") pod \"e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc\" (UID: \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\") " pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"
Jul 18 02:34:57.014403 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:57.014381    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cecbf79-c996-46bd-9dce-bd8110d10eb3-istio-envoy\") pod \"e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc\" (UID: \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\") " pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"
Jul 18 02:34:57.014737 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:57.014715    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cecbf79-c996-46bd-9dce-bd8110d10eb3-istio-podinfo\") pod \"e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc\" (UID: \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\") " pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"
Jul 18 02:34:57.019695 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:57.019676    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cecbf79-c996-46bd-9dce-bd8110d10eb3-istio-token\") pod \"e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc\" (UID: \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\") " pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"
Jul 18 02:34:57.019815 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:57.019792    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btp2r\" (UniqueName: \"kubernetes.io/projected/7cecbf79-c996-46bd-9dce-bd8110d10eb3-kube-api-access-btp2r\") pod \"e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc\" (UID: \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\") " pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"
Jul 18 02:34:57.116708 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:57.116620    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"
Jul 18 02:34:57.259612 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:57.259588    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"]
Jul 18 02:34:57.261504 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:34:57.261471    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cecbf79_c996_46bd_9dce_bd8110d10eb3.slice/crio-e6e6b25687372573427fee5848feccdff1931bccd11023a56c9949feb6dfae4a WatchSource:0}: Error finding container e6e6b25687372573427fee5848feccdff1931bccd11023a56c9949feb6dfae4a: Status 404 returned error can't find the container with id e6e6b25687372573427fee5848feccdff1931bccd11023a56c9949feb6dfae4a
Jul 18 02:34:57.263768 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:57.263733    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 18 02:34:57.263857 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:57.263821    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 18 02:34:57.263857 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:57.263849    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 18 02:34:57.618060 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:57.618021    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc" event={"ID":"7cecbf79-c996-46bd-9dce-bd8110d10eb3","Type":"ContainerStarted","Data":"0c21daf03a129ffe171570054661b83656e4c32d4f25036e42987d7a09259f33"}
Jul 18 02:34:57.618252 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:57.618088    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc" event={"ID":"7cecbf79-c996-46bd-9dce-bd8110d10eb3","Type":"ContainerStarted","Data":"e6e6b25687372573427fee5848feccdff1931bccd11023a56c9949feb6dfae4a"}
Jul 18 02:34:57.637036 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:57.636983    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc" podStartSLOduration=1.6369650519999999 podStartE2EDuration="1.636965052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:34:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:34:57.636834919 +0000 UTC m=+2292.261871587" watchObservedRunningTime="2026-07-18 02:34:57.636965052 +0000 UTC m=+2292.262001718"
Jul 18 02:34:58.117258 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:58.117218    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"
Jul 18 02:34:58.122850 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:58.122823    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"
Jul 18 02:34:58.629009 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:58.628970    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"
Jul 18 02:34:58.629967 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:58.629942    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"
Jul 18 02:34:59.604103 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:34:59.604073    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e5971-dc8779ddc-2n8kd"
Jul 18 02:35:14.670665 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:14.670614    2576 patch_prober.go:28] interesting pod/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 18 02:35:14.671095 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:14.670703    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc" podUID="7cecbf79-c996-46bd-9dce-bd8110d10eb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 18 02:35:15.727394 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:15.727357    2576 generic.go:356] "Generic (PLEG): container finished" podID="7cecbf79-c996-46bd-9dce-bd8110d10eb3" containerID="0c21daf03a129ffe171570054661b83656e4c32d4f25036e42987d7a09259f33" exitCode=0
Jul 18 02:35:15.727835 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:15.727462    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc" event={"ID":"7cecbf79-c996-46bd-9dce-bd8110d10eb3","Type":"ContainerDied","Data":"0c21daf03a129ffe171570054661b83656e4c32d4f25036e42987d7a09259f33"}
Jul 18 02:35:15.727918 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:15.727900    2576 scope.go:117] "RemoveContainer" containerID="0c21daf03a129ffe171570054661b83656e4c32d4f25036e42987d7a09259f33"
Jul 18 02:35:15.728943 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:15.728916    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 18 02:35:15.728990 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:15.728981    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 18 02:35:15.729026 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:15.729013    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 18 02:35:16.734813 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:16.734778    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc" event={"ID":"7cecbf79-c996-46bd-9dce-bd8110d10eb3","Type":"ContainerStarted","Data":"a0128fc1c9ae40baba538a1179ffc350ad655f04398e14bf52e4d90ebea6c80b"}
Jul 18 02:35:16.997479 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:16.997351    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6e5971-8sdcc"]
Jul 18 02:35:17.001917 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:17.001882    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6e5971-8sdcc"
Jul 18 02:35:17.004555 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:17.004530    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g69gp\""
Jul 18 02:35:17.007226 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:17.007194    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6e5971-8sdcc"]
Jul 18 02:35:17.098914 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:17.097795    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71f11c17-f87c-436f-b9d9-2a59811fcdd9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-6e5971-8sdcc\" (UID: \"71f11c17-f87c-436f-b9d9-2a59811fcdd9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6e5971-8sdcc"
Jul 18 02:35:17.117688 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:17.117651    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"
Jul 18 02:35:17.117998 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:17.117974    2576 patch_prober.go:28] interesting pod/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Jul 18 02:35:17.118044 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:17.118025    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc" podUID="7cecbf79-c996-46bd-9dce-bd8110d10eb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Jul 18 02:35:17.199352 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:17.199303    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71f11c17-f87c-436f-b9d9-2a59811fcdd9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-6e5971-8sdcc\" (UID: \"71f11c17-f87c-436f-b9d9-2a59811fcdd9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6e5971-8sdcc"
Jul 18 02:35:17.199979 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:17.199958    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71f11c17-f87c-436f-b9d9-2a59811fcdd9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-6e5971-8sdcc\" (UID: \"71f11c17-f87c-436f-b9d9-2a59811fcdd9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6e5971-8sdcc"
Jul 18 02:35:17.315086 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:17.314981    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6e5971-8sdcc"
Jul 18 02:35:17.500768 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:17.499935    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6e5971-8sdcc"]
Jul 18 02:35:17.741600 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:17.741561    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6e5971-8sdcc" event={"ID":"71f11c17-f87c-436f-b9d9-2a59811fcdd9","Type":"ContainerStarted","Data":"f442de90678fbbb9604945f1ad9d8579f9f1d0b20100dcdb47228b20860ed616"}
Jul 18 02:35:18.117860 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:18.117748    2576 patch_prober.go:28] interesting pod/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Jul 18 02:35:18.117860 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:18.117816    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc" podUID="7cecbf79-c996-46bd-9dce-bd8110d10eb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Jul 18 02:35:18.748510 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:18.748473    2576 generic.go:356] "Generic (PLEG): container finished" podID="71f11c17-f87c-436f-b9d9-2a59811fcdd9" containerID="12218509f627ca050e666cbaad710db3b9eeadad70404c909b2d61cdc33718d4" exitCode=0
Jul 18 02:35:18.748961 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:18.748552    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6e5971-8sdcc" event={"ID":"71f11c17-f87c-436f-b9d9-2a59811fcdd9","Type":"ContainerDied","Data":"12218509f627ca050e666cbaad710db3b9eeadad70404c909b2d61cdc33718d4"}
Jul 18 02:35:19.117904 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:19.117859    2576 patch_prober.go:28] interesting pod/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Jul 18 02:35:19.118131 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:19.117925    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc" podUID="7cecbf79-c996-46bd-9dce-bd8110d10eb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Jul 18 02:35:19.945225 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:19.945201    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6e5971-8sdcc"
Jul 18 02:35:20.025027 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:20.024932    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71f11c17-f87c-436f-b9d9-2a59811fcdd9-maas-api-service-ca\") pod \"71f11c17-f87c-436f-b9d9-2a59811fcdd9\" (UID: \"71f11c17-f87c-436f-b9d9-2a59811fcdd9\") "
Jul 18 02:35:20.025251 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:20.025228    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71f11c17-f87c-436f-b9d9-2a59811fcdd9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "71f11c17-f87c-436f-b9d9-2a59811fcdd9" (UID: "71f11c17-f87c-436f-b9d9-2a59811fcdd9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:35:20.118138 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:20.118104    2576 patch_prober.go:28] interesting pod/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Jul 18 02:35:20.118334 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:20.118162    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc" podUID="7cecbf79-c996-46bd-9dce-bd8110d10eb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Jul 18 02:35:20.125766 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:20.125738    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71f11c17-f87c-436f-b9d9-2a59811fcdd9-maas-api-service-ca\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:35:20.761311 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:20.761274    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6e5971-8sdcc" event={"ID":"71f11c17-f87c-436f-b9d9-2a59811fcdd9","Type":"ContainerDied","Data":"f442de90678fbbb9604945f1ad9d8579f9f1d0b20100dcdb47228b20860ed616"}
Jul 18 02:35:20.761510 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:20.761326    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6e5971-8sdcc"
Jul 18 02:35:20.761510 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:20.761352    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f442de90678fbbb9604945f1ad9d8579f9f1d0b20100dcdb47228b20860ed616"
Jul 18 02:35:21.120599 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:21.120523    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"
Jul 18 02:35:21.120974 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:21.120806    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"
Jul 18 02:35:21.121599 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:21.121581    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"
Jul 18 02:35:32.783064 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:32.782952    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6e5971-dc8779ddc-2n8kd"]
Jul 18 02:35:32.783891 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:32.783856    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e5971-dc8779ddc-2n8kd" podUID="f889d08a-7551-4544-aedb-04c1ec384e27" containerName="maas-api" containerID="cri-o://74a44ea0654d360da6ece63eab59cbe7bd6266736809f1ce12002218c60167ad" gracePeriod=30
Jul 18 02:35:33.042744 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:33.042681    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e5971-dc8779ddc-2n8kd"
Jul 18 02:35:33.050667 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:33.050631    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-66kgc\" (UniqueName: \"kubernetes.io/projected/f889d08a-7551-4544-aedb-04c1ec384e27-kube-api-access-66kgc\") pod \"f889d08a-7551-4544-aedb-04c1ec384e27\" (UID: \"f889d08a-7551-4544-aedb-04c1ec384e27\") "
Jul 18 02:35:33.050807 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:33.050684    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f889d08a-7551-4544-aedb-04c1ec384e27-maas-api-tls\") pod \"f889d08a-7551-4544-aedb-04c1ec384e27\" (UID: \"f889d08a-7551-4544-aedb-04c1ec384e27\") "
Jul 18 02:35:33.050807 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:33.050738    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f889d08a-7551-4544-aedb-04c1ec384e27-openshift-service-ca\") pod \"f889d08a-7551-4544-aedb-04c1ec384e27\" (UID: \"f889d08a-7551-4544-aedb-04c1ec384e27\") "
Jul 18 02:35:33.051244 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:33.051215    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f889d08a-7551-4544-aedb-04c1ec384e27-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f889d08a-7551-4544-aedb-04c1ec384e27" (UID: "f889d08a-7551-4544-aedb-04c1ec384e27"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:35:33.051376 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:33.051293    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f889d08a-7551-4544-aedb-04c1ec384e27-openshift-service-ca\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:35:33.053481 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:33.053449    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f889d08a-7551-4544-aedb-04c1ec384e27-kube-api-access-66kgc" (OuterVolumeSpecName: "kube-api-access-66kgc") pod "f889d08a-7551-4544-aedb-04c1ec384e27" (UID: "f889d08a-7551-4544-aedb-04c1ec384e27"). InnerVolumeSpecName "kube-api-access-66kgc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:35:33.053567 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:33.053467    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f889d08a-7551-4544-aedb-04c1ec384e27-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f889d08a-7551-4544-aedb-04c1ec384e27" (UID: "f889d08a-7551-4544-aedb-04c1ec384e27"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:35:33.152602 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:33.152566    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f889d08a-7551-4544-aedb-04c1ec384e27-maas-api-tls\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:35:33.152602 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:33.152599    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66kgc\" (UniqueName: \"kubernetes.io/projected/f889d08a-7551-4544-aedb-04c1ec384e27-kube-api-access-66kgc\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:35:33.832631 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:33.832600    2576 generic.go:356] "Generic (PLEG): container finished" podID="f889d08a-7551-4544-aedb-04c1ec384e27" containerID="74a44ea0654d360da6ece63eab59cbe7bd6266736809f1ce12002218c60167ad" exitCode=0
Jul 18 02:35:33.833027 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:33.832657    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e5971-dc8779ddc-2n8kd"
Jul 18 02:35:33.833027 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:33.832690    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e5971-dc8779ddc-2n8kd" event={"ID":"f889d08a-7551-4544-aedb-04c1ec384e27","Type":"ContainerDied","Data":"74a44ea0654d360da6ece63eab59cbe7bd6266736809f1ce12002218c60167ad"}
Jul 18 02:35:33.833027 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:33.832772    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e5971-dc8779ddc-2n8kd" event={"ID":"f889d08a-7551-4544-aedb-04c1ec384e27","Type":"ContainerDied","Data":"2b607d31afae9996c4969208214a3dcb643c9143bb0b12a7048731f81bc728da"}
Jul 18 02:35:33.833027 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:33.832799    2576 scope.go:117] "RemoveContainer" containerID="74a44ea0654d360da6ece63eab59cbe7bd6266736809f1ce12002218c60167ad"
Jul 18 02:35:33.842073 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:33.842056    2576 scope.go:117] "RemoveContainer" containerID="74a44ea0654d360da6ece63eab59cbe7bd6266736809f1ce12002218c60167ad"
Jul 18 02:35:33.842331 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:35:33.842312    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74a44ea0654d360da6ece63eab59cbe7bd6266736809f1ce12002218c60167ad\": container with ID starting with 74a44ea0654d360da6ece63eab59cbe7bd6266736809f1ce12002218c60167ad not found: ID does not exist" containerID="74a44ea0654d360da6ece63eab59cbe7bd6266736809f1ce12002218c60167ad"
Jul 18 02:35:33.842381 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:33.842340    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74a44ea0654d360da6ece63eab59cbe7bd6266736809f1ce12002218c60167ad"} err="failed to get container status \"74a44ea0654d360da6ece63eab59cbe7bd6266736809f1ce12002218c60167ad\": rpc error: code = NotFound desc = could not find container \"74a44ea0654d360da6ece63eab59cbe7bd6266736809f1ce12002218c60167ad\": container with ID starting with 74a44ea0654d360da6ece63eab59cbe7bd6266736809f1ce12002218c60167ad not found: ID does not exist"
Jul 18 02:35:33.854154 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:33.854124    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6e5971-dc8779ddc-2n8kd"]
Jul 18 02:35:33.858250 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:33.858222    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6e5971-dc8779ddc-2n8kd"]
Jul 18 02:35:33.970298 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:33.970261    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f889d08a-7551-4544-aedb-04c1ec384e27" path="/var/lib/kubelet/pods/f889d08a-7551-4544-aedb-04c1ec384e27/volumes"
Jul 18 02:35:37.077961 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:37.077921    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6e5971-8sdcc"]
Jul 18 02:35:37.080335 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:37.080307    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6e5971-8sdcc"]
Jul 18 02:35:37.969774 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:37.969728    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71f11c17-f87c-436f-b9d9-2a59811fcdd9" path="/var/lib/kubelet/pods/71f11c17-f87c-436f-b9d9-2a59811fcdd9/volumes"
Jul 18 02:35:43.633457 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:43.633360    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e8ceff-ll7ps"]
Jul 18 02:35:43.633818 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:43.633762    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f889d08a-7551-4544-aedb-04c1ec384e27" containerName="maas-api"
Jul 18 02:35:43.633818 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:43.633774    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f889d08a-7551-4544-aedb-04c1ec384e27" containerName="maas-api"
Jul 18 02:35:43.633818 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:43.633784    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71f11c17-f87c-436f-b9d9-2a59811fcdd9" containerName="revoke-keys"
Jul 18 02:35:43.633818 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:43.633790    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="71f11c17-f87c-436f-b9d9-2a59811fcdd9" containerName="revoke-keys"
Jul 18 02:35:43.633952 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:43.633861    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f889d08a-7551-4544-aedb-04c1ec384e27" containerName="maas-api"
Jul 18 02:35:43.633952 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:43.633871    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="71f11c17-f87c-436f-b9d9-2a59811fcdd9" containerName="revoke-keys"
Jul 18 02:35:43.638125 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:43.638107    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e8ceff-ll7ps"
Jul 18 02:35:43.640727 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:43.640706    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g69gp\""
Jul 18 02:35:43.644752 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:43.644724    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e8ceff-ll7ps"]
Jul 18 02:35:43.748499 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:43.748458    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/677f04b4-93db-4f32-98bd-1af387fab769-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-e8ceff-ll7ps\" (UID: \"677f04b4-93db-4f32-98bd-1af387fab769\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e8ceff-ll7ps"
Jul 18 02:35:43.849398 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:43.849357    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/677f04b4-93db-4f32-98bd-1af387fab769-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-e8ceff-ll7ps\" (UID: \"677f04b4-93db-4f32-98bd-1af387fab769\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e8ceff-ll7ps"
Jul 18 02:35:43.850041 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:43.850022    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/677f04b4-93db-4f32-98bd-1af387fab769-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-e8ceff-ll7ps\" (UID: \"677f04b4-93db-4f32-98bd-1af387fab769\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e8ceff-ll7ps"
Jul 18 02:35:43.949953 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:43.949867    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e8ceff-ll7ps"
Jul 18 02:35:44.091973 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:44.091946    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e8ceff-ll7ps"]
Jul 18 02:35:44.093719 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:35:44.093686    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod677f04b4_93db_4f32_98bd_1af387fab769.slice/crio-4d7ec7dcc707c23d88e94250a23f8e643ced79eedd1b4405cdc25f8abe701d56 WatchSource:0}: Error finding container 4d7ec7dcc707c23d88e94250a23f8e643ced79eedd1b4405cdc25f8abe701d56: Status 404 returned error can't find the container with id 4d7ec7dcc707c23d88e94250a23f8e643ced79eedd1b4405cdc25f8abe701d56
Jul 18 02:35:44.897444 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:44.897404    2576 generic.go:356] "Generic (PLEG): container finished" podID="677f04b4-93db-4f32-98bd-1af387fab769" containerID="7128bfcb60892b6862a6a38a735761a716ea469dd7c9d9ff32a12053cf7b23b7" exitCode=0
Jul 18 02:35:44.897842 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:44.897490    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e8ceff-ll7ps" event={"ID":"677f04b4-93db-4f32-98bd-1af387fab769","Type":"ContainerDied","Data":"7128bfcb60892b6862a6a38a735761a716ea469dd7c9d9ff32a12053cf7b23b7"}
Jul 18 02:35:44.897842 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:44.897571    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e8ceff-ll7ps" event={"ID":"677f04b4-93db-4f32-98bd-1af387fab769","Type":"ContainerStarted","Data":"4d7ec7dcc707c23d88e94250a23f8e643ced79eedd1b4405cdc25f8abe701d56"}
Jul 18 02:35:46.041540 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:46.041515    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e8ceff-ll7ps"
Jul 18 02:35:46.069865 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:46.069835    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/677f04b4-93db-4f32-98bd-1af387fab769-maas-api-service-ca\") pod \"677f04b4-93db-4f32-98bd-1af387fab769\" (UID: \"677f04b4-93db-4f32-98bd-1af387fab769\") "
Jul 18 02:35:46.070214 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:46.070190    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/677f04b4-93db-4f32-98bd-1af387fab769-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "677f04b4-93db-4f32-98bd-1af387fab769" (UID: "677f04b4-93db-4f32-98bd-1af387fab769"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:35:46.171779 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:46.171669    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/677f04b4-93db-4f32-98bd-1af387fab769-maas-api-service-ca\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:35:46.908876 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:46.908840    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e8ceff-ll7ps"
Jul 18 02:35:46.909087 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:46.908880    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e8ceff-ll7ps" event={"ID":"677f04b4-93db-4f32-98bd-1af387fab769","Type":"ContainerDied","Data":"4d7ec7dcc707c23d88e94250a23f8e643ced79eedd1b4405cdc25f8abe701d56"}
Jul 18 02:35:46.909087 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:35:46.908929    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4d7ec7dcc707c23d88e94250a23f8e643ced79eedd1b4405cdc25f8abe701d56"
Jul 18 02:36:03.734163 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:03.734120    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e8ceff-ll7ps"]
Jul 18 02:36:03.736107 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:03.736083    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e8ceff-ll7ps"]
Jul 18 02:36:03.969284 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:03.969248    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="677f04b4-93db-4f32-98bd-1af387fab769" path="/var/lib/kubelet/pods/677f04b4-93db-4f32-98bd-1af387fab769/volumes"
Jul 18 02:36:10.003917 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:10.003883    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"]
Jul 18 02:36:10.004405 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:10.004108    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc" podUID="7cecbf79-c996-46bd-9dce-bd8110d10eb3" containerName="istio-proxy" containerID="cri-o://a0128fc1c9ae40baba538a1179ffc350ad655f04398e14bf52e4d90ebea6c80b" gracePeriod=30
Jul 18 02:36:15.270367 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:15.270274    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"
Jul 18 02:36:15.348710 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:15.348674    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cecbf79-c996-46bd-9dce-bd8110d10eb3-istio-podinfo\") pod \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\" (UID: \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\") "
Jul 18 02:36:15.348710 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:15.348712    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cecbf79-c996-46bd-9dce-bd8110d10eb3-istio-token\") pod \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\" (UID: \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\") "
Jul 18 02:36:15.348958 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:15.348729    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cecbf79-c996-46bd-9dce-bd8110d10eb3-istiod-ca-cert\") pod \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\" (UID: \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\") "
Jul 18 02:36:15.348958 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:15.348766    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cecbf79-c996-46bd-9dce-bd8110d10eb3-workload-socket\") pod \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\" (UID: \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\") "
Jul 18 02:36:15.348958 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:15.348798    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-btp2r\" (UniqueName: \"kubernetes.io/projected/7cecbf79-c996-46bd-9dce-bd8110d10eb3-kube-api-access-btp2r\") pod \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\" (UID: \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\") "
Jul 18 02:36:15.348958 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:15.348816    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cecbf79-c996-46bd-9dce-bd8110d10eb3-istio-data\") pod \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\" (UID: \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\") "
Jul 18 02:36:15.349175 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:15.349011    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cecbf79-c996-46bd-9dce-bd8110d10eb3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7cecbf79-c996-46bd-9dce-bd8110d10eb3" (UID: "7cecbf79-c996-46bd-9dce-bd8110d10eb3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:36:15.349243 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:15.349211    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7cecbf79-c996-46bd-9dce-bd8110d10eb3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7cecbf79-c996-46bd-9dce-bd8110d10eb3" (UID: "7cecbf79-c996-46bd-9dce-bd8110d10eb3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:36:15.349801 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:15.349770    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cecbf79-c996-46bd-9dce-bd8110d10eb3-istio-data" (OuterVolumeSpecName: "istio-data") pod "7cecbf79-c996-46bd-9dce-bd8110d10eb3" (UID: "7cecbf79-c996-46bd-9dce-bd8110d10eb3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:36:15.350947 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:15.350926    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cecbf79-c996-46bd-9dce-bd8110d10eb3-istio-token" (OuterVolumeSpecName: "istio-token") pod "7cecbf79-c996-46bd-9dce-bd8110d10eb3" (UID: "7cecbf79-c996-46bd-9dce-bd8110d10eb3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:36:15.351015 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:15.350972    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7cecbf79-c996-46bd-9dce-bd8110d10eb3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7cecbf79-c996-46bd-9dce-bd8110d10eb3" (UID: "7cecbf79-c996-46bd-9dce-bd8110d10eb3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 18 02:36:15.351015 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:15.350979    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cecbf79-c996-46bd-9dce-bd8110d10eb3-kube-api-access-btp2r" (OuterVolumeSpecName: "kube-api-access-btp2r") pod "7cecbf79-c996-46bd-9dce-bd8110d10eb3" (UID: "7cecbf79-c996-46bd-9dce-bd8110d10eb3"). InnerVolumeSpecName "kube-api-access-btp2r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:36:15.450146 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:15.450108    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cecbf79-c996-46bd-9dce-bd8110d10eb3-credential-socket\") pod \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\" (UID: \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\") "
Jul 18 02:36:15.450146 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:15.450142    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cecbf79-c996-46bd-9dce-bd8110d10eb3-istio-envoy\") pod \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\" (UID: \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\") "
Jul 18 02:36:15.450397 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:15.450169    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cecbf79-c996-46bd-9dce-bd8110d10eb3-workload-certs\") pod \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\" (UID: \"7cecbf79-c996-46bd-9dce-bd8110d10eb3\") "
Jul 18 02:36:15.450397 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:15.450385    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cecbf79-c996-46bd-9dce-bd8110d10eb3-workload-socket\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:36:15.450551 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:15.450406    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btp2r\" (UniqueName: \"kubernetes.io/projected/7cecbf79-c996-46bd-9dce-bd8110d10eb3-kube-api-access-btp2r\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:36:15.450551 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:15.450419    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cecbf79-c996-46bd-9dce-bd8110d10eb3-istio-data\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:36:15.450551 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:15.450419    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cecbf79-c996-46bd-9dce-bd8110d10eb3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7cecbf79-c996-46bd-9dce-bd8110d10eb3" (UID: "7cecbf79-c996-46bd-9dce-bd8110d10eb3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:36:15.450551 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:15.450459    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cecbf79-c996-46bd-9dce-bd8110d10eb3-istio-podinfo\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:36:15.450551 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:15.450499    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cecbf79-c996-46bd-9dce-bd8110d10eb3-istio-token\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:36:15.450551 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:15.450498    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cecbf79-c996-46bd-9dce-bd8110d10eb3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7cecbf79-c996-46bd-9dce-bd8110d10eb3" (UID: "7cecbf79-c996-46bd-9dce-bd8110d10eb3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:36:15.450551 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:15.450513    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cecbf79-c996-46bd-9dce-bd8110d10eb3-istiod-ca-cert\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:36:15.452480 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:15.452456    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cecbf79-c996-46bd-9dce-bd8110d10eb3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7cecbf79-c996-46bd-9dce-bd8110d10eb3" (UID: "7cecbf79-c996-46bd-9dce-bd8110d10eb3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:36:15.551248 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:15.551154    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cecbf79-c996-46bd-9dce-bd8110d10eb3-credential-socket\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:36:15.551248 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:15.551185    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cecbf79-c996-46bd-9dce-bd8110d10eb3-istio-envoy\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:36:15.551248 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:15.551195    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cecbf79-c996-46bd-9dce-bd8110d10eb3-workload-certs\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:36:16.067206 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:16.067173    2576 generic.go:356] "Generic (PLEG): container finished" podID="7cecbf79-c996-46bd-9dce-bd8110d10eb3" containerID="a0128fc1c9ae40baba538a1179ffc350ad655f04398e14bf52e4d90ebea6c80b" exitCode=0
Jul 18 02:36:16.067398 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:16.067235    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"
Jul 18 02:36:16.067398 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:16.067254    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc" event={"ID":"7cecbf79-c996-46bd-9dce-bd8110d10eb3","Type":"ContainerDied","Data":"a0128fc1c9ae40baba538a1179ffc350ad655f04398e14bf52e4d90ebea6c80b"}
Jul 18 02:36:16.067398 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:16.067318    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc" event={"ID":"7cecbf79-c996-46bd-9dce-bd8110d10eb3","Type":"ContainerDied","Data":"e6e6b25687372573427fee5848feccdff1931bccd11023a56c9949feb6dfae4a"}
Jul 18 02:36:16.067398 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:16.067349    2576 scope.go:117] "RemoveContainer" containerID="a0128fc1c9ae40baba538a1179ffc350ad655f04398e14bf52e4d90ebea6c80b"
Jul 18 02:36:16.080870 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:16.080839    2576 scope.go:117] "RemoveContainer" containerID="0c21daf03a129ffe171570054661b83656e4c32d4f25036e42987d7a09259f33"
Jul 18 02:36:16.088492 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:16.088460    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"]
Jul 18 02:36:16.091011 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:16.090993    2576 scope.go:117] "RemoveContainer" containerID="a0128fc1c9ae40baba538a1179ffc350ad655f04398e14bf52e4d90ebea6c80b"
Jul 18 02:36:16.091410 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:36:16.091382    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0128fc1c9ae40baba538a1179ffc350ad655f04398e14bf52e4d90ebea6c80b\": container with ID starting with a0128fc1c9ae40baba538a1179ffc350ad655f04398e14bf52e4d90ebea6c80b not found: ID does not exist" containerID="a0128fc1c9ae40baba538a1179ffc350ad655f04398e14bf52e4d90ebea6c80b"
Jul 18 02:36:16.091663 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:16.091422    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0128fc1c9ae40baba538a1179ffc350ad655f04398e14bf52e4d90ebea6c80b"} err="failed to get container status \"a0128fc1c9ae40baba538a1179ffc350ad655f04398e14bf52e4d90ebea6c80b\": rpc error: code = NotFound desc = could not find container \"a0128fc1c9ae40baba538a1179ffc350ad655f04398e14bf52e4d90ebea6c80b\": container with ID starting with a0128fc1c9ae40baba538a1179ffc350ad655f04398e14bf52e4d90ebea6c80b not found: ID does not exist"
Jul 18 02:36:16.091663 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:16.091471    2576 scope.go:117] "RemoveContainer" containerID="0c21daf03a129ffe171570054661b83656e4c32d4f25036e42987d7a09259f33"
Jul 18 02:36:16.091957 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:36:16.091921    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c21daf03a129ffe171570054661b83656e4c32d4f25036e42987d7a09259f33\": container with ID starting with 0c21daf03a129ffe171570054661b83656e4c32d4f25036e42987d7a09259f33 not found: ID does not exist" containerID="0c21daf03a129ffe171570054661b83656e4c32d4f25036e42987d7a09259f33"
Jul 18 02:36:16.092051 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:16.091961    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c21daf03a129ffe171570054661b83656e4c32d4f25036e42987d7a09259f33"} err="failed to get container status \"0c21daf03a129ffe171570054661b83656e4c32d4f25036e42987d7a09259f33\": rpc error: code = NotFound desc = could not find container \"0c21daf03a129ffe171570054661b83656e4c32d4f25036e42987d7a09259f33\": container with ID starting with 0c21daf03a129ffe171570054661b83656e4c32d4f25036e42987d7a09259f33 not found: ID does not exist"
Jul 18 02:36:16.093484 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:16.093462    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-e8ceff-openshift-default-7f796dbcff-slmnc"]
Jul 18 02:36:17.970557 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:17.970521    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7cecbf79-c996-46bd-9dce-bd8110d10eb3" path="/var/lib/kubelet/pods/7cecbf79-c996-46bd-9dce-bd8110d10eb3/volumes"
Jul 18 02:36:19.112443 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.112392    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"]
Jul 18 02:36:19.112858 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.112814    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="677f04b4-93db-4f32-98bd-1af387fab769" containerName="revoke-keys"
Jul 18 02:36:19.112858 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.112827    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="677f04b4-93db-4f32-98bd-1af387fab769" containerName="revoke-keys"
Jul 18 02:36:19.112858 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.112838    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cecbf79-c996-46bd-9dce-bd8110d10eb3" containerName="istio-proxy"
Jul 18 02:36:19.112858 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.112843    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cecbf79-c996-46bd-9dce-bd8110d10eb3" containerName="istio-proxy"
Jul 18 02:36:19.112858 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.112860    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cecbf79-c996-46bd-9dce-bd8110d10eb3" containerName="istio-proxy"
Jul 18 02:36:19.113055 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.112869    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cecbf79-c996-46bd-9dce-bd8110d10eb3" containerName="istio-proxy"
Jul 18 02:36:19.113055 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.112972    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cecbf79-c996-46bd-9dce-bd8110d10eb3" containerName="istio-proxy"
Jul 18 02:36:19.113055 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.112993    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="677f04b4-93db-4f32-98bd-1af387fab769" containerName="revoke-keys"
Jul 18 02:36:19.117499 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.117478    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"
Jul 18 02:36:19.122361 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.122340    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-922cd9-openshift-default-dockercfg-wrjvg\""
Jul 18 02:36:19.130586 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.130561    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"]
Jul 18 02:36:19.191179 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.191144    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-istio-podinfo\") pod \"e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q\" (UID: \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\") " pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"
Jul 18 02:36:19.191179 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.191178    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stnd5\" (UniqueName: \"kubernetes.io/projected/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-kube-api-access-stnd5\") pod \"e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q\" (UID: \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\") " pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"
Jul 18 02:36:19.191467 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.191206    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-credential-socket\") pod \"e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q\" (UID: \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\") " pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"
Jul 18 02:36:19.191467 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.191224    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-istio-data\") pod \"e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q\" (UID: \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\") " pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"
Jul 18 02:36:19.191467 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.191307    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-istiod-ca-cert\") pod \"e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q\" (UID: \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\") " pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"
Jul 18 02:36:19.191467 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.191364    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-istio-token\") pod \"e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q\" (UID: \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\") " pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"
Jul 18 02:36:19.191467 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.191398    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-workload-certs\") pod \"e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q\" (UID: \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\") " pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"
Jul 18 02:36:19.191663 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.191489    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-workload-socket\") pod \"e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q\" (UID: \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\") " pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"
Jul 18 02:36:19.191663 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.191531    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-istio-envoy\") pod \"e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q\" (UID: \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\") " pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"
Jul 18 02:36:19.292078 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.292042    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-istio-token\") pod \"e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q\" (UID: \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\") " pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"
Jul 18 02:36:19.292266 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.292088    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-workload-certs\") pod \"e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q\" (UID: \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\") " pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"
Jul 18 02:36:19.292266 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.292122    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-workload-socket\") pod \"e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q\" (UID: \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\") " pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"
Jul 18 02:36:19.292266 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.292160    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-istio-envoy\") pod \"e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q\" (UID: \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\") " pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"
Jul 18 02:36:19.292266 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.292187    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-istio-podinfo\") pod \"e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q\" (UID: \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\") " pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"
Jul 18 02:36:19.292266 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.292205    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stnd5\" (UniqueName: \"kubernetes.io/projected/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-kube-api-access-stnd5\") pod \"e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q\" (UID: \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\") " pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"
Jul 18 02:36:19.292266 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.292232    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-credential-socket\") pod \"e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q\" (UID: \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\") " pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"
Jul 18 02:36:19.292266 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.292252    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-istio-data\") pod \"e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q\" (UID: \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\") " pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"
Jul 18 02:36:19.292690 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.292294    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-istiod-ca-cert\") pod \"e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q\" (UID: \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\") " pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"
Jul 18 02:36:19.292690 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.292620    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-workload-socket\") pod \"e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q\" (UID: \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\") " pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"
Jul 18 02:36:19.292796 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.292705    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-workload-certs\") pod \"e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q\" (UID: \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\") " pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"
Jul 18 02:36:19.292796 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.292719    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-credential-socket\") pod \"e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q\" (UID: \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\") " pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"
Jul 18 02:36:19.292894 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.292841    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-istio-data\") pod \"e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q\" (UID: \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\") " pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"
Jul 18 02:36:19.293084 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.293064    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-istiod-ca-cert\") pod \"e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q\" (UID: \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\") " pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"
Jul 18 02:36:19.294615 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.294593    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-istio-envoy\") pod \"e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q\" (UID: \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\") " pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"
Jul 18 02:36:19.294845 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.294825    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-istio-podinfo\") pod \"e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q\" (UID: \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\") " pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"
Jul 18 02:36:19.300640 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.300621    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-istio-token\") pod \"e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q\" (UID: \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\") " pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"
Jul 18 02:36:19.301036 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.301011    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stnd5\" (UniqueName: \"kubernetes.io/projected/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-kube-api-access-stnd5\") pod \"e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q\" (UID: \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\") " pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"
Jul 18 02:36:19.428654 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.428557    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"
Jul 18 02:36:19.570458 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.570413    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"]
Jul 18 02:36:19.573142 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:36:19.573101    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46d76d4e_20a8_4ef3_ad0c_9dd857fd6ff5.slice/crio-0129391bad15d0b019cc98edaa8ae1bd2e6087ffb4b1144bd927bd0861adbf1f WatchSource:0}: Error finding container 0129391bad15d0b019cc98edaa8ae1bd2e6087ffb4b1144bd927bd0861adbf1f: Status 404 returned error can't find the container with id 0129391bad15d0b019cc98edaa8ae1bd2e6087ffb4b1144bd927bd0861adbf1f
Jul 18 02:36:19.575891 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.575848    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 18 02:36:19.576016 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.575936    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 18 02:36:19.576016 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:19.575978    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 18 02:36:20.090804 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:20.090758    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q" event={"ID":"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5","Type":"ContainerStarted","Data":"07928c05e71d908f09527d84f49590e1ff69d8c6fc8de442d35ef65ca4c453a7"}
Jul 18 02:36:20.090981 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:20.090841    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q" event={"ID":"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5","Type":"ContainerStarted","Data":"0129391bad15d0b019cc98edaa8ae1bd2e6087ffb4b1144bd927bd0861adbf1f"}
Jul 18 02:36:20.111530 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:20.111480    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q" podStartSLOduration=1.111463503 podStartE2EDuration="1.111463503s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:36:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:36:20.109398783 +0000 UTC m=+2374.734435441" watchObservedRunningTime="2026-07-18 02:36:20.111463503 +0000 UTC m=+2374.736500170"
Jul 18 02:36:20.428802 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:20.428757    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"
Jul 18 02:36:20.433762 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:20.433734    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"
Jul 18 02:36:21.097388 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:21.097359    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"
Jul 18 02:36:21.098502 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:21.098481    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"
Jul 18 02:36:22.045064 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:22.045028    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-922cd9-84979994c7-jvswr"]
Jul 18 02:36:22.045835 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:22.045811    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cecbf79-c996-46bd-9dce-bd8110d10eb3" containerName="istio-proxy"
Jul 18 02:36:22.049381 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:22.049359    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-922cd9-84979994c7-jvswr"
Jul 18 02:36:22.051932 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:22.051908    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-922cd9-serving-cert\""
Jul 18 02:36:22.060924 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:22.060898    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-922cd9-84979994c7-jvswr"]
Jul 18 02:36:22.119507 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:22.119473    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d64e4196-779f-4785-8459-634d69fd9a29-maas-api-tls\") pod \"maas-api-e2e-shared-b-922cd9-84979994c7-jvswr\" (UID: \"d64e4196-779f-4785-8459-634d69fd9a29\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-922cd9-84979994c7-jvswr"
Jul 18 02:36:22.119672 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:22.119518    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d64e4196-779f-4785-8459-634d69fd9a29-openshift-service-ca\") pod \"maas-api-e2e-shared-b-922cd9-84979994c7-jvswr\" (UID: \"d64e4196-779f-4785-8459-634d69fd9a29\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-922cd9-84979994c7-jvswr"
Jul 18 02:36:22.119672 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:22.119654    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpfgn\" (UniqueName: \"kubernetes.io/projected/d64e4196-779f-4785-8459-634d69fd9a29-kube-api-access-lpfgn\") pod \"maas-api-e2e-shared-b-922cd9-84979994c7-jvswr\" (UID: \"d64e4196-779f-4785-8459-634d69fd9a29\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-922cd9-84979994c7-jvswr"
Jul 18 02:36:22.220814 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:22.220768    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d64e4196-779f-4785-8459-634d69fd9a29-maas-api-tls\") pod \"maas-api-e2e-shared-b-922cd9-84979994c7-jvswr\" (UID: \"d64e4196-779f-4785-8459-634d69fd9a29\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-922cd9-84979994c7-jvswr"
Jul 18 02:36:22.220989 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:22.220836    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d64e4196-779f-4785-8459-634d69fd9a29-openshift-service-ca\") pod \"maas-api-e2e-shared-b-922cd9-84979994c7-jvswr\" (UID: \"d64e4196-779f-4785-8459-634d69fd9a29\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-922cd9-84979994c7-jvswr"
Jul 18 02:36:22.220989 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:36:22.220850    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-922cd9-serving-cert: secret "maas-api-e2e-shared-b-922cd9-serving-cert" not found
Jul 18 02:36:22.220989 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:36:22.220912    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d64e4196-779f-4785-8459-634d69fd9a29-maas-api-tls podName:d64e4196-779f-4785-8459-634d69fd9a29 nodeName:}" failed. No retries permitted until 2026-07-18 02:36:22.720894286 +0000 UTC m=+2377.345930930 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d64e4196-779f-4785-8459-634d69fd9a29-maas-api-tls") pod "maas-api-e2e-shared-b-922cd9-84979994c7-jvswr" (UID: "d64e4196-779f-4785-8459-634d69fd9a29") : secret "maas-api-e2e-shared-b-922cd9-serving-cert" not found
Jul 18 02:36:22.221201 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:22.221022    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpfgn\" (UniqueName: \"kubernetes.io/projected/d64e4196-779f-4785-8459-634d69fd9a29-kube-api-access-lpfgn\") pod \"maas-api-e2e-shared-b-922cd9-84979994c7-jvswr\" (UID: \"d64e4196-779f-4785-8459-634d69fd9a29\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-922cd9-84979994c7-jvswr"
Jul 18 02:36:22.221579 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:22.221555    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d64e4196-779f-4785-8459-634d69fd9a29-openshift-service-ca\") pod \"maas-api-e2e-shared-b-922cd9-84979994c7-jvswr\" (UID: \"d64e4196-779f-4785-8459-634d69fd9a29\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-922cd9-84979994c7-jvswr"
Jul 18 02:36:22.231875 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:22.231854    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpfgn\" (UniqueName: \"kubernetes.io/projected/d64e4196-779f-4785-8459-634d69fd9a29-kube-api-access-lpfgn\") pod \"maas-api-e2e-shared-b-922cd9-84979994c7-jvswr\" (UID: \"d64e4196-779f-4785-8459-634d69fd9a29\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-922cd9-84979994c7-jvswr"
Jul 18 02:36:22.726328 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:22.726292    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d64e4196-779f-4785-8459-634d69fd9a29-maas-api-tls\") pod \"maas-api-e2e-shared-b-922cd9-84979994c7-jvswr\" (UID: \"d64e4196-779f-4785-8459-634d69fd9a29\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-922cd9-84979994c7-jvswr"
Jul 18 02:36:22.728755 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:22.728731    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d64e4196-779f-4785-8459-634d69fd9a29-maas-api-tls\") pod \"maas-api-e2e-shared-b-922cd9-84979994c7-jvswr\" (UID: \"d64e4196-779f-4785-8459-634d69fd9a29\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-922cd9-84979994c7-jvswr"
Jul 18 02:36:22.961559 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:22.961525    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-922cd9-84979994c7-jvswr"
Jul 18 02:36:23.107516 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:23.107473    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-922cd9-84979994c7-jvswr"]
Jul 18 02:36:23.110081 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:36:23.110033    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd64e4196_779f_4785_8459_634d69fd9a29.slice/crio-409ee1eb70d1d61d54dddc6a7c3ea2a1e41f77708b16d39169805d85567346a6 WatchSource:0}: Error finding container 409ee1eb70d1d61d54dddc6a7c3ea2a1e41f77708b16d39169805d85567346a6: Status 404 returned error can't find the container with id 409ee1eb70d1d61d54dddc6a7c3ea2a1e41f77708b16d39169805d85567346a6
Jul 18 02:36:24.124277 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:24.124238    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-922cd9-84979994c7-jvswr" event={"ID":"d64e4196-779f-4785-8459-634d69fd9a29","Type":"ContainerStarted","Data":"54a27f34fb3d736ea871f0f90fc19a3ac2c34dcfa3a5ce1cd1228f8e1ae829ed"}
Jul 18 02:36:24.124801 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:24.124322    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-922cd9-84979994c7-jvswr" event={"ID":"d64e4196-779f-4785-8459-634d69fd9a29","Type":"ContainerStarted","Data":"409ee1eb70d1d61d54dddc6a7c3ea2a1e41f77708b16d39169805d85567346a6"}
Jul 18 02:36:24.124801 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:24.124344    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-922cd9-84979994c7-jvswr"
Jul 18 02:36:24.140498 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:24.140401    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-922cd9-84979994c7-jvswr" podStartSLOduration=1.789664806 podStartE2EDuration="2.140385806s" totalImagesPullingTime="350.721ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:36:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:36:24.139400245 +0000 UTC m=+2378.764436912" watchObservedRunningTime="2026-07-18 02:36:24.140385806 +0000 UTC m=+2378.765422472"
Jul 18 02:36:30.135338 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:30.135307    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-922cd9-84979994c7-jvswr"
Jul 18 02:36:33.179090 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:33.179051    2576 generic.go:356] "Generic (PLEG): container finished" podID="46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5" containerID="07928c05e71d908f09527d84f49590e1ff69d8c6fc8de442d35ef65ca4c453a7" exitCode=0
Jul 18 02:36:33.179499 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:33.179113    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q" event={"ID":"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5","Type":"ContainerDied","Data":"07928c05e71d908f09527d84f49590e1ff69d8c6fc8de442d35ef65ca4c453a7"}
Jul 18 02:36:33.179543 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:33.179528    2576 scope.go:117] "RemoveContainer" containerID="07928c05e71d908f09527d84f49590e1ff69d8c6fc8de442d35ef65ca4c453a7"
Jul 18 02:36:33.180560 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:33.180531    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 18 02:36:33.180622 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:33.180606    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 18 02:36:33.180667 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:33.180639    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 18 02:36:34.190810 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:34.190764    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q" event={"ID":"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5","Type":"ContainerStarted","Data":"9f8c76307d869cff2d5b21438a1b0d38e2540cfa8235ed291c364e60d5b1ecf6"}
Jul 18 02:36:34.429629 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:34.429592    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"
Jul 18 02:36:34.431011 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:34.430987    2576 patch_prober.go:28] interesting pod/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Jul 18 02:36:34.431132 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:34.431050    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q" podUID="46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Jul 18 02:36:35.429709 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:35.429675    2576 patch_prober.go:28] interesting pod/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Jul 18 02:36:35.430174 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:35.429735    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q" podUID="46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Jul 18 02:36:36.098005 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:36.097962    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"
Jul 18 02:36:36.429718 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:36.429680    2576 patch_prober.go:28] interesting pod/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Jul 18 02:36:36.430123 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:36.429750    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q" podUID="46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Jul 18 02:36:37.429383 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:37.429346    2576 patch_prober.go:28] interesting pod/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Jul 18 02:36:37.429605 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:37.429406    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q" podUID="46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Jul 18 02:36:38.433694 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:38.433658    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"
Jul 18 02:36:38.434864 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:38.434844    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"
Jul 18 02:36:46.017254 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:46.017225    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-bfrdg_9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777/console-operator/1.log"
Jul 18 02:36:46.027778 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:46.027747    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-bfrdg_9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777/console-operator/1.log"
Jul 18 02:36:46.205043 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:46.205010    2576 scope.go:117] "RemoveContainer" containerID="d0b6341dece90a80d21138f4bb806ed2cec50b8ebbc548efcd0a81f896d892f3"
Jul 18 02:36:46.214365 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:36:46.214335    2576 scope.go:117] "RemoveContainer" containerID="ee50688b7fc931d16863170db7b682de962278a1464862bb32eaa213c757bc80"
Jul 18 02:41:46.055570 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:41:46.055487    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-bfrdg_9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777/console-operator/1.log"
Jul 18 02:41:46.060983 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:41:46.060961    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-bfrdg_9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777/console-operator/1.log"
Jul 18 02:41:46.274865 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:41:46.274829    2576 scope.go:117] "RemoveContainer" containerID="12218509f627ca050e666cbaad710db3b9eeadad70404c909b2d61cdc33718d4"
Jul 18 02:41:46.286940 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:41:46.286914    2576 scope.go:117] "RemoveContainer" containerID="7128bfcb60892b6862a6a38a735761a716ea469dd7c9d9ff32a12053cf7b23b7"
Jul 18 02:43:16.563352 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:16.563260    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0d6b43-x46wh"]
Jul 18 02:43:16.567293 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:16.567271    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0d6b43-x46wh"
Jul 18 02:43:16.569665 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:16.569643    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g69gp\""
Jul 18 02:43:16.573692 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:16.573660    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0d6b43-x46wh"]
Jul 18 02:43:16.624668 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:16.624627    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93316426-7bd9-461c-9b6c-e5eed9c157bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-0d6b43-x46wh\" (UID: \"93316426-7bd9-461c-9b6c-e5eed9c157bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0d6b43-x46wh"
Jul 18 02:43:16.725603 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:16.725557    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93316426-7bd9-461c-9b6c-e5eed9c157bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-0d6b43-x46wh\" (UID: \"93316426-7bd9-461c-9b6c-e5eed9c157bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0d6b43-x46wh"
Jul 18 02:43:16.726222 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:16.726200    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93316426-7bd9-461c-9b6c-e5eed9c157bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-0d6b43-x46wh\" (UID: \"93316426-7bd9-461c-9b6c-e5eed9c157bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0d6b43-x46wh"
Jul 18 02:43:16.880749 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:16.880715    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0d6b43-x46wh"
Jul 18 02:43:17.021718 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:17.021688    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0d6b43-x46wh"]
Jul 18 02:43:17.023322 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:43:17.023277    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93316426_7bd9_461c_9b6c_e5eed9c157bd.slice/crio-591ce282d7cc86b755a06aef4fc36e30fe04ddc2b4ea929a457954b24e82004e WatchSource:0}: Error finding container 591ce282d7cc86b755a06aef4fc36e30fe04ddc2b4ea929a457954b24e82004e: Status 404 returned error can't find the container with id 591ce282d7cc86b755a06aef4fc36e30fe04ddc2b4ea929a457954b24e82004e
Jul 18 02:43:17.302689 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:17.302650    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0d6b43-x46wh" event={"ID":"93316426-7bd9-461c-9b6c-e5eed9c157bd","Type":"ContainerStarted","Data":"66e346508e683abcc44a218e542af4a7af3c293240cfb1b57f6688ff3687d822"}
Jul 18 02:43:17.302878 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:17.302711    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0d6b43-x46wh" event={"ID":"93316426-7bd9-461c-9b6c-e5eed9c157bd","Type":"ContainerStarted","Data":"591ce282d7cc86b755a06aef4fc36e30fe04ddc2b4ea929a457954b24e82004e"}
Jul 18 02:43:17.316788 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:17.316645    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0d6b43-x46wh" podStartSLOduration=1.316625412 podStartE2EDuration="1.316625412s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:43:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:43:17.316093932 +0000 UTC m=+2791.941130609" watchObservedRunningTime="2026-07-18 02:43:17.316625412 +0000 UTC m=+2791.941662081"
Jul 18 02:43:18.309611 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:18.309576    2576 generic.go:356] "Generic (PLEG): container finished" podID="93316426-7bd9-461c-9b6c-e5eed9c157bd" containerID="66e346508e683abcc44a218e542af4a7af3c293240cfb1b57f6688ff3687d822" exitCode=0
Jul 18 02:43:18.310012 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:18.309663    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0d6b43-x46wh" event={"ID":"93316426-7bd9-461c-9b6c-e5eed9c157bd","Type":"ContainerDied","Data":"66e346508e683abcc44a218e542af4a7af3c293240cfb1b57f6688ff3687d822"}
Jul 18 02:43:19.454848 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:19.454818    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0d6b43-x46wh"
Jul 18 02:43:19.551572 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:19.551536    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93316426-7bd9-461c-9b6c-e5eed9c157bd-maas-api-service-ca\") pod \"93316426-7bd9-461c-9b6c-e5eed9c157bd\" (UID: \"93316426-7bd9-461c-9b6c-e5eed9c157bd\") "
Jul 18 02:43:19.551931 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:19.551911    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93316426-7bd9-461c-9b6c-e5eed9c157bd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "93316426-7bd9-461c-9b6c-e5eed9c157bd" (UID: "93316426-7bd9-461c-9b6c-e5eed9c157bd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:43:19.652777 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:19.652745    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93316426-7bd9-461c-9b6c-e5eed9c157bd-maas-api-service-ca\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:43:20.321634 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:20.321552    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0d6b43-x46wh"
Jul 18 02:43:20.321634 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:20.321566    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0d6b43-x46wh" event={"ID":"93316426-7bd9-461c-9b6c-e5eed9c157bd","Type":"ContainerDied","Data":"591ce282d7cc86b755a06aef4fc36e30fe04ddc2b4ea929a457954b24e82004e"}
Jul 18 02:43:20.321831 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:20.321649    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="591ce282d7cc86b755a06aef4fc36e30fe04ddc2b4ea929a457954b24e82004e"
Jul 18 02:43:31.657533 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:31.657494    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0d6b43-x46wh"]
Jul 18 02:43:31.661938 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:31.661914    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0d6b43-x46wh"]
Jul 18 02:43:31.970112 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:31.970017    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93316426-7bd9-461c-9b6c-e5eed9c157bd" path="/var/lib/kubelet/pods/93316426-7bd9-461c-9b6c-e5eed9c157bd/volumes"
Jul 18 02:43:48.823497 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:48.823460    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-922cd9-84979994c7-jvswr"]
Jul 18 02:43:48.824109 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:48.823690    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-922cd9-84979994c7-jvswr" podUID="d64e4196-779f-4785-8459-634d69fd9a29" containerName="maas-api" containerID="cri-o://54a27f34fb3d736ea871f0f90fc19a3ac2c34dcfa3a5ce1cd1228f8e1ae829ed" gracePeriod=30
Jul 18 02:43:49.274221 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:49.274185    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-922cd9-84979994c7-jvswr"
Jul 18 02:43:49.429128 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:49.429093    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d64e4196-779f-4785-8459-634d69fd9a29-openshift-service-ca\") pod \"d64e4196-779f-4785-8459-634d69fd9a29\" (UID: \"d64e4196-779f-4785-8459-634d69fd9a29\") "
Jul 18 02:43:49.429315 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:49.429144    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lpfgn\" (UniqueName: \"kubernetes.io/projected/d64e4196-779f-4785-8459-634d69fd9a29-kube-api-access-lpfgn\") pod \"d64e4196-779f-4785-8459-634d69fd9a29\" (UID: \"d64e4196-779f-4785-8459-634d69fd9a29\") "
Jul 18 02:43:49.429315 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:49.429265    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d64e4196-779f-4785-8459-634d69fd9a29-maas-api-tls\") pod \"d64e4196-779f-4785-8459-634d69fd9a29\" (UID: \"d64e4196-779f-4785-8459-634d69fd9a29\") "
Jul 18 02:43:49.429524 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:49.429499    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d64e4196-779f-4785-8459-634d69fd9a29-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d64e4196-779f-4785-8459-634d69fd9a29" (UID: "d64e4196-779f-4785-8459-634d69fd9a29"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:43:49.429630 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:49.429616    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d64e4196-779f-4785-8459-634d69fd9a29-openshift-service-ca\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:43:49.431265 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:49.431241    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d64e4196-779f-4785-8459-634d69fd9a29-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d64e4196-779f-4785-8459-634d69fd9a29" (UID: "d64e4196-779f-4785-8459-634d69fd9a29"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:43:49.431408 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:49.431388    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d64e4196-779f-4785-8459-634d69fd9a29-kube-api-access-lpfgn" (OuterVolumeSpecName: "kube-api-access-lpfgn") pod "d64e4196-779f-4785-8459-634d69fd9a29" (UID: "d64e4196-779f-4785-8459-634d69fd9a29"). InnerVolumeSpecName "kube-api-access-lpfgn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:43:49.479709 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:49.479674    2576 generic.go:356] "Generic (PLEG): container finished" podID="d64e4196-779f-4785-8459-634d69fd9a29" containerID="54a27f34fb3d736ea871f0f90fc19a3ac2c34dcfa3a5ce1cd1228f8e1ae829ed" exitCode=0
Jul 18 02:43:49.479877 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:49.479717    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-922cd9-84979994c7-jvswr"
Jul 18 02:43:49.479877 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:49.479763    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-922cd9-84979994c7-jvswr" event={"ID":"d64e4196-779f-4785-8459-634d69fd9a29","Type":"ContainerDied","Data":"54a27f34fb3d736ea871f0f90fc19a3ac2c34dcfa3a5ce1cd1228f8e1ae829ed"}
Jul 18 02:43:49.479877 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:49.479822    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-922cd9-84979994c7-jvswr" event={"ID":"d64e4196-779f-4785-8459-634d69fd9a29","Type":"ContainerDied","Data":"409ee1eb70d1d61d54dddc6a7c3ea2a1e41f77708b16d39169805d85567346a6"}
Jul 18 02:43:49.479877 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:49.479853    2576 scope.go:117] "RemoveContainer" containerID="54a27f34fb3d736ea871f0f90fc19a3ac2c34dcfa3a5ce1cd1228f8e1ae829ed"
Jul 18 02:43:49.489829 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:49.489813    2576 scope.go:117] "RemoveContainer" containerID="54a27f34fb3d736ea871f0f90fc19a3ac2c34dcfa3a5ce1cd1228f8e1ae829ed"
Jul 18 02:43:49.490069 ip-10-0-143-8 kubenswrapper[2576]: E0718 02:43:49.490049    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54a27f34fb3d736ea871f0f90fc19a3ac2c34dcfa3a5ce1cd1228f8e1ae829ed\": container with ID starting with 54a27f34fb3d736ea871f0f90fc19a3ac2c34dcfa3a5ce1cd1228f8e1ae829ed not found: ID does not exist" containerID="54a27f34fb3d736ea871f0f90fc19a3ac2c34dcfa3a5ce1cd1228f8e1ae829ed"
Jul 18 02:43:49.490116 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:49.490076    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54a27f34fb3d736ea871f0f90fc19a3ac2c34dcfa3a5ce1cd1228f8e1ae829ed"} err="failed to get container status \"54a27f34fb3d736ea871f0f90fc19a3ac2c34dcfa3a5ce1cd1228f8e1ae829ed\": rpc error: code = NotFound desc = could not find container \"54a27f34fb3d736ea871f0f90fc19a3ac2c34dcfa3a5ce1cd1228f8e1ae829ed\": container with ID starting with 54a27f34fb3d736ea871f0f90fc19a3ac2c34dcfa3a5ce1cd1228f8e1ae829ed not found: ID does not exist"
Jul 18 02:43:49.503926 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:49.503902    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-922cd9-84979994c7-jvswr"]
Jul 18 02:43:49.509010 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:49.508988    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-922cd9-84979994c7-jvswr"]
Jul 18 02:43:49.530321 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:49.530295    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d64e4196-779f-4785-8459-634d69fd9a29-maas-api-tls\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:43:49.530321 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:49.530322    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lpfgn\" (UniqueName: \"kubernetes.io/projected/d64e4196-779f-4785-8459-634d69fd9a29-kube-api-access-lpfgn\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:43:49.969803 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:43:49.969763    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d64e4196-779f-4785-8459-634d69fd9a29" path="/var/lib/kubelet/pods/d64e4196-779f-4785-8459-634d69fd9a29/volumes"
Jul 18 02:44:00.353174 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:00.353139    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"]
Jul 18 02:44:00.353702 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:00.353359    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q" podUID="46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5" containerName="istio-proxy" containerID="cri-o://9f8c76307d869cff2d5b21438a1b0d38e2540cfa8235ed291c364e60d5b1ecf6" gracePeriod=30
Jul 18 02:44:05.564999 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:05.564963    2576 generic.go:356] "Generic (PLEG): container finished" podID="46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5" containerID="9f8c76307d869cff2d5b21438a1b0d38e2540cfa8235ed291c364e60d5b1ecf6" exitCode=0
Jul 18 02:44:05.565381 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:05.565031    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q" event={"ID":"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5","Type":"ContainerDied","Data":"9f8c76307d869cff2d5b21438a1b0d38e2540cfa8235ed291c364e60d5b1ecf6"}
Jul 18 02:44:05.565381 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:05.565090    2576 scope.go:117] "RemoveContainer" containerID="07928c05e71d908f09527d84f49590e1ff69d8c6fc8de442d35ef65ca4c453a7"
Jul 18 02:44:05.622951 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:05.622890    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"
Jul 18 02:44:05.663839 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:05.663810    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-stnd5\" (UniqueName: \"kubernetes.io/projected/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-kube-api-access-stnd5\") pod \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\" (UID: \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\") "
Jul 18 02:44:05.664037 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:05.663846    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-istiod-ca-cert\") pod \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\" (UID: \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\") "
Jul 18 02:44:05.664037 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:05.663877    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-istio-data\") pod \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\" (UID: \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\") "
Jul 18 02:44:05.664037 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:05.663920    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-credential-socket\") pod \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\" (UID: \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\") "
Jul 18 02:44:05.664037 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:05.663941    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-istio-token\") pod \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\" (UID: \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\") "
Jul 18 02:44:05.664037 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:05.663995    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-istio-envoy\") pod \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\" (UID: \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\") "
Jul 18 02:44:05.664037 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:05.664036    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-istio-podinfo\") pod \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\" (UID: \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\") "
Jul 18 02:44:05.664389 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:05.664066    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-workload-certs\") pod \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\" (UID: \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\") "
Jul 18 02:44:05.664389 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:05.664123    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-workload-socket\") pod \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\" (UID: \"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5\") "
Jul 18 02:44:05.664389 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:05.664355    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5" (UID: "46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:44:05.664599 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:05.664567    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5" (UID: "46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:44:05.665078 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:05.664932    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5" (UID: "46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:44:05.665078 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:05.664987    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5" (UID: "46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:44:05.665286 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:05.665083    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-istio-data" (OuterVolumeSpecName: "istio-data") pod "46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5" (UID: "46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:44:05.666538 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:05.666505    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-istio-token" (OuterVolumeSpecName: "istio-token") pod "46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5" (UID: "46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:44:05.666632 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:05.666542    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-kube-api-access-stnd5" (OuterVolumeSpecName: "kube-api-access-stnd5") pod "46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5" (UID: "46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5"). InnerVolumeSpecName "kube-api-access-stnd5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:44:05.666632 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:05.666557    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5" (UID: "46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 18 02:44:05.666894 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:05.666871    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5" (UID: "46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:44:05.765507 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:05.765473    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-credential-socket\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:44:05.765507 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:05.765502    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-istio-token\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:44:05.765507 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:05.765513    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-istio-envoy\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:44:05.765753 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:05.765522    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-istio-podinfo\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:44:05.765753 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:05.765532    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-workload-certs\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:44:05.765753 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:05.765540    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-workload-socket\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:44:05.765753 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:05.765548    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-stnd5\" (UniqueName: \"kubernetes.io/projected/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-kube-api-access-stnd5\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:44:05.765753 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:05.765557    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-istiod-ca-cert\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:44:05.765753 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:05.765565    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5-istio-data\") on node \"ip-10-0-143-8.ec2.internal\" DevicePath \"\""
Jul 18 02:44:06.322199 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:06.322113    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-jvkvx_47c4d24f-e732-45d7-8fd9-51bae2a117a8/manager/0.log"
Jul 18 02:44:06.434480 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:06.434448    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-gx5mr_2f9ad3be-1156-45cd-924c-a3cb36bdef79/manager/0.log"
Jul 18 02:44:06.553212 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:06.553181    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-86c7b54d87-ks9zb_de375459-cd35-459b-bffd-1a82e0ece9a3/manager/0.log"
Jul 18 02:44:06.572155 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:06.572125    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"
Jul 18 02:44:06.572155 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:06.572132    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q" event={"ID":"46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5","Type":"ContainerDied","Data":"0129391bad15d0b019cc98edaa8ae1bd2e6087ffb4b1144bd927bd0861adbf1f"}
Jul 18 02:44:06.572725 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:06.572225    2576 scope.go:117] "RemoveContainer" containerID="9f8c76307d869cff2d5b21438a1b0d38e2540cfa8235ed291c364e60d5b1ecf6"
Jul 18 02:44:06.589120 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:06.589093    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"]
Jul 18 02:44:06.592442 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:06.592407    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-922cd9-openshift-default-67b86f9995-dst8q"]
Jul 18 02:44:06.662717 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:06.662695    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-c47pl_c123cde0-8a31-4ca0-823d-9a777092f96c/server/0.log"
Jul 18 02:44:06.776244 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:06.776212    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-pl52c_ea7eea6e-5b86-44c5-a721-c0823e3f305b/manager/0.log"
Jul 18 02:44:06.888859 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:06.888832    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d87dd68c9-bgz4c_fc672fd7-3070-477a-9817-00d9fce807d0/copy-manifests/0.log"
Jul 18 02:44:06.894913 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:06.894895    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d87dd68c9-bgz4c_fc672fd7-3070-477a-9817-00d9fce807d0/manager/0.log"
Jul 18 02:44:07.488054 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:07.488022    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5c45978f8c-zkfkw_3e782267-e161-40fd-90c3-2974b1fc3fd7/maas-api/0.log"
Jul 18 02:44:07.596155 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:07.596127    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29739015-b8zw8_c6b0d47f-f2db-4d94-aee7-952dfcbd5f42/cleanup/0.log"
Jul 18 02:44:07.698506 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:07.698467    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29739030-vfp6h_22302086-7e30-4a77-b188-0da393608dd5/cleanup/0.log"
Jul 18 02:44:07.824127 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:07.824049    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-qvntt_c8b3bc8c-aabb-48b1-9c9f-3915bc2095f1/postgres/0.log"
Jul 18 02:44:07.968752 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:07.968718    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5" path="/var/lib/kubelet/pods/46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5/volumes"
Jul 18 02:44:09.291374 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:09.291346    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-hlwwp_09763225-37ca-40e1-b910-8c7e45a3237e/manager/0.log"
Jul 18 02:44:09.639943 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:09.639914    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-7cw4l_9710b342-09a4-4302-b8f2-3fb1f380c7fd/manager/0.log"
Jul 18 02:44:10.338830 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:10.338799    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-gbvb8_30e6fba1-ee78-4462-aace-3adb6da9919b/discovery/0.log"
Jul 18 02:44:10.451821 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:10.451790    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-748465cbd5-42p4f_635497de-6dc6-4d91-bd8a-76287723a4e5/kube-auth-proxy/0.log"
Jul 18 02:44:10.804932 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:10.804902    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-mwlv7_65c0faa6-4db2-4c9b-a49c-001bbfe83956/payload-pre-processing/0.log"
Jul 18 02:44:11.042063 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:11.042033    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5db757d4dd-mt2r5_dc92e8cf-d868-4c6f-963b-04ff7caf51aa/router/0.log"
Jul 18 02:44:11.376058 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:11.376028    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-r9vj5_1eb761e4-f4f1-4cfe-a0aa-9e7c8a0db609/main/0.log"
Jul 18 02:44:20.457785 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:20.457737    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-jnffd_a370ba82-a51e-41aa-9ced-19bab3186de4/global-pull-secret-syncer/0.log"
Jul 18 02:44:20.693396 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:20.693367    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-t7kmc_ad82bb0f-ce9c-416d-b049-6e6e8ddfaba8/konnectivity-agent/0.log"
Jul 18 02:44:20.762771 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:20.762692    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-8.ec2.internal_ef3a0a5df8ac39131447779dfceb447b/haproxy/0.log"
Jul 18 02:44:26.275320 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:26.275281    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-hlwwp_09763225-37ca-40e1-b910-8c7e45a3237e/manager/0.log"
Jul 18 02:44:26.492257 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:26.492225    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-7cw4l_9710b342-09a4-4302-b8f2-3fb1f380c7fd/manager/0.log"
Jul 18 02:44:28.465352 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:28.465318    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-zhjgp_eaae66f5-7295-483b-a1d3-9077f4e9c82b/cluster-monitoring-operator/0.log"
Jul 18 02:44:28.687526 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:28.687493    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b4x4l_db90bbd1-feff-47f9-a0b5-f00cbe34503a/node-exporter/0.log"
Jul 18 02:44:28.709389 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:28.709361    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b4x4l_db90bbd1-feff-47f9-a0b5-f00cbe34503a/kube-rbac-proxy/0.log"
Jul 18 02:44:28.754559 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:28.754480    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b4x4l_db90bbd1-feff-47f9-a0b5-f00cbe34503a/init-textfile/0.log"
Jul 18 02:44:29.239812 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:29.239768    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-58f474ff5c-x87mb_432c67b4-af56-43e9-bccb-af44842443c7/telemeter-client/0.log"
Jul 18 02:44:29.270622 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:29.270596    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-58f474ff5c-x87mb_432c67b4-af56-43e9-bccb-af44842443c7/reload/0.log"
Jul 18 02:44:29.295850 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:29.295819    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-58f474ff5c-x87mb_432c67b4-af56-43e9-bccb-af44842443c7/kube-rbac-proxy/0.log"
Jul 18 02:44:30.340574 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:30.340542    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6vl8f/perf-node-gather-daemonset-gckqx"]
Jul 18 02:44:30.340974 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:30.340913    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d64e4196-779f-4785-8459-634d69fd9a29" containerName="maas-api"
Jul 18 02:44:30.340974 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:30.340924    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d64e4196-779f-4785-8459-634d69fd9a29" containerName="maas-api"
Jul 18 02:44:30.340974 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:30.340938    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93316426-7bd9-461c-9b6c-e5eed9c157bd" containerName="revoke-keys"
Jul 18 02:44:30.340974 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:30.340944    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="93316426-7bd9-461c-9b6c-e5eed9c157bd" containerName="revoke-keys"
Jul 18 02:44:30.340974 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:30.340954    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5" containerName="istio-proxy"
Jul 18 02:44:30.340974 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:30.340960    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5" containerName="istio-proxy"
Jul 18 02:44:30.340974 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:30.340973    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5" containerName="istio-proxy"
Jul 18 02:44:30.340974 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:30.340978    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5" containerName="istio-proxy"
Jul 18 02:44:30.341241 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:30.341033    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d64e4196-779f-4785-8459-634d69fd9a29" containerName="maas-api"
Jul 18 02:44:30.341241 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:30.341041    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="93316426-7bd9-461c-9b6c-e5eed9c157bd" containerName="revoke-keys"
Jul 18 02:44:30.341241 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:30.341046    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5" containerName="istio-proxy"
Jul 18 02:44:30.341241 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:30.341052    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="46d76d4e-20a8-4ef3-ad0c-9dd857fd6ff5" containerName="istio-proxy"
Jul 18 02:44:30.344143 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:30.344122    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-gckqx"
Jul 18 02:44:30.346462 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:30.346412    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6vl8f\"/\"kube-root-ca.crt\""
Jul 18 02:44:30.346462 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:30.346412    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6vl8f\"/\"openshift-service-ca.crt\""
Jul 18 02:44:30.346642 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:30.346473    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6vl8f\"/\"default-dockercfg-hrtqd\""
Jul 18 02:44:30.352673 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:30.352647    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6vl8f/perf-node-gather-daemonset-gckqx"]
Jul 18 02:44:30.493067 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:30.493032    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5c6b95d2-b668-41a4-86fa-7682451e5a12-sys\") pod \"perf-node-gather-daemonset-gckqx\" (UID: \"5c6b95d2-b668-41a4-86fa-7682451e5a12\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-gckqx"
Jul 18 02:44:30.493244 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:30.493073    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5c6b95d2-b668-41a4-86fa-7682451e5a12-lib-modules\") pod \"perf-node-gather-daemonset-gckqx\" (UID: \"5c6b95d2-b668-41a4-86fa-7682451e5a12\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-gckqx"
Jul 18 02:44:30.493244 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:30.493144    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5c6b95d2-b668-41a4-86fa-7682451e5a12-proc\") pod \"perf-node-gather-daemonset-gckqx\" (UID: \"5c6b95d2-b668-41a4-86fa-7682451e5a12\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-gckqx"
Jul 18 02:44:30.493244 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:30.493202    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5c6b95d2-b668-41a4-86fa-7682451e5a12-podres\") pod \"perf-node-gather-daemonset-gckqx\" (UID: \"5c6b95d2-b668-41a4-86fa-7682451e5a12\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-gckqx"
Jul 18 02:44:30.493244 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:30.493235    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdspr\" (UniqueName: \"kubernetes.io/projected/5c6b95d2-b668-41a4-86fa-7682451e5a12-kube-api-access-qdspr\") pod \"perf-node-gather-daemonset-gckqx\" (UID: \"5c6b95d2-b668-41a4-86fa-7682451e5a12\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-gckqx"
Jul 18 02:44:30.571292 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:30.571265    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-rxgl4_aa61e8ac-7c4a-4625-a8c8-97854f4fe565/networking-console-plugin/0.log"
Jul 18 02:44:30.594483 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:30.594388    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5c6b95d2-b668-41a4-86fa-7682451e5a12-sys\") pod \"perf-node-gather-daemonset-gckqx\" (UID: \"5c6b95d2-b668-41a4-86fa-7682451e5a12\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-gckqx"
Jul 18 02:44:30.594483 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:30.594448    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5c6b95d2-b668-41a4-86fa-7682451e5a12-lib-modules\") pod \"perf-node-gather-daemonset-gckqx\" (UID: \"5c6b95d2-b668-41a4-86fa-7682451e5a12\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-gckqx"
Jul 18 02:44:30.594483 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:30.594470    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5c6b95d2-b668-41a4-86fa-7682451e5a12-proc\") pod \"perf-node-gather-daemonset-gckqx\" (UID: \"5c6b95d2-b668-41a4-86fa-7682451e5a12\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-gckqx"
Jul 18 02:44:30.594723 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:30.594496    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5c6b95d2-b668-41a4-86fa-7682451e5a12-podres\") pod \"perf-node-gather-daemonset-gckqx\" (UID: \"5c6b95d2-b668-41a4-86fa-7682451e5a12\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-gckqx"
Jul 18 02:44:30.594723 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:30.594517    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5c6b95d2-b668-41a4-86fa-7682451e5a12-sys\") pod \"perf-node-gather-daemonset-gckqx\" (UID: \"5c6b95d2-b668-41a4-86fa-7682451e5a12\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-gckqx"
Jul 18 02:44:30.594723 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:30.594533    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdspr\" (UniqueName: \"kubernetes.io/projected/5c6b95d2-b668-41a4-86fa-7682451e5a12-kube-api-access-qdspr\") pod \"perf-node-gather-daemonset-gckqx\" (UID: \"5c6b95d2-b668-41a4-86fa-7682451e5a12\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-gckqx"
Jul 18 02:44:30.594723 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:30.594602    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5c6b95d2-b668-41a4-86fa-7682451e5a12-proc\") pod \"perf-node-gather-daemonset-gckqx\" (UID: \"5c6b95d2-b668-41a4-86fa-7682451e5a12\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-gckqx"
Jul 18 02:44:30.594723 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:30.594638    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5c6b95d2-b668-41a4-86fa-7682451e5a12-lib-modules\") pod \"perf-node-gather-daemonset-gckqx\" (UID: \"5c6b95d2-b668-41a4-86fa-7682451e5a12\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-gckqx"
Jul 18 02:44:30.594723 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:30.594657    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5c6b95d2-b668-41a4-86fa-7682451e5a12-podres\") pod \"perf-node-gather-daemonset-gckqx\" (UID: \"5c6b95d2-b668-41a4-86fa-7682451e5a12\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-gckqx"
Jul 18 02:44:30.603053 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:30.603026    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdspr\" (UniqueName: \"kubernetes.io/projected/5c6b95d2-b668-41a4-86fa-7682451e5a12-kube-api-access-qdspr\") pod \"perf-node-gather-daemonset-gckqx\" (UID: \"5c6b95d2-b668-41a4-86fa-7682451e5a12\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-gckqx"
Jul 18 02:44:30.654944 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:30.654913    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-gckqx"
Jul 18 02:44:30.811603 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:30.811490    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6vl8f/perf-node-gather-daemonset-gckqx"]
Jul 18 02:44:30.814052 ip-10-0-143-8 kubenswrapper[2576]: W0718 02:44:30.814024    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5c6b95d2_b668_41a4_86fa_7682451e5a12.slice/crio-fcaf119311a99428aabbf33d8106843d16cb99c4a1caddf14ed0709f1c565c15 WatchSource:0}: Error finding container fcaf119311a99428aabbf33d8106843d16cb99c4a1caddf14ed0709f1c565c15: Status 404 returned error can't find the container with id fcaf119311a99428aabbf33d8106843d16cb99c4a1caddf14ed0709f1c565c15
Jul 18 02:44:31.176302 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:31.176263    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-bfrdg_9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777/console-operator/1.log"
Jul 18 02:44:31.180885 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:31.180865    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-bfrdg_9ae8cf0a-bdff-4e54-b5b8-a9e64e2d0777/console-operator/2.log"
Jul 18 02:44:31.705103 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:31.705063    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-gckqx" event={"ID":"5c6b95d2-b668-41a4-86fa-7682451e5a12","Type":"ContainerStarted","Data":"9615a241109039fa3e0824beb7284dbac0036439ca328202c176586b13ecfa82"}
Jul 18 02:44:31.705630 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:31.705115    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-gckqx" event={"ID":"5c6b95d2-b668-41a4-86fa-7682451e5a12","Type":"ContainerStarted","Data":"fcaf119311a99428aabbf33d8106843d16cb99c4a1caddf14ed0709f1c565c15"}
Jul 18 02:44:31.705630 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:31.705188    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-gckqx"
Jul 18 02:44:31.724675 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:31.724620    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-gckqx" podStartSLOduration=1.72460521 podStartE2EDuration="1.72460521s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 02:44:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:44:31.722653721 +0000 UTC m=+2866.347690388" watchObservedRunningTime="2026-07-18 02:44:31.72460521 +0000 UTC m=+2866.349641877"
Jul 18 02:44:32.310942 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:32.310912    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-qgdh7_a40d1613-67f5-4603-a488-4895ddbe78f0/volume-data-source-validator/0.log"
Jul 18 02:44:33.377130 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:33.377100    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-kj28b_c9fa15c4-1db0-4b36-999d-5507f4cb2d19/dns/0.log"
Jul 18 02:44:33.399147 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:33.399120    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-kj28b_c9fa15c4-1db0-4b36-999d-5507f4cb2d19/kube-rbac-proxy/0.log"
Jul 18 02:44:33.480855 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:33.480823    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-25924_a3f34325-5076-4e41-a22d-5caece5dbece/dns-node-resolver/0.log"
Jul 18 02:44:34.101402 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:34.101365    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-f5kmg_33351d5c-d8c8-4d94-8209-28d15797b7a9/node-ca/0.log"
Jul 18 02:44:35.162198 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:35.162164    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-gbvb8_30e6fba1-ee78-4462-aace-3adb6da9919b/discovery/0.log"
Jul 18 02:44:35.184198 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:35.184169    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-748465cbd5-42p4f_635497de-6dc6-4d91-bd8a-76287723a4e5/kube-auth-proxy/0.log"
Jul 18 02:44:35.328563 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:35.328531    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-mwlv7_65c0faa6-4db2-4c9b-a49c-001bbfe83956/payload-pre-processing/0.log"
Jul 18 02:44:35.390462 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:35.390412    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5db757d4dd-mt2r5_dc92e8cf-d868-4c6f-963b-04ff7caf51aa/router/0.log"
Jul 18 02:44:36.097780 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:36.097742    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tw7tf_66623dd6-501f-42b2-b12a-188cf5520290/serve-healthcheck-canary/0.log"
Jul 18 02:44:36.561892 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:36.561858    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-69sww_dcf31fe7-21f9-4a41-9b37-2f8440a7c439/insights-operator/0.log"
Jul 18 02:44:36.562768 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:36.562747    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-69sww_dcf31fe7-21f9-4a41-9b37-2f8440a7c439/insights-operator/1.log"
Jul 18 02:44:36.646478 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:36.646448    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s78gl_c378a52c-b5db-4a92-bd55-8616ad887e45/kube-rbac-proxy/0.log"
Jul 18 02:44:36.666706 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:36.666677    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s78gl_c378a52c-b5db-4a92-bd55-8616ad887e45/exporter/0.log"
Jul 18 02:44:36.686998 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:36.686959    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s78gl_c378a52c-b5db-4a92-bd55-8616ad887e45/extractor/0.log"
Jul 18 02:44:37.720327 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:37.720302    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-gckqx"
Jul 18 02:44:38.936623 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:38.936594    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-jvkvx_47c4d24f-e732-45d7-8fd9-51bae2a117a8/manager/0.log"
Jul 18 02:44:39.104197 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:39.104165    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-gx5mr_2f9ad3be-1156-45cd-924c-a3cb36bdef79/manager/0.log"
Jul 18 02:44:39.216068 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:39.215995    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-86c7b54d87-ks9zb_de375459-cd35-459b-bffd-1a82e0ece9a3/manager/0.log"
Jul 18 02:44:39.238553 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:39.238527    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-c47pl_c123cde0-8a31-4ca0-823d-9a777092f96c/server/0.log"
Jul 18 02:44:39.315562 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:39.315526    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-pl52c_ea7eea6e-5b86-44c5-a721-c0823e3f305b/manager/0.log"
Jul 18 02:44:39.346090 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:39.346059    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d87dd68c9-bgz4c_fc672fd7-3070-477a-9817-00d9fce807d0/manager/0.log"
Jul 18 02:44:39.369546 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:39.369522    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d87dd68c9-bgz4c_fc672fd7-3070-477a-9817-00d9fce807d0/copy-manifests/0.log"
Jul 18 02:44:40.687323 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:40.687285    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-gbvb8_30e6fba1-ee78-4462-aace-3adb6da9919b/discovery/0.log"
Jul 18 02:44:40.696617 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:40.696593    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-748465cbd5-42p4f_635497de-6dc6-4d91-bd8a-76287723a4e5/kube-auth-proxy/0.log"
Jul 18 02:44:40.820747 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:40.820716    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-mwlv7_65c0faa6-4db2-4c9b-a49c-001bbfe83956/payload-pre-processing/0.log"
Jul 18 02:44:40.860679 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:40.860655    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5db757d4dd-mt2r5_dc92e8cf-d868-4c6f-963b-04ff7caf51aa/router/0.log"
Jul 18 02:44:40.918350 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:40.918318    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-shll7_f614ea92-0430-4e36-a587-d77ab9f14e92/openshift-lws-operator/0.log"
Jul 18 02:44:42.150677 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:42.150634    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-gbvb8_30e6fba1-ee78-4462-aace-3adb6da9919b/discovery/0.log"
Jul 18 02:44:42.163872 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:42.163849    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-748465cbd5-42p4f_635497de-6dc6-4d91-bd8a-76287723a4e5/kube-auth-proxy/0.log"
Jul 18 02:44:42.294115 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:42.294031    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-mwlv7_65c0faa6-4db2-4c9b-a49c-001bbfe83956/payload-pre-processing/0.log"
Jul 18 02:44:42.345892 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:42.345855    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5db757d4dd-mt2r5_dc92e8cf-d868-4c6f-963b-04ff7caf51aa/router/0.log"
Jul 18 02:44:46.811697 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:46.811663    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-tbs7d_28a9ce44-0900-4265-a0ee-67acdcfde240/kube-storage-version-migrator-operator/1.log"
Jul 18 02:44:46.812736 ip-10-0-143-8 kubenswrapper[2576]: I0718 02:44:46.812717    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-tbs7d_28a9ce44-0900-4265-a0ee-67acdcfde240/kube-storage-version-migrator-operator/0.log"