Jul 23 23:11:16.611035 ip-10-0-129-178 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 23 23:11:16.611045 ip-10-0-129-178 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 23 23:11:16.611052 ip-10-0-129-178 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 23 23:11:16.611521 ip-10-0-129-178 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 23 23:11:26.634230 ip-10-0-129-178 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 23 23:11:26.634247 ip-10-0-129-178 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 75d30e6fcdb64f229742683256a3e628 --
Jul 23 23:13:16.989565 ip-10-0-129-178 systemd[1]: Starting Kubernetes Kubelet...
Jul 23 23:13:17.305068 ip-10-0-129-178 kubenswrapper[2548]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 23:13:17.305068 ip-10-0-129-178 kubenswrapper[2548]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 23 23:13:17.305068 ip-10-0-129-178 kubenswrapper[2548]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 23:13:17.305068 ip-10-0-129-178 kubenswrapper[2548]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 23 23:13:17.305068 ip-10-0-129-178 kubenswrapper[2548]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 23:13:17.307091 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.307000    2548 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 23 23:13:17.310304 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310288    2548 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 23:13:17.310304 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310303    2548 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 23:13:17.310368 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310307    2548 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 23:13:17.310368 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310311    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 23:13:17.310368 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310314    2548 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 23:13:17.310368 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310317    2548 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 23:13:17.310368 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310320    2548 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 23:13:17.310368 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310323    2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 23:13:17.310368 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310325    2548 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 23:13:17.310368 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310328    2548 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 23:13:17.310368 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310331    2548 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 23:13:17.310368 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310334    2548 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 23:13:17.310368 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310336    2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 23:13:17.310368 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310341    2548 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 23:13:17.310368 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310345    2548 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 23:13:17.310368 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310348    2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 23:13:17.310368 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310351    2548 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 23:13:17.310368 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310354    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 23:13:17.310368 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310364    2548 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 23:13:17.310368 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310367    2548 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 23:13:17.310368 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310370    2548 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 23:13:17.310368 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310374    2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 23:13:17.310846 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310377    2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 23:13:17.310846 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310380    2548 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 23:13:17.310846 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310383    2548 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 23:13:17.310846 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310386    2548 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 23:13:17.310846 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310389    2548 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 23:13:17.310846 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310391    2548 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 23:13:17.310846 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310394    2548 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 23:13:17.310846 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310398    2548 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 23:13:17.310846 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310401    2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 23:13:17.310846 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310404    2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 23:13:17.310846 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310407    2548 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 23:13:17.310846 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310410    2548 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 23:13:17.310846 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310412    2548 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 23:13:17.310846 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310415    2548 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 23:13:17.310846 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310418    2548 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 23:13:17.310846 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310420    2548 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 23:13:17.310846 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310425    2548 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 23:13:17.310846 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310428    2548 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 23:13:17.310846 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310431    2548 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 23:13:17.311301 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310434    2548 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 23:13:17.311301 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310436    2548 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 23:13:17.311301 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310438    2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 23:13:17.311301 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310441    2548 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 23:13:17.311301 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310443    2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 23:13:17.311301 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310446    2548 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 23:13:17.311301 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310448    2548 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 23:13:17.311301 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310451    2548 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 23:13:17.311301 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310453    2548 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 23:13:17.311301 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310455    2548 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 23:13:17.311301 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310458    2548 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 23:13:17.311301 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310460    2548 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 23:13:17.311301 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310463    2548 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 23:13:17.311301 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310465    2548 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 23:13:17.311301 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310469    2548 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 23:13:17.311301 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310471    2548 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 23:13:17.311301 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310475    2548 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 23:13:17.311301 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310479    2548 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 23:13:17.311301 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310483    2548 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 23:13:17.311301 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310485    2548 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 23:13:17.311807 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310488    2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 23:13:17.311807 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310491    2548 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 23:13:17.311807 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310493    2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 23:13:17.311807 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310496    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 23:13:17.311807 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310498    2548 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 23:13:17.311807 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310501    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 23:13:17.311807 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310503    2548 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 23:13:17.311807 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310506    2548 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 23:13:17.311807 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310508    2548 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 23:13:17.311807 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310511    2548 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 23:13:17.311807 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310514    2548 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 23:13:17.311807 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310517    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 23:13:17.311807 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310520    2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 23:13:17.311807 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310523    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 23:13:17.311807 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310526    2548 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 23:13:17.311807 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310528    2548 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 23:13:17.311807 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310531    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 23:13:17.311807 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310535    2548 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 23:13:17.311807 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310537    2548 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 23:13:17.311807 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310540    2548 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 23:13:17.312302 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310542    2548 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 23:13:17.312302 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310545    2548 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 23:13:17.312302 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310548    2548 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 23:13:17.312302 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310551    2548 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 23:13:17.312302 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310553    2548 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 23:13:17.312302 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310555    2548 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 23:13:17.312302 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310559    2548 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 23:13:17.312302 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310562    2548 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 23:13:17.312302 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310564    2548 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 23:13:17.312302 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310567    2548 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 23:13:17.312302 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310570    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 23:13:17.312302 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310572    2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 23:13:17.312302 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310575    2548 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 23:13:17.312302 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310578    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 23:13:17.312302 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310580    2548 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 23:13:17.312302 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310583    2548 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 23:13:17.312302 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310585    2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 23:13:17.312302 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.310588    2548 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 23:13:17.312302 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311294    2548 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 23:13:17.312302 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311302    2548 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 23:13:17.312834 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311305    2548 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 23:13:17.312834 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311309    2548 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 23:13:17.312834 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311312    2548 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 23:13:17.312834 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311316    2548 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 23:13:17.312834 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311321    2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 23:13:17.312834 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311324    2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 23:13:17.312834 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311328    2548 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 23:13:17.312834 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311331    2548 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 23:13:17.312834 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311334    2548 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 23:13:17.312834 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311336    2548 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 23:13:17.312834 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311339    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 23:13:17.312834 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311342    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 23:13:17.312834 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311344    2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 23:13:17.312834 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311347    2548 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 23:13:17.312834 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311350    2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 23:13:17.312834 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311352    2548 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 23:13:17.312834 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311355    2548 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 23:13:17.312834 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311358    2548 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 23:13:17.312834 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311362    2548 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 23:13:17.313323 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311365    2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 23:13:17.313323 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311369    2548 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 23:13:17.313323 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311372    2548 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 23:13:17.313323 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311375    2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 23:13:17.313323 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311377    2548 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 23:13:17.313323 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311380    2548 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 23:13:17.313323 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311383    2548 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 23:13:17.313323 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311386    2548 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 23:13:17.313323 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311389    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 23:13:17.313323 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311391    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 23:13:17.313323 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311394    2548 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 23:13:17.313323 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311396    2548 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 23:13:17.313323 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311399    2548 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 23:13:17.313323 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311401    2548 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 23:13:17.313323 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311404    2548 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 23:13:17.313323 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311407    2548 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 23:13:17.313323 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311410    2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 23:13:17.313323 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311413    2548 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 23:13:17.313323 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311415    2548 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 23:13:17.313323 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311418    2548 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 23:13:17.313822 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311421    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 23:13:17.313822 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311423    2548 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 23:13:17.313822 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311426    2548 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 23:13:17.313822 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311428    2548 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 23:13:17.313822 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311431    2548 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 23:13:17.313822 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311434    2548 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 23:13:17.313822 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311436    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 23:13:17.313822 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311439    2548 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 23:13:17.313822 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311441    2548 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 23:13:17.313822 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311443    2548 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 23:13:17.313822 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311446    2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 23:13:17.313822 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311451    2548 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 23:13:17.313822 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311454    2548 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 23:13:17.313822 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311456    2548 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 23:13:17.313822 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311459    2548 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 23:13:17.313822 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311462    2548 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 23:13:17.313822 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311464    2548 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 23:13:17.313822 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311467    2548 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 23:13:17.313822 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311469    2548 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 23:13:17.313822 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311472    2548 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 23:13:17.314305 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311474    2548 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 23:13:17.314305 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311477    2548 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 23:13:17.314305 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311479    2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 23:13:17.314305 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311482    2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 23:13:17.314305 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311497    2548 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 23:13:17.314305 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311501    2548 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 23:13:17.314305 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311504    2548 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 23:13:17.314305 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311507    2548 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 23:13:17.314305 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311509    2548 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 23:13:17.314305 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311512    2548 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 23:13:17.314305 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311515    2548 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 23:13:17.314305 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311518    2548 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 23:13:17.314305 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311520    2548 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 23:13:17.314305 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311523    2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 23:13:17.314305 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311525    2548 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 23:13:17.314305 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311528    2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 23:13:17.314305 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311531    2548 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 23:13:17.314305 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311533    2548 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 23:13:17.314305 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311536    2548 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 23:13:17.314765 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311539    2548 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 23:13:17.314765 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311541    2548 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 23:13:17.314765 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311544    2548 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 23:13:17.314765 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311547    2548 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 23:13:17.314765 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311550    2548 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 23:13:17.314765 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311553    2548 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 23:13:17.314765 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311555    2548 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 23:13:17.314765 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311558    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 23:13:17.314765 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311560    2548 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 23:13:17.314765 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311563    2548 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 23:13:17.314765 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311566    2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 23:13:17.314765 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311569    2548 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 23:13:17.314765 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311572    2548 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 23:13:17.314765 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311574    2548 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 23:13:17.314765 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311577    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 23:13:17.314765 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311579    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 23:13:17.314765 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311582    2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 23:13:17.314765 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311584    2548 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 23:13:17.314765 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311587    2548 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 23:13:17.314765 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311980    2548 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 23:13:17.315234 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311985    2548 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 23:13:17.315234 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311988    2548 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 23:13:17.315234 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311990    2548 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 23:13:17.315234 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311994    2548 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 23:13:17.315234 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311996    2548 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 23:13:17.315234 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.311999    2548 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 23:13:17.315234 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312002    2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 23:13:17.315234 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312005    2548 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 23:13:17.315234 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312008    2548 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 23:13:17.315234 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312010    2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 23:13:17.315234 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312013    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 23:13:17.315234 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312015    2548 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 23:13:17.315234 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312018    2548 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 23:13:17.315234 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312020    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 23:13:17.315234 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312025    2548 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 23:13:17.315234 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312030    2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 23:13:17.315234 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312033    2548 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 23:13:17.315234 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312036    2548 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 23:13:17.315234 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312039    2548 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 23:13:17.315694 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312042    2548 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 23:13:17.315694 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312045    2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 23:13:17.315694 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312047    2548 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 23:13:17.315694 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312050    2548 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 23:13:17.315694 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312052    2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 23:13:17.315694 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312055    2548 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 23:13:17.315694 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312058    2548 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 23:13:17.315694 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312060    2548 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 23:13:17.315694 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312063    2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 23:13:17.315694 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312065    2548 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 23:13:17.315694 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312068    2548 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 23:13:17.315694 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312071    2548 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 23:13:17.315694 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312073    2548 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 23:13:17.315694 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312076    2548 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 23:13:17.315694 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312078    2548 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 23:13:17.315694 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312081    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 23:13:17.315694 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312083    2548 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 23:13:17.315694 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312086    2548 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 23:13:17.315694 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312088    2548 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 23:13:17.315694 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312091    2548 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 23:13:17.316157 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312094    2548 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 23:13:17.316157 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312096    2548 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 23:13:17.316157 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312099    2548 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 23:13:17.316157 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312102    2548 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 23:13:17.316157 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312105    2548 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 23:13:17.316157 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312108    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 23:13:17.316157 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312111    2548 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 23:13:17.316157 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312113    2548 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 23:13:17.316157 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312116    2548 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 23:13:17.316157 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312121    2548 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 23:13:17.316157 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312124    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 23:13:17.316157 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312127    2548 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 23:13:17.316157 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312130    2548 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 23:13:17.316157 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312132    2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 23:13:17.316157 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312135    2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 23:13:17.316157 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312138    2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 23:13:17.316157 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312141    2548 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 23:13:17.316157 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312143    2548 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 23:13:17.316157 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312146    2548 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 23:13:17.316157 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312149    2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 23:13:17.316642 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312151    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 23:13:17.316642 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312154    2548 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 23:13:17.316642 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312156    2548 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 23:13:17.316642 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312158    2548 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 23:13:17.316642 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312164    2548 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 23:13:17.316642 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312167    2548 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 23:13:17.316642 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312169    2548 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 23:13:17.316642 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312172    2548 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 23:13:17.316642 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312174    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 23:13:17.316642 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312178    2548 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 23:13:17.316642 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312181    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 23:13:17.316642 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312183    2548 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 23:13:17.316642 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312187    2548 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 23:13:17.316642 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312190    2548 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 23:13:17.316642 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312192    2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 23:13:17.316642 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312195    2548 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 23:13:17.316642 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312197    2548 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 23:13:17.316642 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312200    2548 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 23:13:17.316642 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312202    2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 23:13:17.316642 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312205    2548 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 23:13:17.317161 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312207    2548 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 23:13:17.317161 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312210    2548 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 23:13:17.317161 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312212    2548 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 23:13:17.317161 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312215    2548 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 23:13:17.317161 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312217    2548 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 23:13:17.317161 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312220    2548 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 23:13:17.317161 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312223    2548 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 23:13:17.317161 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312225    2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 23:13:17.317161 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312228    2548 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 23:13:17.317161 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312231    2548 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 23:13:17.317161 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312233    2548 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 23:13:17.317161 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312236    2548 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 23:13:17.317161 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312238    2548 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 23:13:17.317161 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312241    2548 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 23:13:17.317161 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312243    2548 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 23:13:17.317161 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312246    2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 23:13:17.317161 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312250    2548 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 23:13:17.317161 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312266    2548 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 23:13:17.317161 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312269    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 23:13:17.317161 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312343    2548 flags.go:64] FLAG: --address="0.0.0.0"
Jul 23 23:13:17.317646 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312350    2548 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 23 23:13:17.317646 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312365    2548 flags.go:64] FLAG: --anonymous-auth="true"
Jul 23 23:13:17.317646 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312370    2548 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 23 23:13:17.317646 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312375    2548 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 23 23:13:17.317646 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312380    2548 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 23 23:13:17.317646 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312385    2548 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 23 23:13:17.317646 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312389    2548 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 23 23:13:17.317646 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312392    2548 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 23 23:13:17.317646 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312395    2548 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 23 23:13:17.317646 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312399    2548 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 23 23:13:17.317646 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312402    2548 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 23 23:13:17.317646 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312405    2548 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 23 23:13:17.317646 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312408    2548 flags.go:64] FLAG: --cgroup-root=""
Jul 23 23:13:17.317646 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312411    2548 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 23 23:13:17.317646 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312414    2548 flags.go:64] FLAG: --client-ca-file=""
Jul 23 23:13:17.317646 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312417    2548 flags.go:64] FLAG: --cloud-provider="external"
Jul 23 23:13:17.317646 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312420    2548 flags.go:64] FLAG: --cluster-dns="[]"
Jul 23 23:13:17.317646 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312424    2548 flags.go:64] FLAG: --cluster-domain=""
Jul 23 23:13:17.317646 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312427    2548 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 23 23:13:17.317646 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312430    2548 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 23 23:13:17.317646 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312434    2548 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 23 23:13:17.317646 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312437    2548 flags.go:64] FLAG: --container-log-max-files="5"
Jul 23 23:13:17.317646 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312440    2548 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 23 23:13:17.317646 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312443    2548 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 23 23:13:17.318200 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312447    2548 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 23 23:13:17.318200 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312450    2548 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 23 23:13:17.318200 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312453    2548 flags.go:64] FLAG: --contention-profiling="false"
Jul 23 23:13:17.318200 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312456    2548 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 23 23:13:17.318200 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312460    2548 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 23 23:13:17.318200 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312463    2548 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 23 23:13:17.318200 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312466    2548 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 23 23:13:17.318200 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312475    2548 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 23 23:13:17.318200 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312478    2548 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 23 23:13:17.318200 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312481    2548 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 23 23:13:17.318200 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312484    2548 flags.go:64] FLAG: --enable-load-reader="false"
Jul 23 23:13:17.318200 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312487    2548 flags.go:64] FLAG: --enable-server="true"
Jul 23 23:13:17.318200 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312492    2548 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 23 23:13:17.318200 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312497    2548 flags.go:64] FLAG: --event-burst="100"
Jul 23 23:13:17.318200 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312500    2548 flags.go:64] FLAG: --event-qps="50"
Jul 23 23:13:17.318200 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312503    2548 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 23 23:13:17.318200 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312506    2548 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 23 23:13:17.318200 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312509    2548 flags.go:64] FLAG: --eviction-hard=""
Jul 23 23:13:17.318200 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312513    2548 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 23 23:13:17.318200 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312515    2548 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 23 23:13:17.318200 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312518    2548 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 23 23:13:17.318200 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312521    2548 flags.go:64] FLAG: --eviction-soft=""
Jul 23 23:13:17.318200 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312524    2548 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 23 23:13:17.318200 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312527    2548 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 23 23:13:17.318200 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312530    2548 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 23 23:13:17.318915 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312533    2548 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 23 23:13:17.318915 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312536    2548 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 23 23:13:17.318915 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312538    2548 flags.go:64] FLAG: --fail-swap-on="true"
Jul 23 23:13:17.318915 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312541    2548 flags.go:64] FLAG: --feature-gates=""
Jul 23 23:13:17.318915 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312545    2548 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 23 23:13:17.318915 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312548    2548 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 23 23:13:17.318915 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312551    2548 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 23 23:13:17.318915 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312554    2548 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 23 23:13:17.318915 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312557    2548 flags.go:64] FLAG: --healthz-port="10248"
Jul 23 23:13:17.318915 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312559    2548 flags.go:64] FLAG: --help="false"
Jul 23 23:13:17.318915 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312562    2548 flags.go:64] FLAG: --hostname-override="ip-10-0-129-178.ec2.internal"
Jul 23 23:13:17.318915 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312567    2548 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 23 23:13:17.318915 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312570    2548 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 23 23:13:17.318915 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312573    2548 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 23 23:13:17.318915 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312576    2548 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 23 23:13:17.318915 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312579    2548 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 23 23:13:17.318915 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312582    2548 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 23 23:13:17.318915 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312585    2548 flags.go:64] FLAG: --image-service-endpoint=""
Jul 23 23:13:17.318915 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312588    2548 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 23 23:13:17.318915 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312594    2548 flags.go:64] FLAG: --kube-api-burst="100"
Jul 23 23:13:17.318915 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312597    2548 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 23 23:13:17.318915 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312600    2548 flags.go:64] FLAG: --kube-api-qps="50"
Jul 23 23:13:17.318915 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312603    2548 flags.go:64] FLAG: --kube-reserved=""
Jul 23 23:13:17.318915 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312606    2548 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 23 23:13:17.319518 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312609    2548 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 23 23:13:17.319518 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312612    2548 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 23 23:13:17.319518 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312615    2548 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 23 23:13:17.319518 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312617    2548 flags.go:64] FLAG: --lock-file=""
Jul 23 23:13:17.319518 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312620    2548 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 23 23:13:17.319518 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312623    2548 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 23 23:13:17.319518 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312626    2548 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 23 23:13:17.319518 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312632    2548 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 23 23:13:17.319518 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312634    2548 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 23 23:13:17.319518 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312638    2548 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 23 23:13:17.319518 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312640    2548 flags.go:64] FLAG: --logging-format="text"
Jul 23 23:13:17.319518 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312643    2548 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 23 23:13:17.319518 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312646    2548 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 23 23:13:17.319518 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312649    2548 flags.go:64] FLAG: --manifest-url=""
Jul 23 23:13:17.319518 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312652    2548 flags.go:64] FLAG: --manifest-url-header=""
Jul 23 23:13:17.319518 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312656    2548 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 23 23:13:17.319518 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312660    2548 flags.go:64] FLAG: --max-open-files="1000000"
Jul 23 23:13:17.319518 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312664    2548 flags.go:64] FLAG: --max-pods="110"
Jul 23 23:13:17.319518 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312667    2548 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 23 23:13:17.319518 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312671    2548 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 23 23:13:17.319518 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312674    2548 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 23 23:13:17.319518 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312677    2548 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 23 23:13:17.319518 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312680    2548 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 23 23:13:17.319518 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312683    2548 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 23 23:13:17.319518 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312686    2548 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 23 23:13:17.320124 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312693    2548 flags.go:64] FLAG: --node-status-max-images="50"
Jul 23 23:13:17.320124 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312696    2548 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 23 23:13:17.320124 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312700    2548 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 23 23:13:17.320124 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312703    2548 flags.go:64] FLAG: --pod-cidr=""
Jul 23 23:13:17.320124 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312708    2548 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 23 23:13:17.320124 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312713    2548 flags.go:64] FLAG: --pod-manifest-path=""
Jul 23 23:13:17.320124 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312716    2548 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 23 23:13:17.320124 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312719    2548 flags.go:64] FLAG: --pods-per-core="0"
Jul 23 23:13:17.320124 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312722    2548 flags.go:64] FLAG: --port="10250"
Jul 23 23:13:17.320124 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312725    2548 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 23 23:13:17.320124 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312728    2548 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-043384842e0afae9b"
Jul 23 23:13:17.320124 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312731    2548 flags.go:64] FLAG: --qos-reserved=""
Jul 23 23:13:17.320124 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312734    2548 flags.go:64] FLAG: --read-only-port="10255"
Jul 23 23:13:17.320124 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312737    2548 flags.go:64] FLAG: --register-node="true"
Jul 23 23:13:17.320124 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312740    2548 flags.go:64] FLAG: --register-with-taints=""
Jul 23 23:13:17.320124 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312744    2548 flags.go:64] FLAG: --registry-burst="10"
Jul 23 23:13:17.320124 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312747    2548 flags.go:64] FLAG: --registry-qps="5"
Jul 23 23:13:17.320124 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312750    2548 flags.go:64] FLAG: --reserved-cpus=""
Jul 23 23:13:17.320124 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312752    2548 flags.go:64] FLAG: --reserved-memory=""
Jul 23 23:13:17.320124 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312756    2548 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 23 23:13:17.320124 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312759    2548 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 23 23:13:17.320124 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312762    2548 flags.go:64] FLAG: --rotate-certificates="false"
Jul 23 23:13:17.320124 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312765    2548 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 23 23:13:17.320124 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312768    2548 flags.go:64] FLAG: --runonce="false"
Jul 23 23:13:17.320124 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312771    2548 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 23 23:13:17.320742 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312774    2548 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 23 23:13:17.320742 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312778    2548 flags.go:64] FLAG: --seccomp-default="false"
Jul 23 23:13:17.320742 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312781    2548 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 23 23:13:17.320742 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312784    2548 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 23 23:13:17.320742 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312787    2548 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 23 23:13:17.320742 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312790    2548 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 23 23:13:17.320742 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312793    2548 flags.go:64] FLAG: --storage-driver-password="root"
Jul 23 23:13:17.320742 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312796    2548 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 23 23:13:17.320742 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312799    2548 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 23 23:13:17.320742 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312802    2548 flags.go:64] FLAG: --storage-driver-user="root"
Jul 23 23:13:17.320742 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312806    2548 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 23 23:13:17.320742 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312809    2548 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 23 23:13:17.320742 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312812    2548 flags.go:64] FLAG: --system-cgroups=""
Jul 23 23:13:17.320742 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312815    2548 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 23 23:13:17.320742 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312822    2548 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 23 23:13:17.320742 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312825    2548 flags.go:64] FLAG: --tls-cert-file=""
Jul 23 23:13:17.320742 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312827    2548 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 23 23:13:17.320742 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312831    2548 flags.go:64] FLAG: --tls-min-version=""
Jul 23 23:13:17.320742 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312834    2548 flags.go:64] FLAG: --tls-private-key-file=""
Jul 23 23:13:17.320742 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312837    2548 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 23 23:13:17.320742 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312840    2548 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 23 23:13:17.320742 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312843    2548 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 23 23:13:17.320742 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312846    2548 flags.go:64] FLAG: --v="2"
Jul 23 23:13:17.320742 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312850    2548 flags.go:64] FLAG: --version="false"
Jul 23 23:13:17.320742 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312855    2548 flags.go:64] FLAG: --vmodule=""
Jul 23 23:13:17.321350 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312858    2548 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 23 23:13:17.321350 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.312862    2548 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 23 23:13:17.321350 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.312996    2548 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 23:13:17.321350 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313000    2548 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 23:13:17.321350 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313003    2548 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 23:13:17.321350 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313006    2548 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 23:13:17.321350 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313009    2548 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 23:13:17.321350 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313011    2548 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 23:13:17.321350 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313016    2548 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 23:13:17.321350 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313020    2548 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 23:13:17.321350 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313023    2548 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 23:13:17.321350 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313026    2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 23:13:17.321350 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313029    2548 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 23:13:17.321350 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313031    2548 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 23:13:17.321350 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313034    2548 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 23:13:17.321350 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313037    2548 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 23:13:17.321350 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313039    2548 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 23:13:17.321350 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313044    2548 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 23:13:17.321350 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313047    2548 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 23:13:17.321350 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313049    2548 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 23:13:17.321828 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313052    2548 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 23:13:17.321828 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313055    2548 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 23:13:17.321828 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313058    2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 23:13:17.321828 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313060    2548 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 23:13:17.321828 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313063    2548 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 23:13:17.321828 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313066    2548 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 23:13:17.321828 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313068    2548 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 23:13:17.321828 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313071    2548 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 23:13:17.321828 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313074    2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 23:13:17.321828 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313076    2548 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 23:13:17.321828 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313079    2548 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 23:13:17.321828 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313081    2548 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 23:13:17.321828 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313084    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 23:13:17.321828 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313086    2548 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 23:13:17.321828 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313089    2548 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 23:13:17.321828 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313091    2548 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 23:13:17.321828 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313094    2548 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 23:13:17.321828 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313097    2548 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 23:13:17.321828 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313100    2548 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 23:13:17.321828 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313102    2548 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 23:13:17.322354 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313105    2548 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 23:13:17.322354 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313107    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 23:13:17.322354 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313110    2548 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 23:13:17.322354 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313113    2548 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 23:13:17.322354 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313115    2548 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 23:13:17.322354 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313118    2548 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 23:13:17.322354 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313120    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 23:13:17.322354 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313122    2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 23:13:17.322354 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313125    2548 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 23:13:17.322354 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313129    2548 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 23:13:17.322354 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313131    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 23:13:17.322354 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313134    2548 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 23:13:17.322354 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313136    2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 23:13:17.322354 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313139    2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 23:13:17.322354 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313142    2548 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 23:13:17.322354 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313145    2548 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 23:13:17.322354 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313148    2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 23:13:17.322354 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313150    2548 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 23:13:17.322354 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313153    2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 23:13:17.322354 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313155    2548 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 23:13:17.322855 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313158    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 23:13:17.322855 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313160    2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 23:13:17.322855 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313163    2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 23:13:17.322855 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313167    2548 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 23:13:17.322855 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313171    2548 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 23:13:17.322855 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313174    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 23:13:17.322855 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313177    2548 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 23:13:17.322855 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313179    2548 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 23:13:17.322855 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313182    2548 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 23:13:17.322855 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313185    2548 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 23:13:17.322855 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313187    2548 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 23:13:17.322855 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313190    2548 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 23:13:17.322855 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313192    2548 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 23:13:17.322855 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313195    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 23:13:17.322855 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313197    2548 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 23:13:17.322855 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313200    2548 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 23:13:17.322855 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313204    2548 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 23:13:17.322855 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313207    2548 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 23:13:17.322855 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313210    2548 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 23:13:17.323341 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313212    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 23:13:17.323341 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313215    2548 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 23:13:17.323341 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313218    2548 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 23:13:17.323341 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313221    2548 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 23:13:17.323341 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313224    2548 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 23:13:17.323341 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313226    2548 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 23:13:17.323341 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313229    2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 23:13:17.323341 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313231    2548 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 23:13:17.323341 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313234    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 23:13:17.323341 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313236    2548 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 23:13:17.323341 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313239    2548 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 23:13:17.323341 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313241    2548 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 23:13:17.323341 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313244    2548 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 23:13:17.323341 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313246    2548 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 23:13:17.323341 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313249    2548 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 23:13:17.323341 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313251    2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 23:13:17.323341 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313266    2548 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 23:13:17.323341 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313269    2548 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 23:13:17.323341 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313271    2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 23:13:17.323341 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313274    2548 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 23:13:17.323341 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313276    2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 23:13:17.323847 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.313279    2548 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 23:13:17.323847 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.314337    2548 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 23:13:17.323847 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.320810    2548 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 23 23:13:17.323847 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.320824    2548 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 23 23:13:17.323847 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.321332    2548 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 23 23:13:17.323847 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.321343    2548 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 23 23:13:17.323847 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321402    2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 23:13:17.323847 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321407    2548 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 23:13:17.323847 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321411    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 23:13:17.323847 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321414    2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 23:13:17.323847 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321417    2548 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 23:13:17.323847 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321420    2548 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 23:13:17.323847 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321422    2548 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 23:13:17.323847 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321425    2548 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 23:13:17.324194 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321430    2548 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 23:13:17.324194 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321434    2548 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 23:13:17.324194 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321437    2548 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 23:13:17.324194 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321440    2548 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 23:13:17.324194 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321443    2548 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 23:13:17.324194 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321446    2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 23:13:17.324194 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321450    2548 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 23:13:17.324194 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321453    2548 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 23:13:17.324194 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321457    2548 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 23:13:17.324194 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321460    2548 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 23:13:17.324194 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321463    2548 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 23:13:17.324194 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321465    2548 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 23:13:17.324194 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321469    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 23:13:17.324194 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321471    2548 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 23:13:17.324194 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321474    2548 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 23:13:17.324194 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321477    2548 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 23:13:17.324194 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321479    2548 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 23:13:17.324194 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321482    2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 23:13:17.324194 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321485    2548 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 23:13:17.324662 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321487    2548 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 23:13:17.324662 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321491    2548 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 23:13:17.324662 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321495    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 23:13:17.324662 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321499    2548 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 23:13:17.324662 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321502    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 23:13:17.324662 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321505    2548 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 23:13:17.324662 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321507    2548 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 23:13:17.324662 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321510    2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 23:13:17.324662 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321513    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 23:13:17.324662 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321516    2548 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 23:13:17.324662 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321518    2548 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 23:13:17.324662 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321521    2548 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 23:13:17.324662 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321524    2548 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 23:13:17.324662 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321526    2548 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 23:13:17.324662 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321529    2548 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 23:13:17.324662 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321532    2548 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 23:13:17.324662 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321534    2548 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 23:13:17.324662 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321537    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 23:13:17.324662 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321539    2548 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 23:13:17.324662 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321542    2548 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 23:13:17.325137 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321545    2548 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 23:13:17.325137 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321547    2548 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 23:13:17.325137 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321550    2548 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 23:13:17.325137 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321553    2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 23:13:17.325137 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321555    2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 23:13:17.325137 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321558    2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 23:13:17.325137 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321561    2548 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 23:13:17.325137 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321563    2548 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 23:13:17.325137 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321566    2548 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 23:13:17.325137 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321568    2548 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 23:13:17.325137 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321571    2548 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 23:13:17.325137 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321573    2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 23:13:17.325137 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321576    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 23:13:17.325137 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321579    2548 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 23:13:17.325137 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321581    2548 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 23:13:17.325137 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321585    2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 23:13:17.325137 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321587    2548 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 23:13:17.325137 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321590    2548 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 23:13:17.325137 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321593    2548 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 23:13:17.325137 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321595    2548 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 23:13:17.325753 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321598    2548 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 23:13:17.325753 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321600    2548 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 23:13:17.325753 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321603    2548 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 23:13:17.325753 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321605    2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 23:13:17.325753 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321608    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 23:13:17.325753 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321610    2548 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 23:13:17.325753 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321614    2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 23:13:17.325753 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321616    2548 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 23:13:17.325753 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321618    2548 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 23:13:17.325753 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321621    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 23:13:17.325753 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321623    2548 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 23:13:17.325753 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321626    2548 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 23:13:17.325753 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321628    2548 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 23:13:17.325753 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321631    2548 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 23:13:17.325753 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321633    2548 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 23:13:17.325753 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321635    2548 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 23:13:17.325753 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321638    2548 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 23:13:17.325753 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321640    2548 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 23:13:17.325753 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321643    2548 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 23:13:17.326237 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321645    2548 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 23:13:17.326237 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321648    2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 23:13:17.326237 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321650    2548 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 23:13:17.326237 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321653    2548 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 23:13:17.326237 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321655    2548 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 23:13:17.326237 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321657    2548 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 23:13:17.326237 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321660    2548 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 23:13:17.326237 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321663    2548 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 23:13:17.326237 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321665    2548 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 23:13:17.326237 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321669    2548 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 23:13:17.326237 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321672    2548 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 23:13:17.326237 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321674    2548 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 23:13:17.326237 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321677    2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 23:13:17.326237 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.321682    2548 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 23:13:17.326237 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321792    2548 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 23:13:17.326612 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321797    2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 23:13:17.326612 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321800    2548 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 23:13:17.326612 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321803    2548 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 23:13:17.326612 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321806    2548 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 23:13:17.326612 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321809    2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 23:13:17.326612 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321812    2548 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 23:13:17.326612 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321815    2548 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 23:13:17.326612 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321817    2548 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 23:13:17.326612 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321820    2548 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 23:13:17.326612 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321823    2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 23:13:17.326612 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321826    2548 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 23:13:17.326612 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321828    2548 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 23:13:17.326612 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321831    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 23:13:17.326612 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321834    2548 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 23:13:17.326612 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321837    2548 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 23:13:17.326612 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321840    2548 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 23:13:17.326612 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321843    2548 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 23:13:17.326612 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321846    2548 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 23:13:17.326612 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321849    2548 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 23:13:17.326612 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321851    2548 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 23:13:17.327083 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321854    2548 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 23:13:17.327083 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321857    2548 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 23:13:17.327083 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321860    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 23:13:17.327083 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321863    2548 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 23:13:17.327083 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321865    2548 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 23:13:17.327083 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321868    2548 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 23:13:17.327083 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321870    2548 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 23:13:17.327083 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321877    2548 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 23:13:17.327083 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321880    2548 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 23:13:17.327083 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321882    2548 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 23:13:17.327083 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321885    2548 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 23:13:17.327083 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321887    2548 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 23:13:17.327083 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321890    2548 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 23:13:17.327083 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321892    2548 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 23:13:17.327083 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321895    2548 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 23:13:17.327083 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321898    2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 23:13:17.327083 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321900    2548 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 23:13:17.327083 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321903    2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 23:13:17.327083 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321905    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 23:13:17.327083 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321908    2548 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 23:13:17.327736 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321910    2548 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 23:13:17.327736 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321913    2548 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 23:13:17.327736 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321915    2548 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 23:13:17.327736 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321919    2548 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 23:13:17.327736 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321923    2548 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 23:13:17.327736 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321926    2548 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 23:13:17.327736 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321928    2548 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 23:13:17.327736 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321932    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 23:13:17.327736 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321935    2548 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 23:13:17.327736 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321938    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 23:13:17.327736 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321940    2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 23:13:17.327736 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321943    2548 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 23:13:17.327736 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321946    2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 23:13:17.327736 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321948    2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 23:13:17.327736 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321951    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 23:13:17.327736 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321953    2548 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 23:13:17.327736 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321956    2548 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 23:13:17.327736 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321958    2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 23:13:17.327736 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321961    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 23:13:17.327736 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321963    2548 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 23:13:17.328539 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321966    2548 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 23:13:17.328539 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321969    2548 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 23:13:17.328539 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321972    2548 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 23:13:17.328539 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321974    2548 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 23:13:17.328539 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321977    2548 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 23:13:17.328539 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321979    2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 23:13:17.328539 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321982    2548 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 23:13:17.328539 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321985    2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 23:13:17.328539 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321988    2548 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 23:13:17.328539 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321991    2548 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 23:13:17.328539 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321994    2548 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 23:13:17.328539 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.321998    2548 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 23:13:17.328539 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.322001    2548 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 23:13:17.328539 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.322004    2548 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 23:13:17.328539 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.322007    2548 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 23:13:17.328539 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.322009    2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 23:13:17.328539 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.322012    2548 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 23:13:17.328539 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.322014    2548 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 23:13:17.328539 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.322017    2548 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 23:13:17.329098 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.322019    2548 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 23:13:17.329098 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.322022    2548 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 23:13:17.329098 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.322024    2548 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 23:13:17.329098 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.322027    2548 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 23:13:17.329098 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.322030    2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 23:13:17.329098 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.322032    2548 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 23:13:17.329098 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.322035    2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 23:13:17.329098 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.322037    2548 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 23:13:17.329098 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.322040    2548 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 23:13:17.329098 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.322042    2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 23:13:17.329098 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.322044    2548 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 23:13:17.329098 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.322047    2548 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 23:13:17.329098 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.322052    2548 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 23:13:17.329098 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.322055    2548 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 23:13:17.329098 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.322058    2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 23:13:17.329098 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.322061    2548 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 23:13:17.329098 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.322063    2548 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 23:13:17.329098 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.322065    2548 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 23:13:17.329098 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:17.322068    2548 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 23:13:17.329707 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.322073    2548 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 23:13:17.329707 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.322175    2548 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 23 23:13:17.329707 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.324192    2548 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 23 23:13:17.329707 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.325289    2548 server.go:1019] "Starting client certificate rotation"
Jul 23 23:13:17.329707 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.325395    2548 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 23 23:13:17.329707 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.325460    2548 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 23 23:13:17.329707 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.328939    2548 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 23 23:13:17.330677 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.330660    2548 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 23 23:13:17.338926 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.338906    2548 log.go:25] "Validated CRI v1 runtime API"
Jul 23 23:13:17.341375 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.341357    2548 log.go:25] "Validated CRI v1 image API"
Jul 23 23:13:17.344288 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.344272    2548 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 23 23:13:17.348702 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.348682    2548 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 a6f26431-3fce-4f65-9691-63634876a87a:/dev/nvme0n1p3 e95caabd-0b3f-48af-8a7e-0f0b5622fc71:/dev/nvme0n1p4]
Jul 23 23:13:17.348779 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.348701    2548 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 23 23:13:17.354853 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.354704    2548 manager.go:216] Machine: {Timestamp:2026-07-23 23:13:17.352827992 +0000 UTC m=+0.267275102 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100362 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c3208a1e12d57385a05bc524a1be9 SystemUUID:ec2c3208-a1e1-2d57-385a-05bc524a1be9 BootID:75d30e6f-cdb6-4f22-9742-683256a3e628 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:cd:de:22:42:5f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:cd:de:22:42:5f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f6:db:bb:cf:17:47 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 23 23:13:17.354853 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.354844    2548 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 23 23:13:17.354992 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.354922    2548 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 23 23:13:17.357240 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.357209    2548 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 23 23:13:17.357452 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.357243    2548 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-178.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 23 23:13:17.357526 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.357482    2548 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 23 23:13:17.358072 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.358060    2548 topology_manager.go:138] "Creating topology manager with none policy"
Jul 23 23:13:17.358120 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.358079    2548 container_manager_linux.go:309] "Creating device plugin manager"
Jul 23 23:13:17.358120 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.358090    2548 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 23 23:13:17.358194 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.358118    2548 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 23 23:13:17.358874 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.358861    2548 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 23 23:13:17.361489 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.361475    2548 state_mem.go:36] "Initialized new in-memory state store"
Jul 23 23:13:17.361624 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.361613    2548 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 23 23:13:17.361690 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.361653    2548 kubelet.go:486] "Attempting to sync node with API server"
Jul 23 23:13:17.361690 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.361668    2548 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 23 23:13:17.361690 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.361687    2548 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 23 23:13:17.361809 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.361702    2548 kubelet.go:398] "Adding apiserver pod source"
Jul 23 23:13:17.361809 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.361715    2548 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 23 23:13:17.362184 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.362168    2548 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 23 23:13:17.362278 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.362202    2548 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 23 23:13:17.364561 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.364544    2548 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 23 23:13:17.368461 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.368439    2548 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 23 23:13:17.368524 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.368476    2548 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 23 23:13:17.369790 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.369777    2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 23 23:13:17.369850 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.369793    2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 23 23:13:17.369850 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.369800    2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 23 23:13:17.369850 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.369806    2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 23 23:13:17.369850 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.369815    2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 23 23:13:17.369850 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.369821    2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 23 23:13:17.369850 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.369829    2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 23 23:13:17.369850 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.369835    2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 23 23:13:17.369850 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.369842    2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 23 23:13:17.369850 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.369849    2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 23 23:13:17.370073 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.369857    2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 23 23:13:17.370073 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.369863    2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 23 23:13:17.370708 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.370694    2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 23 23:13:17.370739 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.370713    2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 23 23:13:17.373523 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:17.373496    2548 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-178.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 23 23:13:17.373523 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.373506    2548 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-178.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 23 23:13:17.373667 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:17.373555    2548 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 23 23:13:17.374004 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.373991    2548 server.go:1268] "Started kubelet"
Jul 23 23:13:17.374155 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.374073    2548 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 23 23:13:17.374155 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.374109    2548 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 23 23:13:17.374329 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.374275    2548 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 23 23:13:17.374913 ip-10-0-129-178 systemd[1]: Started Kubernetes Kubelet.
Jul 23 23:13:17.375766 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.375749    2548 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 23 23:13:17.376485 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.376471    2548 server.go:310] "Adding debug handlers to kubelet server"
Jul 23 23:13:17.378680 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:17.377919    2548 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-178.ec2.internal.18c50e292f667f18  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-178.ec2.internal,UID:ip-10-0-129-178.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-178.ec2.internal,},FirstTimestamp:2026-07-23 23:13:17.373964056 +0000 UTC m=+0.288411166,LastTimestamp:2026-07-23 23:13:17.373964056 +0000 UTC m=+0.288411166,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-178.ec2.internal,}"
Jul 23 23:13:17.379044 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:17.379021    2548 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 23 23:13:17.383360 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.383344    2548 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 23 23:13:17.383429 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.383343    2548 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 23 23:13:17.383680 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:17.383513    2548 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-178.ec2.internal\" not found"
Jul 23 23:13:17.383987 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.383949    2548 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 23 23:13:17.383987 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.383954    2548 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 23 23:13:17.383987 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.383988    2548 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 23 23:13:17.384156 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.384100    2548 reconstruct.go:97] "Volume reconstruction finished"
Jul 23 23:13:17.384156 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.384108    2548 reconciler.go:29] "Reconciler: start to sync state"
Jul 23 23:13:17.384338 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.384319    2548 factory.go:55] Registering systemd factory
Jul 23 23:13:17.384338 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.384339    2548 factory.go:223] Registration of the systemd container factory successfully
Jul 23 23:13:17.384565 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.384552    2548 factory.go:153] Registering CRI-O factory
Jul 23 23:13:17.384565 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.384568    2548 factory.go:223] Registration of the crio container factory successfully
Jul 23 23:13:17.384663 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.384612    2548 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 23 23:13:17.384663 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.384647    2548 factory.go:103] Registering Raw factory
Jul 23 23:13:17.384663 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.384665    2548 manager.go:1196] Started watching for new ooms in manager
Jul 23 23:13:17.385247 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.385232    2548 manager.go:349] Starting recovery of all containers
Jul 23 23:13:17.387742 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:17.387707    2548 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-178.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 23 23:13:17.387825 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:17.387778    2548 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 23 23:13:17.389735 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.389705    2548 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-msw6f"
Jul 23 23:13:17.397272 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.397233    2548 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-msw6f"
Jul 23 23:13:17.398138 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.398100    2548 manager.go:354] Recovery completed
Jul 23 23:13:17.402363 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.402349    2548 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 23:13:17.404673 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.404660    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-178.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 23:13:17.404725 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.404687    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-178.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 23:13:17.404725 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.404697    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-178.ec2.internal" event="NodeHasSufficientPID"
Jul 23 23:13:17.405195 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.405183    2548 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 23 23:13:17.405195 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.405194    2548 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 23 23:13:17.405352 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.405210    2548 state_mem.go:36] "Initialized new in-memory state store"
Jul 23 23:13:17.405352 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.405249    2548 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 23:13:17.406044 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.406027    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-178.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 23:13:17.406125 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.406053    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-178.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 23:13:17.406125 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.406067    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-178.ec2.internal" event="NodeHasSufficientPID"
Jul 23 23:13:17.406887 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:17.406822    2548 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-178.ec2.internal.18c50e29313b1819  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-178.ec2.internal,UID:ip-10-0-129-178.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-178.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-178.ec2.internal,},FirstTimestamp:2026-07-23 23:13:17.404674073 +0000 UTC m=+0.319121182,LastTimestamp:2026-07-23 23:13:17.404674073 +0000 UTC m=+0.319121182,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-178.ec2.internal,}"
Jul 23 23:13:17.407740 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.407720    2548 policy_none.go:49] "None policy: Start"
Jul 23 23:13:17.407740 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.407740    2548 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 23 23:13:17.407868 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.407755    2548 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 23 23:13:17.409753 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.409742    2548 policy_none.go:47] "Start"
Jul 23 23:13:17.453956 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.453813    2548 manager.go:337] "Starting Device Plugin manager"
Jul 23 23:13:17.454063 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:17.453981    2548 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 23 23:13:17.454063 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.453995    2548 server.go:87] "Starting device plugin registration server"
Jul 23 23:13:17.454302 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.454290    2548 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 23 23:13:17.454348 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.454306    2548 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 23 23:13:17.454437 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.454405    2548 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 23 23:13:17.454538 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.454519    2548 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 23 23:13:17.454538 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.454538    2548 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 23 23:13:17.455020 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:17.455002    2548 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 23 23:13:17.455123 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:17.455044    2548 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-178.ec2.internal\" not found"
Jul 23 23:13:17.554458 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.554431    2548 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 23:13:17.556844 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.556801    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-178.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 23:13:17.556844 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.556834    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-178.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 23:13:17.556844 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.556845    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-178.ec2.internal" event="NodeHasSufficientPID"
Jul 23 23:13:17.556980 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.556872    2548 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-178.ec2.internal"
Jul 23 23:13:17.561713 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.561698    2548 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-178.ec2.internal"
Jul 23 23:13:17.561757 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:17.561721    2548 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-178.ec2.internal\": node \"ip-10-0-129-178.ec2.internal\" not found"
Jul 23 23:13:17.569196 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.569173    2548 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 23 23:13:17.570495 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.570474    2548 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 23 23:13:17.570579 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.570507    2548 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 23 23:13:17.570579 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.570538    2548 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 23 23:13:17.570579 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:17.570569    2548 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 23 23:13:17.572285 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.572247    2548 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 23:13:17.575049 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:17.575032    2548 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-178.ec2.internal\" not found"
Jul 23 23:13:17.670716 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.670648    2548 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-178.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-178.ec2.internal"]
Jul 23 23:13:17.670865 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.670761    2548 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 23:13:17.672646 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.672628    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-178.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 23:13:17.672761 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.672658    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-178.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 23:13:17.672761 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.672668    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-178.ec2.internal" event="NodeHasSufficientPID"
Jul 23 23:13:17.675055 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.675043    2548 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 23:13:17.675146 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:17.675128    2548 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-178.ec2.internal\" not found"
Jul 23 23:13:17.675190 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.675173    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-178.ec2.internal"
Jul 23 23:13:17.675232 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.675224    2548 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 23:13:17.681611 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.681594    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-178.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 23:13:17.681699 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.681596    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-178.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 23:13:17.681699 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.681661    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-178.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 23:13:17.681699 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.681673    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-178.ec2.internal" event="NodeHasSufficientPID"
Jul 23 23:13:17.681699 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.681623    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-178.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 23:13:17.681812 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.681704    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-178.ec2.internal" event="NodeHasSufficientPID"
Jul 23 23:13:17.683993 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.683979    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-178.ec2.internal"
Jul 23 23:13:17.684088 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.684000    2548 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 23:13:17.684802 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.684774    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-178.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 23:13:17.684802 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.684801    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-178.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 23:13:17.684952 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.684815    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-178.ec2.internal" event="NodeHasSufficientPID"
Jul 23 23:13:17.686169 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.686147    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9ba0f66b58401a0fb93564ba1a377674-config\") pod \"kube-apiserver-proxy-ip-10-0-129-178.ec2.internal\" (UID: \"9ba0f66b58401a0fb93564ba1a377674\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-178.ec2.internal"
Jul 23 23:13:17.686225 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.686188    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/45402979716ef826bbc95cd236f845a5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-178.ec2.internal\" (UID: \"45402979716ef826bbc95cd236f845a5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-178.ec2.internal"
Jul 23 23:13:17.686278 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.686233    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/45402979716ef826bbc95cd236f845a5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-178.ec2.internal\" (UID: \"45402979716ef826bbc95cd236f845a5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-178.ec2.internal"
Jul 23 23:13:17.710933 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:17.710913    2548 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-178.ec2.internal\" not found" node="ip-10-0-129-178.ec2.internal"
Jul 23 23:13:17.715213 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:17.715198    2548 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-178.ec2.internal\" not found" node="ip-10-0-129-178.ec2.internal"
Jul 23 23:13:17.776208 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:17.776183    2548 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-178.ec2.internal\" not found"
Jul 23 23:13:17.786702 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.786680    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/45402979716ef826bbc95cd236f845a5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-178.ec2.internal\" (UID: \"45402979716ef826bbc95cd236f845a5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-178.ec2.internal"
Jul 23 23:13:17.786772 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.786708    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/45402979716ef826bbc95cd236f845a5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-178.ec2.internal\" (UID: \"45402979716ef826bbc95cd236f845a5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-178.ec2.internal"
Jul 23 23:13:17.786772 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.786724    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9ba0f66b58401a0fb93564ba1a377674-config\") pod \"kube-apiserver-proxy-ip-10-0-129-178.ec2.internal\" (UID: \"9ba0f66b58401a0fb93564ba1a377674\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-178.ec2.internal"
Jul 23 23:13:17.786772 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.786750    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9ba0f66b58401a0fb93564ba1a377674-config\") pod \"kube-apiserver-proxy-ip-10-0-129-178.ec2.internal\" (UID: \"9ba0f66b58401a0fb93564ba1a377674\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-178.ec2.internal"
Jul 23 23:13:17.786891 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.786774    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/45402979716ef826bbc95cd236f845a5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-178.ec2.internal\" (UID: \"45402979716ef826bbc95cd236f845a5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-178.ec2.internal"
Jul 23 23:13:17.786891 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:17.786794    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/45402979716ef826bbc95cd236f845a5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-178.ec2.internal\" (UID: \"45402979716ef826bbc95cd236f845a5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-178.ec2.internal"
Jul 23 23:13:19.047901 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:19.047858    2548 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-178.ec2.internal\" not found"
Jul 23 23:13:19.148413 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:19.148371    2548 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-178.ec2.internal\" not found"
Jul 23 23:13:19.183919 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.183888    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-178.ec2.internal"
Jul 23 23:13:19.188729 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.188711    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-178.ec2.internal"
Jul 23 23:13:19.248856 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:19.248821    2548 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-178.ec2.internal\" not found"
Jul 23 23:13:19.349323 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:19.349259    2548 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-178.ec2.internal\" not found"
Jul 23 23:13:19.435425 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.435397    2548 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 23:13:19.455266 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.455244    2548 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-178.ec2.internal"
Jul 23 23:13:19.465670 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.465652    2548 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 23 23:13:19.467134 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.467121    2548 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-178.ec2.internal"
Jul 23 23:13:19.475121 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.475107    2548 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 23 23:13:19.496262 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.496235    2548 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 23 23:13:19.496700 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.496409    2548 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 23 23:13:19.496700 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.496417    2548 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 23 23:13:19.533572 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.533541    2548 apiserver.go:52] "Watching apiserver"
Jul 23 23:13:19.550349 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.550315    2548 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 23 23:13:19.551336 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.551311    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sst5n","openshift-cluster-node-tuning-operator/tuned-cwd9t","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-178.ec2.internal","openshift-multus/multus-additional-cni-plugins-wk67r","openshift-multus/multus-bvgxv","openshift-network-operator/iptables-alerter-gqbl6","kube-system/global-pull-secret-syncer-95x9c","kube-system/kube-apiserver-proxy-ip-10-0-129-178.ec2.internal","openshift-dns/node-resolver-mth7q","openshift-image-registry/node-ca-rdphw","openshift-multus/network-metrics-daemon-fkdcf","openshift-network-diagnostics/network-check-target-fbnfl","openshift-ovn-kubernetes/ovnkube-node-55rnx","kube-system/konnectivity-agent-b6n2c"]
Jul 23 23:13:19.554415 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.554386    2548 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 23 23:13:19.554684 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.554662    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:13:19.554790 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:19.554762    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95x9c" podUID="f98e0ab3-2bd3-44a8-9132-6f3521075ba5"
Jul 23 23:13:19.557007 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.556989    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.559351 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.559333    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 23 23:13:19.559574 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.559561    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 23 23:13:19.559648 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.559566    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 23 23:13:19.559648 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.559592    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 23 23:13:19.559752 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.559726    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-b6n2c"
Jul 23 23:13:19.559916 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.559877    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f98e0ab3-2bd3-44a8-9132-6f3521075ba5-kubelet-config\") pod \"global-pull-secret-syncer-95x9c\" (UID: \"f98e0ab3-2bd3-44a8-9132-6f3521075ba5\") " pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:13:19.560046 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.559940    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-host-slash\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.560046 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.559960    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-run-systemd\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.560046 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.559992    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-var-lib-openvswitch\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.560046 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.560014    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-host-cni-bin\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.560046 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.560035    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-ovnkube-config\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.560271 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.560051    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-ovnkube-script-lib\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.560271 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.560071    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f98e0ab3-2bd3-44a8-9132-6f3521075ba5-original-pull-secret\") pod \"global-pull-secret-syncer-95x9c\" (UID: \"f98e0ab3-2bd3-44a8-9132-6f3521075ba5\") " pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:13:19.560271 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.560106    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 23 23:13:19.560271 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.560109    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-etc-openvswitch\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.560271 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.560158    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-run-ovn\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.560271 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.560184    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-node-log\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.560271 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.560193    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7799t\""
Jul 23 23:13:19.560271 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.560199    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdhl9\" (UniqueName: \"kubernetes.io/projected/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-kube-api-access-mdhl9\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.560271 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.560227    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-host-kubelet\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.560271 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.560254    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 23 23:13:19.560271 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.560254    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-host-run-netns\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.560680 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.560291    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-run-openvswitch\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.560680 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.560311    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-log-socket\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.560680 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.560325    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-host-run-ovn-kubernetes\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.560680 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.560339    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.560680 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.560372    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-env-overrides\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.560680 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.560401    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f98e0ab3-2bd3-44a8-9132-6f3521075ba5-dbus\") pod \"global-pull-secret-syncer-95x9c\" (UID: \"f98e0ab3-2bd3-44a8-9132-6f3521075ba5\") " pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:13:19.560680 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.560418    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-systemd-units\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.560680 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.560444    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-host-cni-netd\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.560680 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.560465    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-ovn-node-metrics-cert\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.562449 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.562433    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wk67r"
Jul 23 23:13:19.562579 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.562562    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 23 23:13:19.562643 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.562600    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 23 23:13:19.562779 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.562764    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-ttt7t\""
Jul 23 23:13:19.564728 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.564710    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 23 23:13:19.565024 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.565010    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 23 23:13:19.565137 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.565123    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.565284 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.565269    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 23 23:13:19.565361 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.565346    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-5g97f\""
Jul 23 23:13:19.565489 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.565475    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 23 23:13:19.565610 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.565508    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 23 23:13:19.565610 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.565527    2548 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 23 23:13:19.567607 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.567591    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-2b9vc\""
Jul 23 23:13:19.567748 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.567732    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-gqbl6"
Jul 23 23:13:19.567859 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.567836    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 23 23:13:19.570206 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.570186    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 23 23:13:19.570328 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.570225    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 23 23:13:19.570328 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.570263    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-zbvlq\""
Jul 23 23:13:19.570328 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.570303    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 23 23:13:19.570514 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.570463    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mth7q"
Jul 23 23:13:19.572811 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.572784    2548 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-22 23:08:18 +0000 UTC" deadline="2028-01-02 23:34:37.238511452 +0000 UTC"
Jul 23 23:13:19.572900 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.572812    2548 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12672h21m17.665703198s"
Jul 23 23:13:19.573106 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.573087    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 23 23:13:19.573194 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.573109    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 23 23:13:19.573793 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.573327    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-dbqpq\""
Jul 23 23:13:19.573793 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.573444    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sst5n"
Jul 23 23:13:19.575735 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.575721    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.576110 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.576096    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 23 23:13:19.576180 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.576108    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-kgfkl\""
Jul 23 23:13:19.576180 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.576110    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 23 23:13:19.576180 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.576150    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 23 23:13:19.578295 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.578280    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rdphw"
Jul 23 23:13:19.578447 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.578432    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 23 23:13:19.578641 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.578624    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 23 23:13:19.578727 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.578656    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-d6wdj\""
Jul 23 23:13:19.580504 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.580489    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-t5cvp\""
Jul 23 23:13:19.580763 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.580750    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 23 23:13:19.581166 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.581145    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 23 23:13:19.581511 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.581491    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 23 23:13:19.583716 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.583666    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:13:19.584063 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:19.584036    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fkdcf" podUID="5037079f-e554-410b-973a-adef25cd2493"
Jul 23 23:13:19.586494 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.586478    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fbnfl"
Jul 23 23:13:19.586599 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:19.586533    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fbnfl" podUID="4fa58538-875d-4416-954e-289429d66399"
Jul 23 23:13:19.586653 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.586609    2548 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hrrzc"
Jul 23 23:13:19.594424 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.594406    2548 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hrrzc"
Jul 23 23:13:19.653561 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.653543    2548 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 23:13:19.655412 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.655395    2548 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 23 23:13:19.661526 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.661501    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/61447916-0a62-4e38-ba70-3ec0fd1bca25-host\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.661625 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.661531    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c49mt\" (UniqueName: \"kubernetes.io/projected/ba2ce11c-4a0d-4a34-b106-c34b61838088-kube-api-access-c49mt\") pod \"node-ca-rdphw\" (UID: \"ba2ce11c-4a0d-4a34-b106-c34b61838088\") " pod="openshift-image-registry/node-ca-rdphw"
Jul 23 23:13:19.661625 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.661547    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-system-cni-dir\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.661625 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.661561    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-hostroot\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.661625 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.661575    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-host-run-multus-certs\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.661625 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.661595    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f98e0ab3-2bd3-44a8-9132-6f3521075ba5-original-pull-secret\") pod \"global-pull-secret-syncer-95x9c\" (UID: \"f98e0ab3-2bd3-44a8-9132-6f3521075ba5\") " pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:13:19.661784 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.661637    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-etc-openvswitch\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.661784 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:19.661710    2548 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 23:13:19.661784 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.661742    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/61447916-0a62-4e38-ba70-3ec0fd1bca25-lib-modules\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.661784 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:19.661765    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f98e0ab3-2bd3-44a8-9132-6f3521075ba5-original-pull-secret podName:f98e0ab3-2bd3-44a8-9132-6f3521075ba5 nodeName:}" failed. No retries permitted until 2026-07-23 23:13:20.161749159 +0000 UTC m=+1.905478191 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f98e0ab3-2bd3-44a8-9132-6f3521075ba5-original-pull-secret") pod "global-pull-secret-syncer-95x9c" (UID: "f98e0ab3-2bd3-44a8-9132-6f3521075ba5") : object "kube-system"/"original-pull-secret" not registered
Jul 23 23:13:19.661927 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.661791    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7f715492-d857-4ac5-bb35-db1169ae239e-cni-binary-copy\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.661927 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.661760    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-etc-openvswitch\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.661927 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.661822    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-host-run-netns\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.661927 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.661847    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-host-var-lib-kubelet\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.661927 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.661869    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-multus-conf-dir\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.661927 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.661891    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7f715492-d857-4ac5-bb35-db1169ae239e-multus-daemon-config\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.662244 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.661930    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-host-kubelet\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.662244 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.661998    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-host-kubelet\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.662244 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662012    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-host-run-ovn-kubernetes\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.662244 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662032    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-ovn-node-metrics-cert\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.662244 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662048    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56add3c1-56d8-46a3-aaf7-88b91aad4640-tmp-dir\") pod \"node-resolver-mth7q\" (UID: \"56add3c1-56d8-46a3-aaf7-88b91aad4640\") " pod="openshift-dns/node-resolver-mth7q"
Jul 23 23:13:19.662244 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662057    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-host-run-ovn-kubernetes\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.662244 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662070    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-host-run-k8s-cni-cncf-io\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.662244 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662086    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-host-var-lib-cni-multus\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.662244 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662103    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-systemd-units\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.662244 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662122    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/93aa8031-e34b-4480-9673-aa50ffa4c41a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sst5n\" (UID: \"93aa8031-e34b-4480-9673-aa50ffa4c41a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sst5n"
Jul 23 23:13:19.662244 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662145    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/61447916-0a62-4e38-ba70-3ec0fd1bca25-etc-kubernetes\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.662244 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662161    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-systemd-units\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.662244 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662166    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/61447916-0a62-4e38-ba70-3ec0fd1bca25-run\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.662244 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662190    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-run-systemd\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.662244 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662229    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-run-systemd\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.662244 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662247    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-host-cni-bin\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.662882 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662264    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/93aa8031-e34b-4480-9673-aa50ffa4c41a-device-dir\") pod \"aws-ebs-csi-driver-node-sst5n\" (UID: \"93aa8031-e34b-4480-9673-aa50ffa4c41a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sst5n"
Jul 23 23:13:19.662882 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662284    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/61447916-0a62-4e38-ba70-3ec0fd1bca25-etc-sysctl-d\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.662882 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662300    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkq27\" (UniqueName: \"kubernetes.io/projected/6bdd18e7-c4ba-4506-8b81-0443611d06da-kube-api-access-rkq27\") pod \"multus-additional-cni-plugins-wk67r\" (UID: \"6bdd18e7-c4ba-4506-8b81-0443611d06da\") " pod="openshift-multus/multus-additional-cni-plugins-wk67r"
Jul 23 23:13:19.662882 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662303    2548 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 23 23:13:19.662882 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662318    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-run-ovn\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.662882 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662297    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-host-cni-bin\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.662882 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662345    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-node-log\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.662882 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662364    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-run-ovn\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.662882 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662373    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdhl9\" (UniqueName: \"kubernetes.io/projected/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-kube-api-access-mdhl9\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.662882 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662392    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-node-log\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.662882 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662417    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/93aa8031-e34b-4480-9673-aa50ffa4c41a-socket-dir\") pod \"aws-ebs-csi-driver-node-sst5n\" (UID: \"93aa8031-e34b-4480-9673-aa50ffa4c41a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sst5n"
Jul 23 23:13:19.662882 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662441    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/54cba885-3128-45cb-a350-f05f7dc12dbe-agent-certs\") pod \"konnectivity-agent-b6n2c\" (UID: \"54cba885-3128-45cb-a350-f05f7dc12dbe\") " pod="kube-system/konnectivity-agent-b6n2c"
Jul 23 23:13:19.662882 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662465    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xvtn\" (UniqueName: \"kubernetes.io/projected/0bc0f46b-0b82-4a65-95b7-3c355a05c679-kube-api-access-6xvtn\") pod \"iptables-alerter-gqbl6\" (UID: \"0bc0f46b-0b82-4a65-95b7-3c355a05c679\") " pod="openshift-network-operator/iptables-alerter-gqbl6"
Jul 23 23:13:19.662882 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662493    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smvx6\" (UniqueName: \"kubernetes.io/projected/7f715492-d857-4ac5-bb35-db1169ae239e-kube-api-access-smvx6\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.662882 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662531    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/61447916-0a62-4e38-ba70-3ec0fd1bca25-etc-sysctl-conf\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.662882 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662590    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-etc-kubernetes\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.662882 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662622    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/93aa8031-e34b-4480-9673-aa50ffa4c41a-registration-dir\") pod \"aws-ebs-csi-driver-node-sst5n\" (UID: \"93aa8031-e34b-4480-9673-aa50ffa4c41a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sst5n"
Jul 23 23:13:19.663662 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662650    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6bdd18e7-c4ba-4506-8b81-0443611d06da-os-release\") pod \"multus-additional-cni-plugins-wk67r\" (UID: \"6bdd18e7-c4ba-4506-8b81-0443611d06da\") " pod="openshift-multus/multus-additional-cni-plugins-wk67r"
Jul 23 23:13:19.663662 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662675    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-multus-cni-dir\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.663662 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662706    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-multus-socket-dir-parent\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.663662 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662734    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f98e0ab3-2bd3-44a8-9132-6f3521075ba5-kubelet-config\") pod \"global-pull-secret-syncer-95x9c\" (UID: \"f98e0ab3-2bd3-44a8-9132-6f3521075ba5\") " pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:13:19.663662 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662754    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-var-lib-openvswitch\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.663662 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662784    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-ovnkube-config\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.663662 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662785    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f98e0ab3-2bd3-44a8-9132-6f3521075ba5-kubelet-config\") pod \"global-pull-secret-syncer-95x9c\" (UID: \"f98e0ab3-2bd3-44a8-9132-6f3521075ba5\") " pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:13:19.663662 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662810    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-var-lib-openvswitch\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.663662 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662837    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/56add3c1-56d8-46a3-aaf7-88b91aad4640-hosts-file\") pod \"node-resolver-mth7q\" (UID: \"56add3c1-56d8-46a3-aaf7-88b91aad4640\") " pod="openshift-dns/node-resolver-mth7q"
Jul 23 23:13:19.663662 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662867    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5037079f-e554-410b-973a-adef25cd2493-metrics-certs\") pod \"network-metrics-daemon-fkdcf\" (UID: \"5037079f-e554-410b-973a-adef25cd2493\") " pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:13:19.663662 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662890    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6bdd18e7-c4ba-4506-8b81-0443611d06da-system-cni-dir\") pod \"multus-additional-cni-plugins-wk67r\" (UID: \"6bdd18e7-c4ba-4506-8b81-0443611d06da\") " pod="openshift-multus/multus-additional-cni-plugins-wk67r"
Jul 23 23:13:19.663662 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662917    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ba2ce11c-4a0d-4a34-b106-c34b61838088-serviceca\") pod \"node-ca-rdphw\" (UID: \"ba2ce11c-4a0d-4a34-b106-c34b61838088\") " pod="openshift-image-registry/node-ca-rdphw"
Jul 23 23:13:19.663662 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662942    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkqlv\" (UniqueName: \"kubernetes.io/projected/5037079f-e554-410b-973a-adef25cd2493-kube-api-access-lkqlv\") pod \"network-metrics-daemon-fkdcf\" (UID: \"5037079f-e554-410b-973a-adef25cd2493\") " pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:13:19.663662 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.662967    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6bdd18e7-c4ba-4506-8b81-0443611d06da-cnibin\") pod \"multus-additional-cni-plugins-wk67r\" (UID: \"6bdd18e7-c4ba-4506-8b81-0443611d06da\") " pod="openshift-multus/multus-additional-cni-plugins-wk67r"
Jul 23 23:13:19.663662 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.663006    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwqtd\" (UniqueName: \"kubernetes.io/projected/4fa58538-875d-4416-954e-289429d66399-kube-api-access-mwqtd\") pod \"network-check-target-fbnfl\" (UID: \"4fa58538-875d-4416-954e-289429d66399\") " pod="openshift-network-diagnostics/network-check-target-fbnfl"
Jul 23 23:13:19.663662 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.663031    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-os-release\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.664351 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.663055    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-host-var-lib-cni-bin\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.664351 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.663082    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-host-run-netns\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.664351 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.663104    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/93aa8031-e34b-4480-9673-aa50ffa4c41a-sys-fs\") pod \"aws-ebs-csi-driver-node-sst5n\" (UID: \"93aa8031-e34b-4480-9673-aa50ffa4c41a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sst5n"
Jul 23 23:13:19.664351 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.663137    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c77rb\" (UniqueName: \"kubernetes.io/projected/93aa8031-e34b-4480-9673-aa50ffa4c41a-kube-api-access-c77rb\") pod \"aws-ebs-csi-driver-node-sst5n\" (UID: \"93aa8031-e34b-4480-9673-aa50ffa4c41a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sst5n"
Jul 23 23:13:19.664351 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.663159    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba2ce11c-4a0d-4a34-b106-c34b61838088-host\") pod \"node-ca-rdphw\" (UID: \"ba2ce11c-4a0d-4a34-b106-c34b61838088\") " pod="openshift-image-registry/node-ca-rdphw"
Jul 23 23:13:19.664351 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.663180    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-host-run-netns\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.664351 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.663182    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6bdd18e7-c4ba-4506-8b81-0443611d06da-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wk67r\" (UID: \"6bdd18e7-c4ba-4506-8b81-0443611d06da\") " pod="openshift-multus/multus-additional-cni-plugins-wk67r"
Jul 23 23:13:19.664351 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.663228    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6bdd18e7-c4ba-4506-8b81-0443611d06da-cni-binary-copy\") pod \"multus-additional-cni-plugins-wk67r\" (UID: \"6bdd18e7-c4ba-4506-8b81-0443611d06da\") " pod="openshift-multus/multus-additional-cni-plugins-wk67r"
Jul 23 23:13:19.664351 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.663256    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0bc0f46b-0b82-4a65-95b7-3c355a05c679-iptables-alerter-script\") pod \"iptables-alerter-gqbl6\" (UID: \"0bc0f46b-0b82-4a65-95b7-3c355a05c679\") " pod="openshift-network-operator/iptables-alerter-gqbl6"
Jul 23 23:13:19.664351 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.663280    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0bc0f46b-0b82-4a65-95b7-3c355a05c679-host-slash\") pod \"iptables-alerter-gqbl6\" (UID: \"0bc0f46b-0b82-4a65-95b7-3c355a05c679\") " pod="openshift-network-operator/iptables-alerter-gqbl6"
Jul 23 23:13:19.664351 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.663310    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-ovnkube-script-lib\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.664351 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.663324    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-ovnkube-config\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.664351 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.663345    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/61447916-0a62-4e38-ba70-3ec0fd1bca25-etc-modprobe-d\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.664351 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.663382    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61447916-0a62-4e38-ba70-3ec0fd1bca25-var-lib-kubelet\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.664351 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.663410    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/54cba885-3128-45cb-a350-f05f7dc12dbe-konnectivity-ca\") pod \"konnectivity-agent-b6n2c\" (UID: \"54cba885-3128-45cb-a350-f05f7dc12dbe\") " pod="kube-system/konnectivity-agent-b6n2c"
Jul 23 23:13:19.664351 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.663435    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxxc5\" (UniqueName: \"kubernetes.io/projected/56add3c1-56d8-46a3-aaf7-88b91aad4640-kube-api-access-nxxc5\") pod \"node-resolver-mth7q\" (UID: \"56add3c1-56d8-46a3-aaf7-88b91aad4640\") " pod="openshift-dns/node-resolver-mth7q"
Jul 23 23:13:19.664892 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.663470    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-run-openvswitch\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.664892 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.663496    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-log-socket\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.664892 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.663514    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-run-openvswitch\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.664892 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.663544    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.664892 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.663564    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-log-socket\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.664892 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.663609    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-env-overrides\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.664892 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.663635    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.664892 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.663721    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/93aa8031-e34b-4480-9673-aa50ffa4c41a-etc-selinux\") pod \"aws-ebs-csi-driver-node-sst5n\" (UID: \"93aa8031-e34b-4480-9673-aa50ffa4c41a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sst5n"
Jul 23 23:13:19.664892 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.663763    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/61447916-0a62-4e38-ba70-3ec0fd1bca25-etc-sysconfig\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.664892 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.663798    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/61447916-0a62-4e38-ba70-3ec0fd1bca25-etc-tuned\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.664892 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.663816    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-ovnkube-script-lib\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.664892 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.663836    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6bdd18e7-c4ba-4506-8b81-0443611d06da-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wk67r\" (UID: \"6bdd18e7-c4ba-4506-8b81-0443611d06da\") " pod="openshift-multus/multus-additional-cni-plugins-wk67r"
Jul 23 23:13:19.664892 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.663871    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f98e0ab3-2bd3-44a8-9132-6f3521075ba5-dbus\") pod \"global-pull-secret-syncer-95x9c\" (UID: \"f98e0ab3-2bd3-44a8-9132-6f3521075ba5\") " pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:13:19.664892 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.663880    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-env-overrides\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.664892 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.663897    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-host-cni-netd\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.664892 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.663926    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/61447916-0a62-4e38-ba70-3ec0fd1bca25-etc-systemd\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.664892 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.663947    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61447916-0a62-4e38-ba70-3ec0fd1bca25-sys\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.665762 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.663965    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61447916-0a62-4e38-ba70-3ec0fd1bca25-tmp\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.665762 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.663993    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f98e0ab3-2bd3-44a8-9132-6f3521075ba5-dbus\") pod \"global-pull-secret-syncer-95x9c\" (UID: \"f98e0ab3-2bd3-44a8-9132-6f3521075ba5\") " pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:13:19.665762 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.664007    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xj5lt\" (UniqueName: \"kubernetes.io/projected/61447916-0a62-4e38-ba70-3ec0fd1bca25-kube-api-access-xj5lt\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.665762 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.664033    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6bdd18e7-c4ba-4506-8b81-0443611d06da-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wk67r\" (UID: \"6bdd18e7-c4ba-4506-8b81-0443611d06da\") " pod="openshift-multus/multus-additional-cni-plugins-wk67r"
Jul 23 23:13:19.665762 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.664042    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-host-cni-netd\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.665762 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.664071    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-cnibin\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.665762 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.664110    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-host-slash\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.665762 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.664179    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-host-slash\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.665762 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.665631    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-ovn-node-metrics-cert\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.673939 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.673915    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdhl9\" (UniqueName: \"kubernetes.io/projected/fad85ae6-00e4-4c30-9d2b-84c83484eaa4-kube-api-access-mdhl9\") pod \"ovnkube-node-55rnx\" (UID: \"fad85ae6-00e4-4c30-9d2b-84c83484eaa4\") " pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.726093 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:19.726065    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ba0f66b58401a0fb93564ba1a377674.slice/crio-e573350f00bb055f7ae82db01cd71fdb1b74804c94ff001a67814ff5221b3a5e WatchSource:0}: Error finding container e573350f00bb055f7ae82db01cd71fdb1b74804c94ff001a67814ff5221b3a5e: Status 404 returned error can't find the container with id e573350f00bb055f7ae82db01cd71fdb1b74804c94ff001a67814ff5221b3a5e
Jul 23 23:13:19.727816 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:19.727791    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45402979716ef826bbc95cd236f845a5.slice/crio-353b1e59c91a2a07f2e2fca7164ee0fe1ce30df63bec4f0923d02a8cf2f297b1 WatchSource:0}: Error finding container 353b1e59c91a2a07f2e2fca7164ee0fe1ce30df63bec4f0923d02a8cf2f297b1: Status 404 returned error can't find the container with id 353b1e59c91a2a07f2e2fca7164ee0fe1ce30df63bec4f0923d02a8cf2f297b1
Jul 23 23:13:19.731153 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.731134    2548 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 23:13:19.744450 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.744410    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-178.ec2.internal" event={"ID":"9ba0f66b58401a0fb93564ba1a377674","Type":"ContainerStarted","Data":"e573350f00bb055f7ae82db01cd71fdb1b74804c94ff001a67814ff5221b3a5e"}
Jul 23 23:13:19.745749 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.745727    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-178.ec2.internal" event={"ID":"45402979716ef826bbc95cd236f845a5","Type":"ContainerStarted","Data":"353b1e59c91a2a07f2e2fca7164ee0fe1ce30df63bec4f0923d02a8cf2f297b1"}
Jul 23 23:13:19.764659 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.764634    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/61447916-0a62-4e38-ba70-3ec0fd1bca25-etc-sysctl-conf\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.764659 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.764659    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-etc-kubernetes\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.764790 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.764675    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/93aa8031-e34b-4480-9673-aa50ffa4c41a-registration-dir\") pod \"aws-ebs-csi-driver-node-sst5n\" (UID: \"93aa8031-e34b-4480-9673-aa50ffa4c41a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sst5n"
Jul 23 23:13:19.764790 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.764691    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6bdd18e7-c4ba-4506-8b81-0443611d06da-os-release\") pod \"multus-additional-cni-plugins-wk67r\" (UID: \"6bdd18e7-c4ba-4506-8b81-0443611d06da\") " pod="openshift-multus/multus-additional-cni-plugins-wk67r"
Jul 23 23:13:19.764790 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.764712    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-multus-cni-dir\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.764790 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.764741    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-etc-kubernetes\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.764790 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.764748    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-multus-socket-dir-parent\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.764790 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.764778    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/93aa8031-e34b-4480-9673-aa50ffa4c41a-registration-dir\") pod \"aws-ebs-csi-driver-node-sst5n\" (UID: \"93aa8031-e34b-4480-9673-aa50ffa4c41a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sst5n"
Jul 23 23:13:19.765012 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.764795    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-multus-socket-dir-parent\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.765012 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.764802    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-multus-cni-dir\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.765012 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.764810    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/56add3c1-56d8-46a3-aaf7-88b91aad4640-hosts-file\") pod \"node-resolver-mth7q\" (UID: \"56add3c1-56d8-46a3-aaf7-88b91aad4640\") " pod="openshift-dns/node-resolver-mth7q"
Jul 23 23:13:19.765012 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.764832    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/56add3c1-56d8-46a3-aaf7-88b91aad4640-hosts-file\") pod \"node-resolver-mth7q\" (UID: \"56add3c1-56d8-46a3-aaf7-88b91aad4640\") " pod="openshift-dns/node-resolver-mth7q"
Jul 23 23:13:19.765012 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.764833    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6bdd18e7-c4ba-4506-8b81-0443611d06da-os-release\") pod \"multus-additional-cni-plugins-wk67r\" (UID: \"6bdd18e7-c4ba-4506-8b81-0443611d06da\") " pod="openshift-multus/multus-additional-cni-plugins-wk67r"
Jul 23 23:13:19.765012 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.764841    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5037079f-e554-410b-973a-adef25cd2493-metrics-certs\") pod \"network-metrics-daemon-fkdcf\" (UID: \"5037079f-e554-410b-973a-adef25cd2493\") " pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:13:19.765012 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.764858    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/61447916-0a62-4e38-ba70-3ec0fd1bca25-etc-sysctl-conf\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.765012 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.764884    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6bdd18e7-c4ba-4506-8b81-0443611d06da-system-cni-dir\") pod \"multus-additional-cni-plugins-wk67r\" (UID: \"6bdd18e7-c4ba-4506-8b81-0443611d06da\") " pod="openshift-multus/multus-additional-cni-plugins-wk67r"
Jul 23 23:13:19.765012 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.764904    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ba2ce11c-4a0d-4a34-b106-c34b61838088-serviceca\") pod \"node-ca-rdphw\" (UID: \"ba2ce11c-4a0d-4a34-b106-c34b61838088\") " pod="openshift-image-registry/node-ca-rdphw"
Jul 23 23:13:19.765012 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:19.764912    2548 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 23:13:19.765012 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.764916    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6bdd18e7-c4ba-4506-8b81-0443611d06da-system-cni-dir\") pod \"multus-additional-cni-plugins-wk67r\" (UID: \"6bdd18e7-c4ba-4506-8b81-0443611d06da\") " pod="openshift-multus/multus-additional-cni-plugins-wk67r"
Jul 23 23:13:19.765012 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.764921    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkqlv\" (UniqueName: \"kubernetes.io/projected/5037079f-e554-410b-973a-adef25cd2493-kube-api-access-lkqlv\") pod \"network-metrics-daemon-fkdcf\" (UID: \"5037079f-e554-410b-973a-adef25cd2493\") " pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:13:19.765012 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.764951    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6bdd18e7-c4ba-4506-8b81-0443611d06da-cnibin\") pod \"multus-additional-cni-plugins-wk67r\" (UID: \"6bdd18e7-c4ba-4506-8b81-0443611d06da\") " pod="openshift-multus/multus-additional-cni-plugins-wk67r"
Jul 23 23:13:19.765012 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:19.764988    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5037079f-e554-410b-973a-adef25cd2493-metrics-certs podName:5037079f-e554-410b-973a-adef25cd2493 nodeName:}" failed. No retries permitted until 2026-07-23 23:13:20.264953741 +0000 UTC m=+2.008682779 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5037079f-e554-410b-973a-adef25cd2493-metrics-certs") pod "network-metrics-daemon-fkdcf" (UID: "5037079f-e554-410b-973a-adef25cd2493") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 23:13:19.765012 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.764991    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6bdd18e7-c4ba-4506-8b81-0443611d06da-cnibin\") pod \"multus-additional-cni-plugins-wk67r\" (UID: \"6bdd18e7-c4ba-4506-8b81-0443611d06da\") " pod="openshift-multus/multus-additional-cni-plugins-wk67r"
Jul 23 23:13:19.765012 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765016    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwqtd\" (UniqueName: \"kubernetes.io/projected/4fa58538-875d-4416-954e-289429d66399-kube-api-access-mwqtd\") pod \"network-check-target-fbnfl\" (UID: \"4fa58538-875d-4416-954e-289429d66399\") " pod="openshift-network-diagnostics/network-check-target-fbnfl"
Jul 23 23:13:19.765787 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765041    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-os-release\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.765787 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765064    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-host-var-lib-cni-bin\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.765787 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765088    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/93aa8031-e34b-4480-9673-aa50ffa4c41a-sys-fs\") pod \"aws-ebs-csi-driver-node-sst5n\" (UID: \"93aa8031-e34b-4480-9673-aa50ffa4c41a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sst5n"
Jul 23 23:13:19.765787 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765114    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c77rb\" (UniqueName: \"kubernetes.io/projected/93aa8031-e34b-4480-9673-aa50ffa4c41a-kube-api-access-c77rb\") pod \"aws-ebs-csi-driver-node-sst5n\" (UID: \"93aa8031-e34b-4480-9673-aa50ffa4c41a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sst5n"
Jul 23 23:13:19.765787 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765123    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-os-release\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.765787 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765141    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba2ce11c-4a0d-4a34-b106-c34b61838088-host\") pod \"node-ca-rdphw\" (UID: \"ba2ce11c-4a0d-4a34-b106-c34b61838088\") " pod="openshift-image-registry/node-ca-rdphw"
Jul 23 23:13:19.765787 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765167    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6bdd18e7-c4ba-4506-8b81-0443611d06da-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wk67r\" (UID: \"6bdd18e7-c4ba-4506-8b81-0443611d06da\") " pod="openshift-multus/multus-additional-cni-plugins-wk67r"
Jul 23 23:13:19.765787 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765174    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/93aa8031-e34b-4480-9673-aa50ffa4c41a-sys-fs\") pod \"aws-ebs-csi-driver-node-sst5n\" (UID: \"93aa8031-e34b-4480-9673-aa50ffa4c41a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sst5n"
Jul 23 23:13:19.765787 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765181    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba2ce11c-4a0d-4a34-b106-c34b61838088-host\") pod \"node-ca-rdphw\" (UID: \"ba2ce11c-4a0d-4a34-b106-c34b61838088\") " pod="openshift-image-registry/node-ca-rdphw"
Jul 23 23:13:19.765787 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765121    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-host-var-lib-cni-bin\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.765787 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765195    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6bdd18e7-c4ba-4506-8b81-0443611d06da-cni-binary-copy\") pod \"multus-additional-cni-plugins-wk67r\" (UID: \"6bdd18e7-c4ba-4506-8b81-0443611d06da\") " pod="openshift-multus/multus-additional-cni-plugins-wk67r"
Jul 23 23:13:19.765787 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765243    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0bc0f46b-0b82-4a65-95b7-3c355a05c679-iptables-alerter-script\") pod \"iptables-alerter-gqbl6\" (UID: \"0bc0f46b-0b82-4a65-95b7-3c355a05c679\") " pod="openshift-network-operator/iptables-alerter-gqbl6"
Jul 23 23:13:19.765787 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765269    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0bc0f46b-0b82-4a65-95b7-3c355a05c679-host-slash\") pod \"iptables-alerter-gqbl6\" (UID: \"0bc0f46b-0b82-4a65-95b7-3c355a05c679\") " pod="openshift-network-operator/iptables-alerter-gqbl6"
Jul 23 23:13:19.765787 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765292    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/61447916-0a62-4e38-ba70-3ec0fd1bca25-etc-modprobe-d\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.765787 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765319    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61447916-0a62-4e38-ba70-3ec0fd1bca25-var-lib-kubelet\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.765787 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765338    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0bc0f46b-0b82-4a65-95b7-3c355a05c679-host-slash\") pod \"iptables-alerter-gqbl6\" (UID: \"0bc0f46b-0b82-4a65-95b7-3c355a05c679\") " pod="openshift-network-operator/iptables-alerter-gqbl6"
Jul 23 23:13:19.765787 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765354    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/54cba885-3128-45cb-a350-f05f7dc12dbe-konnectivity-ca\") pod \"konnectivity-agent-b6n2c\" (UID: \"54cba885-3128-45cb-a350-f05f7dc12dbe\") " pod="kube-system/konnectivity-agent-b6n2c"
Jul 23 23:13:19.766596 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765379    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxxc5\" (UniqueName: \"kubernetes.io/projected/56add3c1-56d8-46a3-aaf7-88b91aad4640-kube-api-access-nxxc5\") pod \"node-resolver-mth7q\" (UID: \"56add3c1-56d8-46a3-aaf7-88b91aad4640\") " pod="openshift-dns/node-resolver-mth7q"
Jul 23 23:13:19.766596 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765423    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/93aa8031-e34b-4480-9673-aa50ffa4c41a-etc-selinux\") pod \"aws-ebs-csi-driver-node-sst5n\" (UID: \"93aa8031-e34b-4480-9673-aa50ffa4c41a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sst5n"
Jul 23 23:13:19.766596 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765445    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/61447916-0a62-4e38-ba70-3ec0fd1bca25-etc-modprobe-d\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.766596 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765447    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/61447916-0a62-4e38-ba70-3ec0fd1bca25-etc-sysconfig\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.766596 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765486    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/61447916-0a62-4e38-ba70-3ec0fd1bca25-etc-tuned\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.766596 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765513    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6bdd18e7-c4ba-4506-8b81-0443611d06da-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wk67r\" (UID: \"6bdd18e7-c4ba-4506-8b81-0443611d06da\") " pod="openshift-multus/multus-additional-cni-plugins-wk67r"
Jul 23 23:13:19.766596 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765524    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ba2ce11c-4a0d-4a34-b106-c34b61838088-serviceca\") pod \"node-ca-rdphw\" (UID: \"ba2ce11c-4a0d-4a34-b106-c34b61838088\") " pod="openshift-image-registry/node-ca-rdphw"
Jul 23 23:13:19.766596 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765543    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/61447916-0a62-4e38-ba70-3ec0fd1bca25-etc-systemd\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.766596 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765580    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61447916-0a62-4e38-ba70-3ec0fd1bca25-sys\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.766596 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765592    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/61447916-0a62-4e38-ba70-3ec0fd1bca25-etc-systemd\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.766596 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765611    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61447916-0a62-4e38-ba70-3ec0fd1bca25-tmp\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.766596 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765635    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/61447916-0a62-4e38-ba70-3ec0fd1bca25-etc-sysconfig\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.766596 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765639    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xj5lt\" (UniqueName: \"kubernetes.io/projected/61447916-0a62-4e38-ba70-3ec0fd1bca25-kube-api-access-xj5lt\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.766596 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765669    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6bdd18e7-c4ba-4506-8b81-0443611d06da-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wk67r\" (UID: \"6bdd18e7-c4ba-4506-8b81-0443611d06da\") " pod="openshift-multus/multus-additional-cni-plugins-wk67r"
Jul 23 23:13:19.766596 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765693    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-cnibin\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.766596 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765717    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/61447916-0a62-4e38-ba70-3ec0fd1bca25-host\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.766596 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765741    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c49mt\" (UniqueName: \"kubernetes.io/projected/ba2ce11c-4a0d-4a34-b106-c34b61838088-kube-api-access-c49mt\") pod \"node-ca-rdphw\" (UID: \"ba2ce11c-4a0d-4a34-b106-c34b61838088\") " pod="openshift-image-registry/node-ca-rdphw"
Jul 23 23:13:19.767297 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765758    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/93aa8031-e34b-4480-9673-aa50ffa4c41a-etc-selinux\") pod \"aws-ebs-csi-driver-node-sst5n\" (UID: \"93aa8031-e34b-4480-9673-aa50ffa4c41a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sst5n"
Jul 23 23:13:19.767297 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765764    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-system-cni-dir\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.767297 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765799    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6bdd18e7-c4ba-4506-8b81-0443611d06da-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wk67r\" (UID: \"6bdd18e7-c4ba-4506-8b81-0443611d06da\") " pod="openshift-multus/multus-additional-cni-plugins-wk67r"
Jul 23 23:13:19.767297 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765788    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-hostroot\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.767297 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765839    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61447916-0a62-4e38-ba70-3ec0fd1bca25-var-lib-kubelet\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.767297 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765840    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-host-run-multus-certs\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.767297 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765852    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6bdd18e7-c4ba-4506-8b81-0443611d06da-cni-binary-copy\") pod \"multus-additional-cni-plugins-wk67r\" (UID: \"6bdd18e7-c4ba-4506-8b81-0443611d06da\") " pod="openshift-multus/multus-additional-cni-plugins-wk67r"
Jul 23 23:13:19.767297 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765868    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0bc0f46b-0b82-4a65-95b7-3c355a05c679-iptables-alerter-script\") pod \"iptables-alerter-gqbl6\" (UID: \"0bc0f46b-0b82-4a65-95b7-3c355a05c679\") " pod="openshift-network-operator/iptables-alerter-gqbl6"
Jul 23 23:13:19.767297 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765886    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-host-run-multus-certs\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.767297 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765906    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/61447916-0a62-4e38-ba70-3ec0fd1bca25-lib-modules\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.767297 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765943    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7f715492-d857-4ac5-bb35-db1169ae239e-cni-binary-copy\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.767297 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765952    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6bdd18e7-c4ba-4506-8b81-0443611d06da-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wk67r\" (UID: \"6bdd18e7-c4ba-4506-8b81-0443611d06da\") " pod="openshift-multus/multus-additional-cni-plugins-wk67r"
Jul 23 23:13:19.767297 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765970    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-host-run-netns\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.767297 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765969    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/54cba885-3128-45cb-a350-f05f7dc12dbe-konnectivity-ca\") pod \"konnectivity-agent-b6n2c\" (UID: \"54cba885-3128-45cb-a350-f05f7dc12dbe\") " pod="kube-system/konnectivity-agent-b6n2c"
Jul 23 23:13:19.767297 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.766015    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-host-var-lib-kubelet\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.767297 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.766022    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61447916-0a62-4e38-ba70-3ec0fd1bca25-sys\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.767297 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.766034    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/61447916-0a62-4e38-ba70-3ec0fd1bca25-lib-modules\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.767297 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.766041    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-multus-conf-dir\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.767839 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.766051    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-system-cni-dir\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.767839 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.765969    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/61447916-0a62-4e38-ba70-3ec0fd1bca25-host\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.767839 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.766065    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7f715492-d857-4ac5-bb35-db1169ae239e-multus-daemon-config\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.767839 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.766106    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56add3c1-56d8-46a3-aaf7-88b91aad4640-tmp-dir\") pod \"node-resolver-mth7q\" (UID: \"56add3c1-56d8-46a3-aaf7-88b91aad4640\") " pod="openshift-dns/node-resolver-mth7q"
Jul 23 23:13:19.767839 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.766131    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-host-run-k8s-cni-cncf-io\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.767839 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.766155    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-host-var-lib-cni-multus\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.767839 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.766181    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/93aa8031-e34b-4480-9673-aa50ffa4c41a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sst5n\" (UID: \"93aa8031-e34b-4480-9673-aa50ffa4c41a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sst5n"
Jul 23 23:13:19.767839 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.766203    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/61447916-0a62-4e38-ba70-3ec0fd1bca25-etc-kubernetes\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.767839 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.766230    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/61447916-0a62-4e38-ba70-3ec0fd1bca25-run\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.767839 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.766255    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/93aa8031-e34b-4480-9673-aa50ffa4c41a-device-dir\") pod \"aws-ebs-csi-driver-node-sst5n\" (UID: \"93aa8031-e34b-4480-9673-aa50ffa4c41a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sst5n"
Jul 23 23:13:19.767839 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.766278    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-host-var-lib-cni-multus\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.767839 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.766283    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-host-run-netns\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.767839 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.766325    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/61447916-0a62-4e38-ba70-3ec0fd1bca25-etc-kubernetes\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.767839 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.766281    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/61447916-0a62-4e38-ba70-3ec0fd1bca25-etc-sysctl-d\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.767839 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.766348    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-cnibin\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.767839 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.766353    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6bdd18e7-c4ba-4506-8b81-0443611d06da-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wk67r\" (UID: \"6bdd18e7-c4ba-4506-8b81-0443611d06da\") " pod="openshift-multus/multus-additional-cni-plugins-wk67r"
Jul 23 23:13:19.767839 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.766377    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-hostroot\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.767839 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.766378    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/61447916-0a62-4e38-ba70-3ec0fd1bca25-etc-sysctl-d\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.768402 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.766421    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkq27\" (UniqueName: \"kubernetes.io/projected/6bdd18e7-c4ba-4506-8b81-0443611d06da-kube-api-access-rkq27\") pod \"multus-additional-cni-plugins-wk67r\" (UID: \"6bdd18e7-c4ba-4506-8b81-0443611d06da\") " pod="openshift-multus/multus-additional-cni-plugins-wk67r"
Jul 23 23:13:19.768402 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.766435    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/93aa8031-e34b-4480-9673-aa50ffa4c41a-device-dir\") pod \"aws-ebs-csi-driver-node-sst5n\" (UID: \"93aa8031-e34b-4480-9673-aa50ffa4c41a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sst5n"
Jul 23 23:13:19.768402 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.766477    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-multus-conf-dir\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.768402 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.766480    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-host-var-lib-kubelet\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.768402 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.766486    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/93aa8031-e34b-4480-9673-aa50ffa4c41a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sst5n\" (UID: \"93aa8031-e34b-4480-9673-aa50ffa4c41a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sst5n"
Jul 23 23:13:19.768402 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.766519    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7f715492-d857-4ac5-bb35-db1169ae239e-host-run-k8s-cni-cncf-io\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.768402 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.766517    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/93aa8031-e34b-4480-9673-aa50ffa4c41a-socket-dir\") pod \"aws-ebs-csi-driver-node-sst5n\" (UID: \"93aa8031-e34b-4480-9673-aa50ffa4c41a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sst5n"
Jul 23 23:13:19.768402 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.766554    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/61447916-0a62-4e38-ba70-3ec0fd1bca25-run\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.768402 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.766575    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/54cba885-3128-45cb-a350-f05f7dc12dbe-agent-certs\") pod \"konnectivity-agent-b6n2c\" (UID: \"54cba885-3128-45cb-a350-f05f7dc12dbe\") " pod="kube-system/konnectivity-agent-b6n2c"
Jul 23 23:13:19.768402 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.766604    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7f715492-d857-4ac5-bb35-db1169ae239e-multus-daemon-config\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.768402 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.766626    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xvtn\" (UniqueName: \"kubernetes.io/projected/0bc0f46b-0b82-4a65-95b7-3c355a05c679-kube-api-access-6xvtn\") pod \"iptables-alerter-gqbl6\" (UID: \"0bc0f46b-0b82-4a65-95b7-3c355a05c679\") " pod="openshift-network-operator/iptables-alerter-gqbl6"
Jul 23 23:13:19.768402 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.766653    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smvx6\" (UniqueName: \"kubernetes.io/projected/7f715492-d857-4ac5-bb35-db1169ae239e-kube-api-access-smvx6\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.768402 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.766720    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56add3c1-56d8-46a3-aaf7-88b91aad4640-tmp-dir\") pod \"node-resolver-mth7q\" (UID: \"56add3c1-56d8-46a3-aaf7-88b91aad4640\") " pod="openshift-dns/node-resolver-mth7q"
Jul 23 23:13:19.768402 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.766741    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7f715492-d857-4ac5-bb35-db1169ae239e-cni-binary-copy\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.768402 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.766723    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/93aa8031-e34b-4480-9673-aa50ffa4c41a-socket-dir\") pod \"aws-ebs-csi-driver-node-sst5n\" (UID: \"93aa8031-e34b-4480-9673-aa50ffa4c41a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sst5n"
Jul 23 23:13:19.768402 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.767924    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/61447916-0a62-4e38-ba70-3ec0fd1bca25-etc-tuned\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.768402 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.768320    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61447916-0a62-4e38-ba70-3ec0fd1bca25-tmp\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.768927 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.768639    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/54cba885-3128-45cb-a350-f05f7dc12dbe-agent-certs\") pod \"konnectivity-agent-b6n2c\" (UID: \"54cba885-3128-45cb-a350-f05f7dc12dbe\") " pod="kube-system/konnectivity-agent-b6n2c"
Jul 23 23:13:19.772030 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:19.772010    2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 23:13:19.772125 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:19.772033    2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 23:13:19.772125 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:19.772046    2548 projected.go:196] Error preparing data for projected volume kube-api-access-mwqtd for pod openshift-network-diagnostics/network-check-target-fbnfl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 23:13:19.772125 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:19.772110    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4fa58538-875d-4416-954e-289429d66399-kube-api-access-mwqtd podName:4fa58538-875d-4416-954e-289429d66399 nodeName:}" failed. No retries permitted until 2026-07-23 23:13:20.272092164 +0000 UTC m=+2.015821194 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-mwqtd" (UniqueName: "kubernetes.io/projected/4fa58538-875d-4416-954e-289429d66399-kube-api-access-mwqtd") pod "network-check-target-fbnfl" (UID: "4fa58538-875d-4416-954e-289429d66399") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 23:13:19.773984 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.773944    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxxc5\" (UniqueName: \"kubernetes.io/projected/56add3c1-56d8-46a3-aaf7-88b91aad4640-kube-api-access-nxxc5\") pod \"node-resolver-mth7q\" (UID: \"56add3c1-56d8-46a3-aaf7-88b91aad4640\") " pod="openshift-dns/node-resolver-mth7q"
Jul 23 23:13:19.774083 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.774014    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkq27\" (UniqueName: \"kubernetes.io/projected/6bdd18e7-c4ba-4506-8b81-0443611d06da-kube-api-access-rkq27\") pod \"multus-additional-cni-plugins-wk67r\" (UID: \"6bdd18e7-c4ba-4506-8b81-0443611d06da\") " pod="openshift-multus/multus-additional-cni-plugins-wk67r"
Jul 23 23:13:19.774501 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.774482    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkqlv\" (UniqueName: \"kubernetes.io/projected/5037079f-e554-410b-973a-adef25cd2493-kube-api-access-lkqlv\") pod \"network-metrics-daemon-fkdcf\" (UID: \"5037079f-e554-410b-973a-adef25cd2493\") " pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:13:19.774884 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.774863    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c77rb\" (UniqueName: \"kubernetes.io/projected/93aa8031-e34b-4480-9673-aa50ffa4c41a-kube-api-access-c77rb\") pod \"aws-ebs-csi-driver-node-sst5n\" (UID: \"93aa8031-e34b-4480-9673-aa50ffa4c41a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sst5n"
Jul 23 23:13:19.775449 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.775431    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smvx6\" (UniqueName: \"kubernetes.io/projected/7f715492-d857-4ac5-bb35-db1169ae239e-kube-api-access-smvx6\") pod \"multus-bvgxv\" (UID: \"7f715492-d857-4ac5-bb35-db1169ae239e\") " pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.775543 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.775528    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c49mt\" (UniqueName: \"kubernetes.io/projected/ba2ce11c-4a0d-4a34-b106-c34b61838088-kube-api-access-c49mt\") pod \"node-ca-rdphw\" (UID: \"ba2ce11c-4a0d-4a34-b106-c34b61838088\") " pod="openshift-image-registry/node-ca-rdphw"
Jul 23 23:13:19.776067 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.776046    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xvtn\" (UniqueName: \"kubernetes.io/projected/0bc0f46b-0b82-4a65-95b7-3c355a05c679-kube-api-access-6xvtn\") pod \"iptables-alerter-gqbl6\" (UID: \"0bc0f46b-0b82-4a65-95b7-3c355a05c679\") " pod="openshift-network-operator/iptables-alerter-gqbl6"
Jul 23 23:13:19.776121 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.776046    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xj5lt\" (UniqueName: \"kubernetes.io/projected/61447916-0a62-4e38-ba70-3ec0fd1bca25-kube-api-access-xj5lt\") pod \"tuned-cwd9t\" (UID: \"61447916-0a62-4e38-ba70-3ec0fd1bca25\") " pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:19.887644 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.887559    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:19.891525 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.891503    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-b6n2c"
Jul 23 23:13:19.897670 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:19.897647    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfad85ae6_00e4_4c30_9d2b_84c83484eaa4.slice/crio-af39e8a13dfc808e4f260a7bf37e0701cab588c1ca2dff2734c499a5ef8d5cd3 WatchSource:0}: Error finding container af39e8a13dfc808e4f260a7bf37e0701cab588c1ca2dff2734c499a5ef8d5cd3: Status 404 returned error can't find the container with id af39e8a13dfc808e4f260a7bf37e0701cab588c1ca2dff2734c499a5ef8d5cd3
Jul 23 23:13:19.900165 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:19.900140    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54cba885_3128_45cb_a350_f05f7dc12dbe.slice/crio-e381b5ecae0fe8012e4f7deb60ba7d015d66217795f28e5256255049b7d4fb6e WatchSource:0}: Error finding container e381b5ecae0fe8012e4f7deb60ba7d015d66217795f28e5256255049b7d4fb6e: Status 404 returned error can't find the container with id e381b5ecae0fe8012e4f7deb60ba7d015d66217795f28e5256255049b7d4fb6e
Jul 23 23:13:19.903614 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.903595    2548 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 23:13:19.918732 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.918713    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wk67r"
Jul 23 23:13:19.927717 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:19.927694    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bdd18e7_c4ba_4506_8b81_0443611d06da.slice/crio-382400f57b8d5f4844378283f81114a1083724c573a99d3bede709e7b0e981c8 WatchSource:0}: Error finding container 382400f57b8d5f4844378283f81114a1083724c573a99d3bede709e7b0e981c8: Status 404 returned error can't find the container with id 382400f57b8d5f4844378283f81114a1083724c573a99d3bede709e7b0e981c8
Jul 23 23:13:19.931649 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.931633    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bvgxv"
Jul 23 23:13:19.939131 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:19.939110    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f715492_d857_4ac5_bb35_db1169ae239e.slice/crio-7402cfdfb71b83663e8e16077385a6aa695b7b49c8dd857206c55728c22e501e WatchSource:0}: Error finding container 7402cfdfb71b83663e8e16077385a6aa695b7b49c8dd857206c55728c22e501e: Status 404 returned error can't find the container with id 7402cfdfb71b83663e8e16077385a6aa695b7b49c8dd857206c55728c22e501e
Jul 23 23:13:19.948597 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.948582    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-gqbl6"
Jul 23 23:13:19.954379 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.954355    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mth7q"
Jul 23 23:13:19.956247 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:19.956226    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bc0f46b_0b82_4a65_95b7_3c355a05c679.slice/crio-dda30159c494bc3e0d48fb67ff5a5f0f26f52e3a63192089790d5c4e5b75969e WatchSource:0}: Error finding container dda30159c494bc3e0d48fb67ff5a5f0f26f52e3a63192089790d5c4e5b75969e: Status 404 returned error can't find the container with id dda30159c494bc3e0d48fb67ff5a5f0f26f52e3a63192089790d5c4e5b75969e
Jul 23 23:13:19.962358 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:19.962333    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56add3c1_56d8_46a3_aaf7_88b91aad4640.slice/crio-2dfb76eb260511bc96fe4e6e0ed9383e01dead39f27717be7bda6ec522114d7d WatchSource:0}: Error finding container 2dfb76eb260511bc96fe4e6e0ed9383e01dead39f27717be7bda6ec522114d7d: Status 404 returned error can't find the container with id 2dfb76eb260511bc96fe4e6e0ed9383e01dead39f27717be7bda6ec522114d7d
Jul 23 23:13:19.979948 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.979922    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sst5n"
Jul 23 23:13:19.989311 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:19.989289    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93aa8031_e34b_4480_9673_aa50ffa4c41a.slice/crio-6739fca4556a626cec997c2dc8ca3044e7e4f3fa8c0160592824a55ad70e7165 WatchSource:0}: Error finding container 6739fca4556a626cec997c2dc8ca3044e7e4f3fa8c0160592824a55ad70e7165: Status 404 returned error can't find the container with id 6739fca4556a626cec997c2dc8ca3044e7e4f3fa8c0160592824a55ad70e7165
Jul 23 23:13:19.995399 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:19.995379    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cwd9t"
Jul 23 23:13:20.000038 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:20.000020    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rdphw"
Jul 23 23:13:20.005407 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:20.005378    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61447916_0a62_4e38_ba70_3ec0fd1bca25.slice/crio-df0d88ad50f3ca20c1d127f1be6e66ecc0f3c5e7357112c602d1e1a8d7701bea WatchSource:0}: Error finding container df0d88ad50f3ca20c1d127f1be6e66ecc0f3c5e7357112c602d1e1a8d7701bea: Status 404 returned error can't find the container with id df0d88ad50f3ca20c1d127f1be6e66ecc0f3c5e7357112c602d1e1a8d7701bea
Jul 23 23:13:20.010045 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:13:20.010024    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba2ce11c_4a0d_4a34_b106_c34b61838088.slice/crio-832fb1804b387afaaed7a5db5de86c2ea02ea2ae126c869ffb66c5388289b8fc WatchSource:0}: Error finding container 832fb1804b387afaaed7a5db5de86c2ea02ea2ae126c869ffb66c5388289b8fc: Status 404 returned error can't find the container with id 832fb1804b387afaaed7a5db5de86c2ea02ea2ae126c869ffb66c5388289b8fc
Jul 23 23:13:20.170579 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:20.170548    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f98e0ab3-2bd3-44a8-9132-6f3521075ba5-original-pull-secret\") pod \"global-pull-secret-syncer-95x9c\" (UID: \"f98e0ab3-2bd3-44a8-9132-6f3521075ba5\") " pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:13:20.170758 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:20.170687    2548 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 23:13:20.170758 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:20.170750    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f98e0ab3-2bd3-44a8-9132-6f3521075ba5-original-pull-secret podName:f98e0ab3-2bd3-44a8-9132-6f3521075ba5 nodeName:}" failed. No retries permitted until 2026-07-23 23:13:21.170734422 +0000 UTC m=+2.914463450 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f98e0ab3-2bd3-44a8-9132-6f3521075ba5-original-pull-secret") pod "global-pull-secret-syncer-95x9c" (UID: "f98e0ab3-2bd3-44a8-9132-6f3521075ba5") : object "kube-system"/"original-pull-secret" not registered
Jul 23 23:13:20.271071 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:20.271040    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5037079f-e554-410b-973a-adef25cd2493-metrics-certs\") pod \"network-metrics-daemon-fkdcf\" (UID: \"5037079f-e554-410b-973a-adef25cd2493\") " pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:13:20.271224 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:20.271191    2548 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 23:13:20.271294 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:20.271261    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5037079f-e554-410b-973a-adef25cd2493-metrics-certs podName:5037079f-e554-410b-973a-adef25cd2493 nodeName:}" failed. No retries permitted until 2026-07-23 23:13:21.271242937 +0000 UTC m=+3.014971968 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5037079f-e554-410b-973a-adef25cd2493-metrics-certs") pod "network-metrics-daemon-fkdcf" (UID: "5037079f-e554-410b-973a-adef25cd2493") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 23:13:20.372521 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:20.372453    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwqtd\" (UniqueName: \"kubernetes.io/projected/4fa58538-875d-4416-954e-289429d66399-kube-api-access-mwqtd\") pod \"network-check-target-fbnfl\" (UID: \"4fa58538-875d-4416-954e-289429d66399\") " pod="openshift-network-diagnostics/network-check-target-fbnfl"
Jul 23 23:13:20.372694 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:20.372642    2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 23:13:20.372694 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:20.372669    2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 23:13:20.372694 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:20.372682    2548 projected.go:196] Error preparing data for projected volume kube-api-access-mwqtd for pod openshift-network-diagnostics/network-check-target-fbnfl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 23:13:20.372907 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:20.372745    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4fa58538-875d-4416-954e-289429d66399-kube-api-access-mwqtd podName:4fa58538-875d-4416-954e-289429d66399 nodeName:}" failed. No retries permitted until 2026-07-23 23:13:21.372725254 +0000 UTC m=+3.116454290 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-mwqtd" (UniqueName: "kubernetes.io/projected/4fa58538-875d-4416-954e-289429d66399-kube-api-access-mwqtd") pod "network-check-target-fbnfl" (UID: "4fa58538-875d-4416-954e-289429d66399") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 23:13:20.598354 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:20.598114    2548 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 23:08:19 +0000 UTC" deadline="2028-04-18 05:23:29.041159066 +0000 UTC"
Jul 23 23:13:20.598354 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:20.598174    2548 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15222h10m8.442989676s"
Jul 23 23:13:20.772253 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:20.772214    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rdphw" event={"ID":"ba2ce11c-4a0d-4a34-b106-c34b61838088","Type":"ContainerStarted","Data":"832fb1804b387afaaed7a5db5de86c2ea02ea2ae126c869ffb66c5388289b8fc"}
Jul 23 23:13:20.777991 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:20.777958    2548 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 23:13:20.792963 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:20.792851    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sst5n" event={"ID":"93aa8031-e34b-4480-9673-aa50ffa4c41a","Type":"ContainerStarted","Data":"6739fca4556a626cec997c2dc8ca3044e7e4f3fa8c0160592824a55ad70e7165"}
Jul 23 23:13:20.797304 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:20.797212    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mth7q" event={"ID":"56add3c1-56d8-46a3-aaf7-88b91aad4640","Type":"ContainerStarted","Data":"2dfb76eb260511bc96fe4e6e0ed9383e01dead39f27717be7bda6ec522114d7d"}
Jul 23 23:13:20.823563 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:20.823435    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bvgxv" event={"ID":"7f715492-d857-4ac5-bb35-db1169ae239e","Type":"ContainerStarted","Data":"7402cfdfb71b83663e8e16077385a6aa695b7b49c8dd857206c55728c22e501e"}
Jul 23 23:13:20.836498 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:20.835368    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cwd9t" event={"ID":"61447916-0a62-4e38-ba70-3ec0fd1bca25","Type":"ContainerStarted","Data":"df0d88ad50f3ca20c1d127f1be6e66ecc0f3c5e7357112c602d1e1a8d7701bea"}
Jul 23 23:13:20.863732 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:20.863603    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-gqbl6" event={"ID":"0bc0f46b-0b82-4a65-95b7-3c355a05c679","Type":"ContainerStarted","Data":"dda30159c494bc3e0d48fb67ff5a5f0f26f52e3a63192089790d5c4e5b75969e"}
Jul 23 23:13:20.887346 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:20.887261    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wk67r" event={"ID":"6bdd18e7-c4ba-4506-8b81-0443611d06da","Type":"ContainerStarted","Data":"382400f57b8d5f4844378283f81114a1083724c573a99d3bede709e7b0e981c8"}
Jul 23 23:13:20.914082 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:20.914013    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-b6n2c" event={"ID":"54cba885-3128-45cb-a350-f05f7dc12dbe","Type":"ContainerStarted","Data":"e381b5ecae0fe8012e4f7deb60ba7d015d66217795f28e5256255049b7d4fb6e"}
Jul 23 23:13:20.922808 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:20.922770    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-55rnx" event={"ID":"fad85ae6-00e4-4c30-9d2b-84c83484eaa4","Type":"ContainerStarted","Data":"af39e8a13dfc808e4f260a7bf37e0701cab588c1ca2dff2734c499a5ef8d5cd3"}
Jul 23 23:13:21.130803 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:21.130723    2548 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 23:13:21.179714 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:21.179489    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f98e0ab3-2bd3-44a8-9132-6f3521075ba5-original-pull-secret\") pod \"global-pull-secret-syncer-95x9c\" (UID: \"f98e0ab3-2bd3-44a8-9132-6f3521075ba5\") " pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:13:21.179714 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:21.179676    2548 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 23:13:21.179957 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:21.179741    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f98e0ab3-2bd3-44a8-9132-6f3521075ba5-original-pull-secret podName:f98e0ab3-2bd3-44a8-9132-6f3521075ba5 nodeName:}" failed. No retries permitted until 2026-07-23 23:13:23.179722829 +0000 UTC m=+4.923451862 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f98e0ab3-2bd3-44a8-9132-6f3521075ba5-original-pull-secret") pod "global-pull-secret-syncer-95x9c" (UID: "f98e0ab3-2bd3-44a8-9132-6f3521075ba5") : object "kube-system"/"original-pull-secret" not registered
Jul 23 23:13:21.280433 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:21.280394    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5037079f-e554-410b-973a-adef25cd2493-metrics-certs\") pod \"network-metrics-daemon-fkdcf\" (UID: \"5037079f-e554-410b-973a-adef25cd2493\") " pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:13:21.280700 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:21.280665    2548 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 23:13:21.280771 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:21.280742    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5037079f-e554-410b-973a-adef25cd2493-metrics-certs podName:5037079f-e554-410b-973a-adef25cd2493 nodeName:}" failed. No retries permitted until 2026-07-23 23:13:23.280722128 +0000 UTC m=+5.024451157 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5037079f-e554-410b-973a-adef25cd2493-metrics-certs") pod "network-metrics-daemon-fkdcf" (UID: "5037079f-e554-410b-973a-adef25cd2493") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 23:13:21.380937 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:21.380847    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwqtd\" (UniqueName: \"kubernetes.io/projected/4fa58538-875d-4416-954e-289429d66399-kube-api-access-mwqtd\") pod \"network-check-target-fbnfl\" (UID: \"4fa58538-875d-4416-954e-289429d66399\") " pod="openshift-network-diagnostics/network-check-target-fbnfl"
Jul 23 23:13:21.381125 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:21.381078    2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 23:13:21.381125 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:21.381100    2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 23:13:21.381125 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:21.381114    2548 projected.go:196] Error preparing data for projected volume kube-api-access-mwqtd for pod openshift-network-diagnostics/network-check-target-fbnfl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 23:13:21.381291 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:21.381181    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4fa58538-875d-4416-954e-289429d66399-kube-api-access-mwqtd podName:4fa58538-875d-4416-954e-289429d66399 nodeName:}" failed. No retries permitted until 2026-07-23 23:13:23.38116295 +0000 UTC m=+5.124891982 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-mwqtd" (UniqueName: "kubernetes.io/projected/4fa58538-875d-4416-954e-289429d66399-kube-api-access-mwqtd") pod "network-check-target-fbnfl" (UID: "4fa58538-875d-4416-954e-289429d66399") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 23:13:21.598564 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:21.598518    2548 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 23:08:19 +0000 UTC" deadline="2028-02-15 13:13:53.401193848 +0000 UTC"
Jul 23 23:13:21.598564 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:21.598562    2548 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13718h0m31.802636077s"
Jul 23 23:13:21.742122 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:21.742082    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:13:21.742295 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:21.742220    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95x9c" podUID="f98e0ab3-2bd3-44a8-9132-6f3521075ba5"
Jul 23 23:13:21.742649 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:21.742625    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fbnfl"
Jul 23 23:13:21.742786 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:21.742725    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fbnfl" podUID="4fa58538-875d-4416-954e-289429d66399"
Jul 23 23:13:21.742901 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:21.742842    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:13:21.742968 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:21.742929    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fkdcf" podUID="5037079f-e554-410b-973a-adef25cd2493"
Jul 23 23:13:23.196822 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:23.196223    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f98e0ab3-2bd3-44a8-9132-6f3521075ba5-original-pull-secret\") pod \"global-pull-secret-syncer-95x9c\" (UID: \"f98e0ab3-2bd3-44a8-9132-6f3521075ba5\") " pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:13:23.196822 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:23.196409    2548 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 23:13:23.196822 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:23.196469    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f98e0ab3-2bd3-44a8-9132-6f3521075ba5-original-pull-secret podName:f98e0ab3-2bd3-44a8-9132-6f3521075ba5 nodeName:}" failed. No retries permitted until 2026-07-23 23:13:27.196451879 +0000 UTC m=+8.940180909 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f98e0ab3-2bd3-44a8-9132-6f3521075ba5-original-pull-secret") pod "global-pull-secret-syncer-95x9c" (UID: "f98e0ab3-2bd3-44a8-9132-6f3521075ba5") : object "kube-system"/"original-pull-secret" not registered
Jul 23 23:13:23.297829 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:23.297236    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5037079f-e554-410b-973a-adef25cd2493-metrics-certs\") pod \"network-metrics-daemon-fkdcf\" (UID: \"5037079f-e554-410b-973a-adef25cd2493\") " pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:13:23.297829 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:23.297415    2548 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 23:13:23.297829 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:23.297488    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5037079f-e554-410b-973a-adef25cd2493-metrics-certs podName:5037079f-e554-410b-973a-adef25cd2493 nodeName:}" failed. No retries permitted until 2026-07-23 23:13:27.297468768 +0000 UTC m=+9.041197810 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5037079f-e554-410b-973a-adef25cd2493-metrics-certs") pod "network-metrics-daemon-fkdcf" (UID: "5037079f-e554-410b-973a-adef25cd2493") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 23:13:23.398515 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:23.397878    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwqtd\" (UniqueName: \"kubernetes.io/projected/4fa58538-875d-4416-954e-289429d66399-kube-api-access-mwqtd\") pod \"network-check-target-fbnfl\" (UID: \"4fa58538-875d-4416-954e-289429d66399\") " pod="openshift-network-diagnostics/network-check-target-fbnfl"
Jul 23 23:13:23.398515 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:23.398061    2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 23:13:23.398515 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:23.398081    2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 23:13:23.398515 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:23.398094    2548 projected.go:196] Error preparing data for projected volume kube-api-access-mwqtd for pod openshift-network-diagnostics/network-check-target-fbnfl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 23:13:23.398515 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:23.398150    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4fa58538-875d-4416-954e-289429d66399-kube-api-access-mwqtd podName:4fa58538-875d-4416-954e-289429d66399 nodeName:}" failed. No retries permitted until 2026-07-23 23:13:27.398133006 +0000 UTC m=+9.141862035 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-mwqtd" (UniqueName: "kubernetes.io/projected/4fa58538-875d-4416-954e-289429d66399-kube-api-access-mwqtd") pod "network-check-target-fbnfl" (UID: "4fa58538-875d-4416-954e-289429d66399") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 23:13:23.742447 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:23.742365    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:13:23.742623 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:23.742550    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:13:23.742695 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:23.742670    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95x9c" podUID="f98e0ab3-2bd3-44a8-9132-6f3521075ba5"
Jul 23 23:13:23.742750 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:23.742698    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fbnfl"
Jul 23 23:13:23.742750 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:23.742536    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fkdcf" podUID="5037079f-e554-410b-973a-adef25cd2493"
Jul 23 23:13:23.742824 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:23.742792    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fbnfl" podUID="4fa58538-875d-4416-954e-289429d66399"
Jul 23 23:13:25.741899 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:25.741769    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:13:25.741899 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:25.741769    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:13:25.741899 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:25.741768    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fbnfl"
Jul 23 23:13:25.742491 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:25.741921    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fkdcf" podUID="5037079f-e554-410b-973a-adef25cd2493"
Jul 23 23:13:25.742491 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:25.742053    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fbnfl" podUID="4fa58538-875d-4416-954e-289429d66399"
Jul 23 23:13:25.742491 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:25.742143    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95x9c" podUID="f98e0ab3-2bd3-44a8-9132-6f3521075ba5"
Jul 23 23:13:27.229355 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:27.229323    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f98e0ab3-2bd3-44a8-9132-6f3521075ba5-original-pull-secret\") pod \"global-pull-secret-syncer-95x9c\" (UID: \"f98e0ab3-2bd3-44a8-9132-6f3521075ba5\") " pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:13:27.229809 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:27.229472    2548 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 23:13:27.229809 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:27.229530    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f98e0ab3-2bd3-44a8-9132-6f3521075ba5-original-pull-secret podName:f98e0ab3-2bd3-44a8-9132-6f3521075ba5 nodeName:}" failed. No retries permitted until 2026-07-23 23:13:35.229512643 +0000 UTC m=+16.973241691 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f98e0ab3-2bd3-44a8-9132-6f3521075ba5-original-pull-secret") pod "global-pull-secret-syncer-95x9c" (UID: "f98e0ab3-2bd3-44a8-9132-6f3521075ba5") : object "kube-system"/"original-pull-secret" not registered
Jul 23 23:13:27.329962 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:27.329864    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5037079f-e554-410b-973a-adef25cd2493-metrics-certs\") pod \"network-metrics-daemon-fkdcf\" (UID: \"5037079f-e554-410b-973a-adef25cd2493\") " pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:13:27.330131 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:27.330097    2548 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 23:13:27.330181 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:27.330170    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5037079f-e554-410b-973a-adef25cd2493-metrics-certs podName:5037079f-e554-410b-973a-adef25cd2493 nodeName:}" failed. No retries permitted until 2026-07-23 23:13:35.330150792 +0000 UTC m=+17.073879825 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5037079f-e554-410b-973a-adef25cd2493-metrics-certs") pod "network-metrics-daemon-fkdcf" (UID: "5037079f-e554-410b-973a-adef25cd2493") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 23:13:27.430610 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:27.430567    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwqtd\" (UniqueName: \"kubernetes.io/projected/4fa58538-875d-4416-954e-289429d66399-kube-api-access-mwqtd\") pod \"network-check-target-fbnfl\" (UID: \"4fa58538-875d-4416-954e-289429d66399\") " pod="openshift-network-diagnostics/network-check-target-fbnfl"
Jul 23 23:13:27.430809 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:27.430770    2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 23:13:27.430809 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:27.430789    2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 23:13:27.430809 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:27.430803    2548 projected.go:196] Error preparing data for projected volume kube-api-access-mwqtd for pod openshift-network-diagnostics/network-check-target-fbnfl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 23:13:27.431024 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:27.430862    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4fa58538-875d-4416-954e-289429d66399-kube-api-access-mwqtd podName:4fa58538-875d-4416-954e-289429d66399 nodeName:}" failed. No retries permitted until 2026-07-23 23:13:35.430842933 +0000 UTC m=+17.174571984 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-mwqtd" (UniqueName: "kubernetes.io/projected/4fa58538-875d-4416-954e-289429d66399-kube-api-access-mwqtd") pod "network-check-target-fbnfl" (UID: "4fa58538-875d-4416-954e-289429d66399") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 23:13:27.742562 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:27.742479    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fbnfl"
Jul 23 23:13:27.742749 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:27.742660    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fbnfl" podUID="4fa58538-875d-4416-954e-289429d66399"
Jul 23 23:13:27.743224 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:27.742859    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:13:27.743224 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:27.742914    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:13:27.743224 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:27.743088    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95x9c" podUID="f98e0ab3-2bd3-44a8-9132-6f3521075ba5"
Jul 23 23:13:27.743224 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:27.743193    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fkdcf" podUID="5037079f-e554-410b-973a-adef25cd2493"
Jul 23 23:13:29.742224 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:29.742183    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fbnfl"
Jul 23 23:13:29.742719 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:29.742242    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:13:29.742719 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:29.742183    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:13:29.742719 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:29.742324    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fbnfl" podUID="4fa58538-875d-4416-954e-289429d66399"
Jul 23 23:13:29.742719 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:29.742479    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95x9c" podUID="f98e0ab3-2bd3-44a8-9132-6f3521075ba5"
Jul 23 23:13:29.742719 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:29.742600    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fkdcf" podUID="5037079f-e554-410b-973a-adef25cd2493"
Jul 23 23:13:31.742034 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:31.741992    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:13:31.742034 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:31.742019    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:13:31.742558 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:31.741992    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fbnfl"
Jul 23 23:13:31.742558 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:31.742119    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95x9c" podUID="f98e0ab3-2bd3-44a8-9132-6f3521075ba5"
Jul 23 23:13:31.742558 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:31.742176    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fbnfl" podUID="4fa58538-875d-4416-954e-289429d66399"
Jul 23 23:13:31.742558 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:31.742248    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fkdcf" podUID="5037079f-e554-410b-973a-adef25cd2493"
Jul 23 23:13:33.742040 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:33.742000    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:13:33.742479 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:33.742000    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:13:33.742479 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:33.742129    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95x9c" podUID="f98e0ab3-2bd3-44a8-9132-6f3521075ba5"
Jul 23 23:13:33.742479 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:33.742005    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fbnfl"
Jul 23 23:13:33.742479 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:33.742193    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fkdcf" podUID="5037079f-e554-410b-973a-adef25cd2493"
Jul 23 23:13:33.742479 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:33.742308    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fbnfl" podUID="4fa58538-875d-4416-954e-289429d66399"
Jul 23 23:13:35.288894 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:35.288817    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f98e0ab3-2bd3-44a8-9132-6f3521075ba5-original-pull-secret\") pod \"global-pull-secret-syncer-95x9c\" (UID: \"f98e0ab3-2bd3-44a8-9132-6f3521075ba5\") " pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:13:35.289409 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:35.289012    2548 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 23:13:35.289409 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:35.289098    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f98e0ab3-2bd3-44a8-9132-6f3521075ba5-original-pull-secret podName:f98e0ab3-2bd3-44a8-9132-6f3521075ba5 nodeName:}" failed. No retries permitted until 2026-07-23 23:13:51.289074722 +0000 UTC m=+33.032803762 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f98e0ab3-2bd3-44a8-9132-6f3521075ba5-original-pull-secret") pod "global-pull-secret-syncer-95x9c" (UID: "f98e0ab3-2bd3-44a8-9132-6f3521075ba5") : object "kube-system"/"original-pull-secret" not registered
Jul 23 23:13:35.389647 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:35.389610    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5037079f-e554-410b-973a-adef25cd2493-metrics-certs\") pod \"network-metrics-daemon-fkdcf\" (UID: \"5037079f-e554-410b-973a-adef25cd2493\") " pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:13:35.389821 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:35.389790    2548 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 23:13:35.389876 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:35.389868    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5037079f-e554-410b-973a-adef25cd2493-metrics-certs podName:5037079f-e554-410b-973a-adef25cd2493 nodeName:}" failed. No retries permitted until 2026-07-23 23:13:51.389851892 +0000 UTC m=+33.133580920 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5037079f-e554-410b-973a-adef25cd2493-metrics-certs") pod "network-metrics-daemon-fkdcf" (UID: "5037079f-e554-410b-973a-adef25cd2493") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 23:13:35.490761 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:35.490725    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwqtd\" (UniqueName: \"kubernetes.io/projected/4fa58538-875d-4416-954e-289429d66399-kube-api-access-mwqtd\") pod \"network-check-target-fbnfl\" (UID: \"4fa58538-875d-4416-954e-289429d66399\") " pod="openshift-network-diagnostics/network-check-target-fbnfl"
Jul 23 23:13:35.490953 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:35.490892    2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 23:13:35.490953 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:35.490913    2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 23:13:35.490953 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:35.490923    2548 projected.go:196] Error preparing data for projected volume kube-api-access-mwqtd for pod openshift-network-diagnostics/network-check-target-fbnfl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 23:13:35.491108 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:35.490989    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4fa58538-875d-4416-954e-289429d66399-kube-api-access-mwqtd podName:4fa58538-875d-4416-954e-289429d66399 nodeName:}" failed. No retries permitted until 2026-07-23 23:13:51.49096147 +0000 UTC m=+33.234690500 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-mwqtd" (UniqueName: "kubernetes.io/projected/4fa58538-875d-4416-954e-289429d66399-kube-api-access-mwqtd") pod "network-check-target-fbnfl" (UID: "4fa58538-875d-4416-954e-289429d66399") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 23:13:35.741672 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:35.741636    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fbnfl"
Jul 23 23:13:35.741942 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:35.741644    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:13:35.741942 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:35.741756    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fbnfl" podUID="4fa58538-875d-4416-954e-289429d66399"
Jul 23 23:13:35.741942 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:35.741853    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fkdcf" podUID="5037079f-e554-410b-973a-adef25cd2493"
Jul 23 23:13:35.741942 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:35.741644    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:13:35.742158 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:35.741954    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95x9c" podUID="f98e0ab3-2bd3-44a8-9132-6f3521075ba5"
Jul 23 23:13:37.742084 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:37.742051    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:13:37.742561 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:37.742051    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fbnfl"
Jul 23 23:13:37.742561 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:37.742187    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95x9c" podUID="f98e0ab3-2bd3-44a8-9132-6f3521075ba5"
Jul 23 23:13:37.742561 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:37.742055    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:13:37.742561 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:37.742257    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fbnfl" podUID="4fa58538-875d-4416-954e-289429d66399"
Jul 23 23:13:37.742561 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:37.742319    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fkdcf" podUID="5037079f-e554-410b-973a-adef25cd2493"
Jul 23 23:13:39.741836 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:39.741557    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:13:39.742271 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:39.741645    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fbnfl"
Jul 23 23:13:39.742271 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:39.741872    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fkdcf" podUID="5037079f-e554-410b-973a-adef25cd2493"
Jul 23 23:13:39.742271 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:39.741691    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:13:39.742271 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:39.742085    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fbnfl" podUID="4fa58538-875d-4416-954e-289429d66399"
Jul 23 23:13:39.742271 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:39.742127    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95x9c" podUID="f98e0ab3-2bd3-44a8-9132-6f3521075ba5"
Jul 23 23:13:39.969857 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:39.969623    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-178.ec2.internal" event={"ID":"9ba0f66b58401a0fb93564ba1a377674","Type":"ContainerStarted","Data":"46383dfb864b25573f3d07815534ab4dcf8b00864f5ae123944245acf2288156"}
Jul 23 23:13:39.971612 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:39.971580    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bvgxv" event={"ID":"7f715492-d857-4ac5-bb35-db1169ae239e","Type":"ContainerStarted","Data":"dac6ce0361a70961ad0e55a679a0be86cb61a7dd8699db7d799d17d1f44cdb57"}
Jul 23 23:13:39.983304 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:39.983262    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cwd9t" event={"ID":"61447916-0a62-4e38-ba70-3ec0fd1bca25","Type":"ContainerStarted","Data":"73737dc3f3878645aa6411ab6726369acccc5548496c0436f04dddd225d6dbf8"}
Jul 23 23:13:39.989677 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:39.989654    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-55rnx" event={"ID":"fad85ae6-00e4-4c30-9d2b-84c83484eaa4","Type":"ContainerStarted","Data":"4c71d9e03b3065418fa3ab97197250d1a9ac3494da112a8cddecc55510b879e2"}
Jul 23 23:13:39.989783 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:39.989733    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-55rnx" event={"ID":"fad85ae6-00e4-4c30-9d2b-84c83484eaa4","Type":"ContainerStarted","Data":"0e28e43edb3424ba836dccbd3be97bb32be2f54e78659b6c7a9f4421b4e8c0db"}
Jul 23 23:13:40.004996 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:40.004900    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-bvgxv" podStartSLOduration=2.260843547 podStartE2EDuration="22.004877549s" totalImagesPullingTime="19.744034002s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 23:13:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:13:40.003821114 +0000 UTC m=+21.747550175" watchObservedRunningTime="2026-07-23 23:13:40.004877549 +0000 UTC m=+21.748606598"
Jul 23 23:13:40.005338 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:40.005282    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-178.ec2.internal" podStartSLOduration=21.005267255 podStartE2EDuration="21.005267255s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 23:13:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:13:39.988661804 +0000 UTC m=+21.732390860" watchObservedRunningTime="2026-07-23 23:13:40.005267255 +0000 UTC m=+21.748996305"
Jul 23 23:13:40.020757 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:40.020680    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-cwd9t" podStartSLOduration=2.387810661 podStartE2EDuration="22.020665535s" totalImagesPullingTime="19.632854874s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 23:13:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:13:40.020638924 +0000 UTC m=+21.764367974" watchObservedRunningTime="2026-07-23 23:13:40.020665535 +0000 UTC m=+21.764394591"
Jul 23 23:13:40.993807 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:40.993771    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-gqbl6" event={"ID":"0bc0f46b-0b82-4a65-95b7-3c355a05c679","Type":"ContainerStarted","Data":"4ee8bb932ebdab1ae02abe1f20d42eabc4a15f0161ada625f48112fee405e394"}
Jul 23 23:13:40.996061 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:40.995956    2548 generic.go:356] "Generic (PLEG): container finished" podID="6bdd18e7-c4ba-4506-8b81-0443611d06da" containerID="41c71045c53305e1f7386abaa74ba09bd07dc086ba9c3b0c5c02dfd2ab6a9cf7" exitCode=0
Jul 23 23:13:40.996061 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:40.996013    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wk67r" event={"ID":"6bdd18e7-c4ba-4506-8b81-0443611d06da","Type":"ContainerDied","Data":"41c71045c53305e1f7386abaa74ba09bd07dc086ba9c3b0c5c02dfd2ab6a9cf7"}
Jul 23 23:13:40.997745 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:40.997721    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-b6n2c" event={"ID":"54cba885-3128-45cb-a350-f05f7dc12dbe","Type":"ContainerStarted","Data":"8ceece9c2aff07e9e4a0f473e0bbaaeafbcc5704703fe00d5c8bec10873f9a75"}
Jul 23 23:13:41.001445 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:41.001413    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-55rnx" event={"ID":"fad85ae6-00e4-4c30-9d2b-84c83484eaa4","Type":"ContainerStarted","Data":"c34062beb3d4b96b23485a8e4be2a948d52eb845c18d5c3a9aeb8cb35f85bb90"}
Jul 23 23:13:41.001540 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:41.001493    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-55rnx" event={"ID":"fad85ae6-00e4-4c30-9d2b-84c83484eaa4","Type":"ContainerStarted","Data":"36f2e194ff33392285ab6ce6a9ec516536e5ab30e0e4f4bd2f0348124ef94b65"}
Jul 23 23:13:41.001540 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:41.001538    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-55rnx" event={"ID":"fad85ae6-00e4-4c30-9d2b-84c83484eaa4","Type":"ContainerStarted","Data":"14329c9fe86bf3ee0c2eed438b07c633bd8f5f25d732ffef36e434bd9bf066fe"}
Jul 23 23:13:41.001644 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:41.001569    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-55rnx" event={"ID":"fad85ae6-00e4-4c30-9d2b-84c83484eaa4","Type":"ContainerStarted","Data":"3c75cd397b36b0926c36ade71f187eb5a48d03e48faf7a07aa006aa0b19f542c"}
Jul 23 23:13:41.003382 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:41.003359    2548 generic.go:356] "Generic (PLEG): container finished" podID="45402979716ef826bbc95cd236f845a5" containerID="61188e7f98691a7b2a31ad3d9c9552803d3570c96e3cf1f6b0013d9e8686ede5" exitCode=0
Jul 23 23:13:41.003483 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:41.003444    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-178.ec2.internal" event={"ID":"45402979716ef826bbc95cd236f845a5","Type":"ContainerDied","Data":"61188e7f98691a7b2a31ad3d9c9552803d3570c96e3cf1f6b0013d9e8686ede5"}
Jul 23 23:13:41.005465 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:41.005437    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rdphw" event={"ID":"ba2ce11c-4a0d-4a34-b106-c34b61838088","Type":"ContainerStarted","Data":"d4c7230d6a1fa70c96ff8c5115b374dfe33064879894a238bc4b09c1e89e6cb7"}
Jul 23 23:13:41.007075 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:41.007056    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sst5n" event={"ID":"93aa8031-e34b-4480-9673-aa50ffa4c41a","Type":"ContainerStarted","Data":"c32f63d3009f82ab906a1329eaa1504305dd98a5814d193e50ff92a79d15b3a3"}
Jul 23 23:13:41.008429 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:41.008409    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mth7q" event={"ID":"56add3c1-56d8-46a3-aaf7-88b91aad4640","Type":"ContainerStarted","Data":"beb79c1cfbeb6f7cd8c2136c44e66871f1fe7637a2a145e4644b6e693760ee13"}
Jul 23 23:13:41.025917 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:41.025882    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-gqbl6" podStartSLOduration=3.3644545790000002 podStartE2EDuration="23.025867747s" totalImagesPullingTime="19.661413168s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 23:13:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:13:41.01082991 +0000 UTC m=+22.754558959" watchObservedRunningTime="2026-07-23 23:13:41.025867747 +0000 UTC m=+22.769596778"
Jul 23 23:13:41.026101 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:41.026077    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rdphw" podStartSLOduration=3.41828987 podStartE2EDuration="23.026071944s" totalImagesPullingTime="19.607782074s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 23:13:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:13:41.025992946 +0000 UTC m=+22.769721976" watchObservedRunningTime="2026-07-23 23:13:41.026071944 +0000 UTC m=+22.769800996"
Jul 23 23:13:41.056863 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:41.056819    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-b6n2c" podStartSLOduration=3.3370118189999998 podStartE2EDuration="23.056803341s" totalImagesPullingTime="19.719791522s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:13:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:13:41.056691545 +0000 UTC m=+22.800420598" watchObservedRunningTime="2026-07-23 23:13:41.056803341 +0000 UTC m=+22.800532426"
Jul 23 23:13:41.100890 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:41.100842    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mth7q" podStartSLOduration=3.425745569 podStartE2EDuration="23.100827412s" totalImagesPullingTime="19.675081843s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 23:13:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:13:41.10037134 +0000 UTC m=+22.844100388" watchObservedRunningTime="2026-07-23 23:13:41.100827412 +0000 UTC m=+22.844556461"
Jul 23 23:13:41.476805 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:41.476657    2548 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 23 23:13:41.636877 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:41.636728    2548 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-23T23:13:41.476801393Z","UUID":"aa3e1bf2-9946-4f30-8566-0a9763fe14e1","Handler":null,"Name":"","Endpoint":""}
Jul 23 23:13:41.638586 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:41.638558    2548 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 23 23:13:41.638721 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:41.638605    2548 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 23 23:13:41.741633 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:41.741605    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:13:41.741832 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:41.741720    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fbnfl"
Jul 23 23:13:41.741832 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:41.741729    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:13:41.741832 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:41.741726    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95x9c" podUID="f98e0ab3-2bd3-44a8-9132-6f3521075ba5"
Jul 23 23:13:41.741832 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:41.741820    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fbnfl" podUID="4fa58538-875d-4416-954e-289429d66399"
Jul 23 23:13:41.742069 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:41.741896    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fkdcf" podUID="5037079f-e554-410b-973a-adef25cd2493"
Jul 23 23:13:42.013291 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:42.013256    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-178.ec2.internal" event={"ID":"45402979716ef826bbc95cd236f845a5","Type":"ContainerStarted","Data":"69f02a577e142134b891bad719ef5938a6f6acf9d566b2f3ccb0d2d34533783c"}
Jul 23 23:13:42.015768 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:42.015635    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sst5n" event={"ID":"93aa8031-e34b-4480-9673-aa50ffa4c41a","Type":"ContainerStarted","Data":"286163d9ddfe8f5fbc6bc7579ea6ce34efb419b2672c691bfb1a246c1d251a89"}
Jul 23 23:13:42.034857 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:42.034809    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-178.ec2.internal" podStartSLOduration=23.03479468 podStartE2EDuration="23.03479468s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 23:13:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:13:42.034264824 +0000 UTC m=+23.777993874" watchObservedRunningTime="2026-07-23 23:13:42.03479468 +0000 UTC m=+23.778523730"
Jul 23 23:13:43.021702 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:43.021601    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-55rnx" event={"ID":"fad85ae6-00e4-4c30-9d2b-84c83484eaa4","Type":"ContainerStarted","Data":"1a73cfed0c084695cdf61aa48c3da10efc4ee98c480e8eee6366883fe0642447"}
Jul 23 23:13:43.024174 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:43.024136    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sst5n" event={"ID":"93aa8031-e34b-4480-9673-aa50ffa4c41a","Type":"ContainerStarted","Data":"f246c7e96c71f79422724d810a3d99e634566d5f5a72feee06a921d4cc60f5e3"}
Jul 23 23:13:43.043607 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:43.043563    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sst5n" podStartSLOduration=2.8798616729999997 podStartE2EDuration="25.043549333s" totalImagesPullingTime="22.16368766s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 23:13:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:13:43.043075111 +0000 UTC m=+24.786804160" watchObservedRunningTime="2026-07-23 23:13:43.043549333 +0000 UTC m=+24.787278382"
Jul 23 23:13:43.146942 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:43.146917    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mth7q_56add3c1-56d8-46a3-aaf7-88b91aad4640/dns-node-resolver/0.log"
Jul 23 23:13:43.742125 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:43.742092    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:13:43.742313 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:43.742104    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:13:43.742313 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:43.742220    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95x9c" podUID="f98e0ab3-2bd3-44a8-9132-6f3521075ba5"
Jul 23 23:13:43.742313 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:43.742296    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fkdcf" podUID="5037079f-e554-410b-973a-adef25cd2493"
Jul 23 23:13:43.742482 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:43.742104    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fbnfl"
Jul 23 23:13:43.742482 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:43.742369    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fbnfl" podUID="4fa58538-875d-4416-954e-289429d66399"
Jul 23 23:13:44.505783 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:44.505751    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rdphw_ba2ce11c-4a0d-4a34-b106-c34b61838088/node-ca/0.log"
Jul 23 23:13:44.877837 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:44.877754    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-b6n2c"
Jul 23 23:13:44.878456 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:44.878424    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-b6n2c"
Jul 23 23:13:45.028725 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:45.028696    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-b6n2c"
Jul 23 23:13:45.029266 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:45.029244    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-b6n2c"
Jul 23 23:13:45.741674 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:45.741482    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:13:45.742083 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:45.741506    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fbnfl"
Jul 23 23:13:45.742083 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:45.741775    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95x9c" podUID="f98e0ab3-2bd3-44a8-9132-6f3521075ba5"
Jul 23 23:13:45.742083 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:45.741541    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:13:45.742083 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:45.741827    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fbnfl" podUID="4fa58538-875d-4416-954e-289429d66399"
Jul 23 23:13:45.742083 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:45.741944    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fkdcf" podUID="5037079f-e554-410b-973a-adef25cd2493"
Jul 23 23:13:46.032646 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:46.032610    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wk67r" event={"ID":"6bdd18e7-c4ba-4506-8b81-0443611d06da","Type":"ContainerStarted","Data":"78c290e89d2b84559f8f533073ce91105cff30d5dc9d39954a79709370447ed3"}
Jul 23 23:13:46.036306 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:46.036254    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-55rnx" event={"ID":"fad85ae6-00e4-4c30-9d2b-84c83484eaa4","Type":"ContainerStarted","Data":"bf08ca4b68e858cb01bdff2b24225270ec9d8d74773d0e0955ba9c41e8ac9464"}
Jul 23 23:13:46.104645 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:46.104601    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-55rnx" podStartSLOduration=8.304625078 podStartE2EDuration="28.104586889s" totalImagesPullingTime="19.799961811s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 23:13:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:13:46.103147629 +0000 UTC m=+27.846876678" watchObservedRunningTime="2026-07-23 23:13:46.104586889 +0000 UTC m=+27.848315938"
Jul 23 23:13:47.040416 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:47.040326    2548 generic.go:356] "Generic (PLEG): container finished" podID="6bdd18e7-c4ba-4506-8b81-0443611d06da" containerID="78c290e89d2b84559f8f533073ce91105cff30d5dc9d39954a79709370447ed3" exitCode=0
Jul 23 23:13:47.041268 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:47.040419    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wk67r" event={"ID":"6bdd18e7-c4ba-4506-8b81-0443611d06da","Type":"ContainerDied","Data":"78c290e89d2b84559f8f533073ce91105cff30d5dc9d39954a79709370447ed3"}
Jul 23 23:13:47.041268 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:47.040545    2548 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 23 23:13:47.041268 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:47.041175    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:47.041268 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:47.041267    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:47.056683 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:47.056660    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:47.056807 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:47.056788    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:47.741599 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:47.741563    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fbnfl"
Jul 23 23:13:47.741599 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:47.741598    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:13:47.741801 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:47.741601    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:13:47.741801 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:47.741667    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fbnfl" podUID="4fa58538-875d-4416-954e-289429d66399"
Jul 23 23:13:47.741801 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:47.741762    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95x9c" podUID="f98e0ab3-2bd3-44a8-9132-6f3521075ba5"
Jul 23 23:13:47.741904 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:47.741835    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fkdcf" podUID="5037079f-e554-410b-973a-adef25cd2493"
Jul 23 23:13:48.043052 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:48.042992    2548 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 23 23:13:48.825853 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:48.825813    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:13:49.046425 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:49.046388    2548 generic.go:356] "Generic (PLEG): container finished" podID="6bdd18e7-c4ba-4506-8b81-0443611d06da" containerID="83efaf5f2fff70b49ce06d2697296ba1105e2ce5ad93488d45a26e4047b2e5d7" exitCode=0
Jul 23 23:13:49.046808 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:49.046483    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wk67r" event={"ID":"6bdd18e7-c4ba-4506-8b81-0443611d06da","Type":"ContainerDied","Data":"83efaf5f2fff70b49ce06d2697296ba1105e2ce5ad93488d45a26e4047b2e5d7"}
Jul 23 23:13:49.742186 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:49.742134    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fbnfl"
Jul 23 23:13:49.742385 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:49.742239    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:13:49.742385 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:49.742244    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fbnfl" podUID="4fa58538-875d-4416-954e-289429d66399"
Jul 23 23:13:49.742385 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:49.742291    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:13:49.742385 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:49.742368    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fkdcf" podUID="5037079f-e554-410b-973a-adef25cd2493"
Jul 23 23:13:49.742605 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:49.742417    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95x9c" podUID="f98e0ab3-2bd3-44a8-9132-6f3521075ba5"
Jul 23 23:13:50.063257 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:50.063180    2548 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-55rnx" podUID="fad85ae6-00e4-4c30-9d2b-84c83484eaa4" containerName="ovnkube-controller" probeResult="failure" output=""
Jul 23 23:13:51.053720 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:51.053682    2548 generic.go:356] "Generic (PLEG): container finished" podID="6bdd18e7-c4ba-4506-8b81-0443611d06da" containerID="406bb9a3a693a4d270ffba81666432e5cefa4b6485082f14f0261a998ca8148e" exitCode=0
Jul 23 23:13:51.053914 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:51.053745    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wk67r" event={"ID":"6bdd18e7-c4ba-4506-8b81-0443611d06da","Type":"ContainerDied","Data":"406bb9a3a693a4d270ffba81666432e5cefa4b6485082f14f0261a998ca8148e"}
Jul 23 23:13:51.307930 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:51.307849    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f98e0ab3-2bd3-44a8-9132-6f3521075ba5-original-pull-secret\") pod \"global-pull-secret-syncer-95x9c\" (UID: \"f98e0ab3-2bd3-44a8-9132-6f3521075ba5\") " pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:13:51.308298 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:51.308029    2548 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 23:13:51.308298 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:51.308091    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f98e0ab3-2bd3-44a8-9132-6f3521075ba5-original-pull-secret podName:f98e0ab3-2bd3-44a8-9132-6f3521075ba5 nodeName:}" failed. No retries permitted until 2026-07-23 23:14:23.308075132 +0000 UTC m=+65.051804164 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f98e0ab3-2bd3-44a8-9132-6f3521075ba5-original-pull-secret") pod "global-pull-secret-syncer-95x9c" (UID: "f98e0ab3-2bd3-44a8-9132-6f3521075ba5") : object "kube-system"/"original-pull-secret" not registered
Jul 23 23:13:51.408211 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:51.408174    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5037079f-e554-410b-973a-adef25cd2493-metrics-certs\") pod \"network-metrics-daemon-fkdcf\" (UID: \"5037079f-e554-410b-973a-adef25cd2493\") " pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:13:51.408386 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:51.408297    2548 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 23:13:51.408386 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:51.408359    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5037079f-e554-410b-973a-adef25cd2493-metrics-certs podName:5037079f-e554-410b-973a-adef25cd2493 nodeName:}" failed. No retries permitted until 2026-07-23 23:14:23.408343731 +0000 UTC m=+65.152072779 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5037079f-e554-410b-973a-adef25cd2493-metrics-certs") pod "network-metrics-daemon-fkdcf" (UID: "5037079f-e554-410b-973a-adef25cd2493") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 23:13:51.508853 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:51.508821    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwqtd\" (UniqueName: \"kubernetes.io/projected/4fa58538-875d-4416-954e-289429d66399-kube-api-access-mwqtd\") pod \"network-check-target-fbnfl\" (UID: \"4fa58538-875d-4416-954e-289429d66399\") " pod="openshift-network-diagnostics/network-check-target-fbnfl"
Jul 23 23:13:51.509026 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:51.508990    2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 23:13:51.509026 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:51.509011    2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 23:13:51.509026 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:51.509020    2548 projected.go:196] Error preparing data for projected volume kube-api-access-mwqtd for pod openshift-network-diagnostics/network-check-target-fbnfl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 23:13:51.509130 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:51.509071    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4fa58538-875d-4416-954e-289429d66399-kube-api-access-mwqtd podName:4fa58538-875d-4416-954e-289429d66399 nodeName:}" failed. No retries permitted until 2026-07-23 23:14:23.509054122 +0000 UTC m=+65.252783148 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-mwqtd" (UniqueName: "kubernetes.io/projected/4fa58538-875d-4416-954e-289429d66399-kube-api-access-mwqtd") pod "network-check-target-fbnfl" (UID: "4fa58538-875d-4416-954e-289429d66399") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 23:13:51.742406 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:51.742375    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:13:51.742559 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:51.742375    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:13:51.742559 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:51.742487    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fkdcf" podUID="5037079f-e554-410b-973a-adef25cd2493"
Jul 23 23:13:51.742559 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:51.742375    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fbnfl"
Jul 23 23:13:51.742714 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:51.742552    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95x9c" podUID="f98e0ab3-2bd3-44a8-9132-6f3521075ba5"
Jul 23 23:13:51.742714 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:51.742655    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fbnfl" podUID="4fa58538-875d-4416-954e-289429d66399"
Jul 23 23:13:53.742290 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:53.742257    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fbnfl"
Jul 23 23:13:53.742714 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:53.742257    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:13:53.742714 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:53.742398    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fbnfl" podUID="4fa58538-875d-4416-954e-289429d66399"
Jul 23 23:13:53.742714 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:53.742483    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95x9c" podUID="f98e0ab3-2bd3-44a8-9132-6f3521075ba5"
Jul 23 23:13:53.742714 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:53.742257    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:13:53.742714 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:53.742598    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fkdcf" podUID="5037079f-e554-410b-973a-adef25cd2493"
Jul 23 23:13:55.742303 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:55.741967    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:13:55.742303 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:55.742236    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:13:55.742679 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:55.742361    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fkdcf" podUID="5037079f-e554-410b-973a-adef25cd2493"
Jul 23 23:13:55.742679 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:55.742401    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95x9c" podUID="f98e0ab3-2bd3-44a8-9132-6f3521075ba5"
Jul 23 23:13:55.742679 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:55.742466    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fbnfl"
Jul 23 23:13:55.742679 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:55.742570    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fbnfl" podUID="4fa58538-875d-4416-954e-289429d66399"
Jul 23 23:13:56.067595 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:56.067484    2548 generic.go:356] "Generic (PLEG): container finished" podID="6bdd18e7-c4ba-4506-8b81-0443611d06da" containerID="34d2d3c86b3be54f6c6f93d7e7ab1bf6250fbcdd4ee5af018c6cdfe8b081a837" exitCode=0
Jul 23 23:13:56.067595 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:56.067561    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wk67r" event={"ID":"6bdd18e7-c4ba-4506-8b81-0443611d06da","Type":"ContainerDied","Data":"34d2d3c86b3be54f6c6f93d7e7ab1bf6250fbcdd4ee5af018c6cdfe8b081a837"}
Jul 23 23:13:57.072285 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:57.072253    2548 generic.go:356] "Generic (PLEG): container finished" podID="6bdd18e7-c4ba-4506-8b81-0443611d06da" containerID="555ce148843acd135cbfd995fb11a64848eaf525632983439b61835aded9ae61" exitCode=0
Jul 23 23:13:57.072667 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:57.072317    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wk67r" event={"ID":"6bdd18e7-c4ba-4506-8b81-0443611d06da","Type":"ContainerDied","Data":"555ce148843acd135cbfd995fb11a64848eaf525632983439b61835aded9ae61"}
Jul 23 23:13:57.742030 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:57.741954    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:13:57.742030 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:57.742034    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:13:57.742249 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:57.741989    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fbnfl"
Jul 23 23:13:57.742249 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:57.742102    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95x9c" podUID="f98e0ab3-2bd3-44a8-9132-6f3521075ba5"
Jul 23 23:13:57.742249 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:57.742185    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fbnfl" podUID="4fa58538-875d-4416-954e-289429d66399"
Jul 23 23:13:57.742339 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:57.742249    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fkdcf" podUID="5037079f-e554-410b-973a-adef25cd2493"
Jul 23 23:13:58.078010 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:58.077918    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wk67r" event={"ID":"6bdd18e7-c4ba-4506-8b81-0443611d06da","Type":"ContainerStarted","Data":"3c222567fd6659735989fba404f98ec11368458ac2147c0ba103d4214ea0ff7a"}
Jul 23 23:13:58.104057 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:58.104010    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wk67r" podStartSLOduration=8.732395426 podStartE2EDuration="40.103995311s" totalImagesPullingTime="31.371599885s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 23:13:18 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:13:58.102903887 +0000 UTC m=+39.846632935" watchObservedRunningTime="2026-07-23 23:13:58.103995311 +0000 UTC m=+39.847724351"
Jul 23 23:13:59.741929 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:59.741886    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:13:59.741929 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:59.741927    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fbnfl"
Jul 23 23:13:59.742477 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:13:59.741995    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:13:59.742477 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:59.742019    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95x9c" podUID="f98e0ab3-2bd3-44a8-9132-6f3521075ba5"
Jul 23 23:13:59.742477 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:59.742163    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fkdcf" podUID="5037079f-e554-410b-973a-adef25cd2493"
Jul 23 23:13:59.742477 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:13:59.742299    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fbnfl" podUID="4fa58538-875d-4416-954e-289429d66399"
Jul 23 23:14:01.742030 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:01.741998    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:14:01.742499 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:01.741995    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fbnfl"
Jul 23 23:14:01.742499 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:14:01.742100    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fkdcf" podUID="5037079f-e554-410b-973a-adef25cd2493"
Jul 23 23:14:01.742499 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:01.742125    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:14:01.742499 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:14:01.742222    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fbnfl" podUID="4fa58538-875d-4416-954e-289429d66399"
Jul 23 23:14:01.742499 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:14:01.742282    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95x9c" podUID="f98e0ab3-2bd3-44a8-9132-6f3521075ba5"
Jul 23 23:14:03.742429 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:03.742394    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:14:03.742908 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:03.742395    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:14:03.742908 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:14:03.742518    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fkdcf" podUID="5037079f-e554-410b-973a-adef25cd2493"
Jul 23 23:14:03.742908 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:03.742395    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fbnfl"
Jul 23 23:14:03.742908 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:14:03.742580    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95x9c" podUID="f98e0ab3-2bd3-44a8-9132-6f3521075ba5"
Jul 23 23:14:03.742908 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:14:03.742641    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fbnfl" podUID="4fa58538-875d-4416-954e-289429d66399"
Jul 23 23:14:05.742356 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:05.742324    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:14:05.742802 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:05.742324    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fbnfl"
Jul 23 23:14:05.742802 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:05.742455    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:14:05.742802 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:14:05.742460    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fkdcf" podUID="5037079f-e554-410b-973a-adef25cd2493"
Jul 23 23:14:05.742802 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:14:05.742521    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fbnfl" podUID="4fa58538-875d-4416-954e-289429d66399"
Jul 23 23:14:05.742802 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:14:05.742578    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95x9c" podUID="f98e0ab3-2bd3-44a8-9132-6f3521075ba5"
Jul 23 23:14:05.806915 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:05.806880    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fbnfl"]
Jul 23 23:14:05.807122 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:05.807036    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-95x9c"]
Jul 23 23:14:05.819755 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:05.819720    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fkdcf"]
Jul 23 23:14:06.095419 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:06.095386    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fbnfl"
Jul 23 23:14:06.095419 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:06.095409    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:14:06.095630 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:14:06.095500    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fbnfl" podUID="4fa58538-875d-4416-954e-289429d66399"
Jul 23 23:14:06.095630 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:06.095524    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:14:06.095630 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:14:06.095597    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95x9c" podUID="f98e0ab3-2bd3-44a8-9132-6f3521075ba5"
Jul 23 23:14:06.095789 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:14:06.095704    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fkdcf" podUID="5037079f-e554-410b-973a-adef25cd2493"
Jul 23 23:14:07.742086 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:07.741880    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:14:07.742525 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:07.741925    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:14:07.742525 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:14:07.742167    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fkdcf" podUID="5037079f-e554-410b-973a-adef25cd2493"
Jul 23 23:14:07.742525 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:07.741925    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fbnfl"
Jul 23 23:14:07.742525 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:14:07.742232    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95x9c" podUID="f98e0ab3-2bd3-44a8-9132-6f3521075ba5"
Jul 23 23:14:07.742525 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:14:07.742312    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fbnfl" podUID="4fa58538-875d-4416-954e-289429d66399"
Jul 23 23:14:09.653021 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.652997    2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-178.ec2.internal" event="NodeReady"
Jul 23 23:14:09.653510 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.653119    2548 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 23 23:14:09.685407 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.685376    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6ccb666b9d-s5gmb"]
Jul 23 23:14:09.700267 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.700241    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-tqqp6"]
Jul 23 23:14:09.700429 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.700368    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6ccb666b9d-s5gmb"
Jul 23 23:14:09.703167 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.703144    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 23 23:14:09.703307 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.703166    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 23 23:14:09.703753 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.703724    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 23 23:14:09.707941 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.707917    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-hxqlc\""
Jul 23 23:14:09.712804 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.712786    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 23 23:14:09.715121 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.715101    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6ccb666b9d-s5gmb"]
Jul 23 23:14:09.715225 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.715140    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-96p7w"]
Jul 23 23:14:09.715285 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.715242    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tqqp6"
Jul 23 23:14:09.717803 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.717781    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 23 23:14:09.717803 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.717794    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-mmpgv\""
Jul 23 23:14:09.717959 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.717798    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 23 23:14:09.717959 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.717784    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 23 23:14:09.730469 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.730451    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tqqp6"]
Jul 23 23:14:09.730469 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.730480    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-96p7w"]
Jul 23 23:14:09.730594 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.730561    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-96p7w"
Jul 23 23:14:09.732993 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.732965    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 23 23:14:09.733056 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.733002    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-m52bp\""
Jul 23 23:14:09.733096 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.733057    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 23 23:14:09.741387 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.741369    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fbnfl"
Jul 23 23:14:09.741567 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.741553    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:14:09.741737 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.741719    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:14:09.743575 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.743534    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 23 23:14:09.743720 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.743712    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rl54t\""
Jul 23 23:14:09.743816 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.743800    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 23 23:14:09.743891 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.743874    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 23 23:14:09.744075 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.744049    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-86k6d\""
Jul 23 23:14:09.744075 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.744067    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 23 23:14:09.802708 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.802674    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-z8bsv"]
Jul 23 23:14:09.822358 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.822333    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-z8bsv"]
Jul 23 23:14:09.822486 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.822451    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-z8bsv"
Jul 23 23:14:09.825191 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.825168    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 23 23:14:09.825315 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.825209    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 23 23:14:09.825315 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.825238    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 23 23:14:09.825315 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.825209    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 23 23:14:09.825315 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.825309    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ckkjh\""
Jul 23 23:14:09.845217 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.845194    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9d88d4a3-c264-4f45-844a-6a2d39e1a1aa-trusted-ca\") pod \"image-registry-6ccb666b9d-s5gmb\" (UID: \"9d88d4a3-c264-4f45-844a-6a2d39e1a1aa\") " pod="openshift-image-registry/image-registry-6ccb666b9d-s5gmb"
Jul 23 23:14:09.845317 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.845232    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9d88d4a3-c264-4f45-844a-6a2d39e1a1aa-registry-tls\") pod \"image-registry-6ccb666b9d-s5gmb\" (UID: \"9d88d4a3-c264-4f45-844a-6a2d39e1a1aa\") " pod="openshift-image-registry/image-registry-6ccb666b9d-s5gmb"
Jul 23 23:14:09.845317 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.845270    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c2143110-0125-4cee-ac64-ac116965e143-cert\") pod \"ingress-canary-tqqp6\" (UID: \"c2143110-0125-4cee-ac64-ac116965e143\") " pod="openshift-ingress-canary/ingress-canary-tqqp6"
Jul 23 23:14:09.845317 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.845305    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9d88d4a3-c264-4f45-844a-6a2d39e1a1aa-registry-certificates\") pod \"image-registry-6ccb666b9d-s5gmb\" (UID: \"9d88d4a3-c264-4f45-844a-6a2d39e1a1aa\") " pod="openshift-image-registry/image-registry-6ccb666b9d-s5gmb"
Jul 23 23:14:09.845413 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.845356    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee636a48-3311-4ae5-a214-5c69fb0f4fc0-tmp-dir\") pod \"dns-default-96p7w\" (UID: \"ee636a48-3311-4ae5-a214-5c69fb0f4fc0\") " pod="openshift-dns/dns-default-96p7w"
Jul 23 23:14:09.845413 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.845381    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9d88d4a3-c264-4f45-844a-6a2d39e1a1aa-installation-pull-secrets\") pod \"image-registry-6ccb666b9d-s5gmb\" (UID: \"9d88d4a3-c264-4f45-844a-6a2d39e1a1aa\") " pod="openshift-image-registry/image-registry-6ccb666b9d-s5gmb"
Jul 23 23:14:09.845413 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.845397    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tswfr\" (UniqueName: \"kubernetes.io/projected/9d88d4a3-c264-4f45-844a-6a2d39e1a1aa-kube-api-access-tswfr\") pod \"image-registry-6ccb666b9d-s5gmb\" (UID: \"9d88d4a3-c264-4f45-844a-6a2d39e1a1aa\") " pod="openshift-image-registry/image-registry-6ccb666b9d-s5gmb"
Jul 23 23:14:09.845500 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.845414    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9d88d4a3-c264-4f45-844a-6a2d39e1a1aa-ca-trust-extracted\") pod \"image-registry-6ccb666b9d-s5gmb\" (UID: \"9d88d4a3-c264-4f45-844a-6a2d39e1a1aa\") " pod="openshift-image-registry/image-registry-6ccb666b9d-s5gmb"
Jul 23 23:14:09.845500 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.845489    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jq4cp\" (UniqueName: \"kubernetes.io/projected/c2143110-0125-4cee-ac64-ac116965e143-kube-api-access-jq4cp\") pod \"ingress-canary-tqqp6\" (UID: \"c2143110-0125-4cee-ac64-ac116965e143\") " pod="openshift-ingress-canary/ingress-canary-tqqp6"
Jul 23 23:14:09.845561 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.845514    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee636a48-3311-4ae5-a214-5c69fb0f4fc0-metrics-tls\") pod \"dns-default-96p7w\" (UID: \"ee636a48-3311-4ae5-a214-5c69fb0f4fc0\") " pod="openshift-dns/dns-default-96p7w"
Jul 23 23:14:09.845561 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.845535    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2jc8\" (UniqueName: \"kubernetes.io/projected/ee636a48-3311-4ae5-a214-5c69fb0f4fc0-kube-api-access-n2jc8\") pod \"dns-default-96p7w\" (UID: \"ee636a48-3311-4ae5-a214-5c69fb0f4fc0\") " pod="openshift-dns/dns-default-96p7w"
Jul 23 23:14:09.845561 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.845552    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ee636a48-3311-4ae5-a214-5c69fb0f4fc0-config-volume\") pod \"dns-default-96p7w\" (UID: \"ee636a48-3311-4ae5-a214-5c69fb0f4fc0\") " pod="openshift-dns/dns-default-96p7w"
Jul 23 23:14:09.845645 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.845568    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9d88d4a3-c264-4f45-844a-6a2d39e1a1aa-image-registry-private-configuration\") pod \"image-registry-6ccb666b9d-s5gmb\" (UID: \"9d88d4a3-c264-4f45-844a-6a2d39e1a1aa\") " pod="openshift-image-registry/image-registry-6ccb666b9d-s5gmb"
Jul 23 23:14:09.845645 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.845592    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9d88d4a3-c264-4f45-844a-6a2d39e1a1aa-bound-sa-token\") pod \"image-registry-6ccb666b9d-s5gmb\" (UID: \"9d88d4a3-c264-4f45-844a-6a2d39e1a1aa\") " pod="openshift-image-registry/image-registry-6ccb666b9d-s5gmb"
Jul 23 23:14:09.946711 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.946625    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9d88d4a3-c264-4f45-844a-6a2d39e1a1aa-registry-tls\") pod \"image-registry-6ccb666b9d-s5gmb\" (UID: \"9d88d4a3-c264-4f45-844a-6a2d39e1a1aa\") " pod="openshift-image-registry/image-registry-6ccb666b9d-s5gmb"
Jul 23 23:14:09.946868 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.946788    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c2143110-0125-4cee-ac64-ac116965e143-cert\") pod \"ingress-canary-tqqp6\" (UID: \"c2143110-0125-4cee-ac64-ac116965e143\") " pod="openshift-ingress-canary/ingress-canary-tqqp6"
Jul 23 23:14:09.946868 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.946820    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9d88d4a3-c264-4f45-844a-6a2d39e1a1aa-registry-certificates\") pod \"image-registry-6ccb666b9d-s5gmb\" (UID: \"9d88d4a3-c264-4f45-844a-6a2d39e1a1aa\") " pod="openshift-image-registry/image-registry-6ccb666b9d-s5gmb"
Jul 23 23:14:09.946868 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.946852    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee636a48-3311-4ae5-a214-5c69fb0f4fc0-tmp-dir\") pod \"dns-default-96p7w\" (UID: \"ee636a48-3311-4ae5-a214-5c69fb0f4fc0\") " pod="openshift-dns/dns-default-96p7w"
Jul 23 23:14:09.947047 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.946888    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9d88d4a3-c264-4f45-844a-6a2d39e1a1aa-installation-pull-secrets\") pod \"image-registry-6ccb666b9d-s5gmb\" (UID: \"9d88d4a3-c264-4f45-844a-6a2d39e1a1aa\") " pod="openshift-image-registry/image-registry-6ccb666b9d-s5gmb"
Jul 23 23:14:09.947047 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.946915    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tswfr\" (UniqueName: \"kubernetes.io/projected/9d88d4a3-c264-4f45-844a-6a2d39e1a1aa-kube-api-access-tswfr\") pod \"image-registry-6ccb666b9d-s5gmb\" (UID: \"9d88d4a3-c264-4f45-844a-6a2d39e1a1aa\") " pod="openshift-image-registry/image-registry-6ccb666b9d-s5gmb"
Jul 23 23:14:09.947047 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.946946    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9d88d4a3-c264-4f45-844a-6a2d39e1a1aa-ca-trust-extracted\") pod \"image-registry-6ccb666b9d-s5gmb\" (UID: \"9d88d4a3-c264-4f45-844a-6a2d39e1a1aa\") " pod="openshift-image-registry/image-registry-6ccb666b9d-s5gmb"
Jul 23 23:14:09.947047 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.946971    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4bdbd7d3-94a5-4f5a-8bec-5da77e14deab-crio-socket\") pod \"insights-runtime-extractor-z8bsv\" (UID: \"4bdbd7d3-94a5-4f5a-8bec-5da77e14deab\") " pod="openshift-insights/insights-runtime-extractor-z8bsv"
Jul 23 23:14:09.947047 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.947024    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4bdbd7d3-94a5-4f5a-8bec-5da77e14deab-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z8bsv\" (UID: \"4bdbd7d3-94a5-4f5a-8bec-5da77e14deab\") " pod="openshift-insights/insights-runtime-extractor-z8bsv"
Jul 23 23:14:09.947287 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.947049    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4bdbd7d3-94a5-4f5a-8bec-5da77e14deab-data-volume\") pod \"insights-runtime-extractor-z8bsv\" (UID: \"4bdbd7d3-94a5-4f5a-8bec-5da77e14deab\") " pod="openshift-insights/insights-runtime-extractor-z8bsv"
Jul 23 23:14:09.947287 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.947124    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jq4cp\" (UniqueName: \"kubernetes.io/projected/c2143110-0125-4cee-ac64-ac116965e143-kube-api-access-jq4cp\") pod \"ingress-canary-tqqp6\" (UID: \"c2143110-0125-4cee-ac64-ac116965e143\") " pod="openshift-ingress-canary/ingress-canary-tqqp6"
Jul 23 23:14:09.947287 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.947146    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee636a48-3311-4ae5-a214-5c69fb0f4fc0-metrics-tls\") pod \"dns-default-96p7w\" (UID: \"ee636a48-3311-4ae5-a214-5c69fb0f4fc0\") " pod="openshift-dns/dns-default-96p7w"
Jul 23 23:14:09.947287 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.947173    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2jc8\" (UniqueName: \"kubernetes.io/projected/ee636a48-3311-4ae5-a214-5c69fb0f4fc0-kube-api-access-n2jc8\") pod \"dns-default-96p7w\" (UID: \"ee636a48-3311-4ae5-a214-5c69fb0f4fc0\") " pod="openshift-dns/dns-default-96p7w"
Jul 23 23:14:09.947287 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.947196    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4bdbd7d3-94a5-4f5a-8bec-5da77e14deab-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z8bsv\" (UID: \"4bdbd7d3-94a5-4f5a-8bec-5da77e14deab\") " pod="openshift-insights/insights-runtime-extractor-z8bsv"
Jul 23 23:14:09.947287 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.947216    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee636a48-3311-4ae5-a214-5c69fb0f4fc0-tmp-dir\") pod \"dns-default-96p7w\" (UID: \"ee636a48-3311-4ae5-a214-5c69fb0f4fc0\") " pod="openshift-dns/dns-default-96p7w"
Jul 23 23:14:09.947287 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.947224    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ee636a48-3311-4ae5-a214-5c69fb0f4fc0-config-volume\") pod \"dns-default-96p7w\" (UID: \"ee636a48-3311-4ae5-a214-5c69fb0f4fc0\") " pod="openshift-dns/dns-default-96p7w"
Jul 23 23:14:09.947287 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.947247    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9d88d4a3-c264-4f45-844a-6a2d39e1a1aa-image-registry-private-configuration\") pod \"image-registry-6ccb666b9d-s5gmb\" (UID: \"9d88d4a3-c264-4f45-844a-6a2d39e1a1aa\") " pod="openshift-image-registry/image-registry-6ccb666b9d-s5gmb"
Jul 23 23:14:09.947287 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.947273    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9d88d4a3-c264-4f45-844a-6a2d39e1a1aa-bound-sa-token\") pod \"image-registry-6ccb666b9d-s5gmb\" (UID: \"9d88d4a3-c264-4f45-844a-6a2d39e1a1aa\") " pod="openshift-image-registry/image-registry-6ccb666b9d-s5gmb"
Jul 23 23:14:09.947709 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.947309    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9d88d4a3-c264-4f45-844a-6a2d39e1a1aa-trusted-ca\") pod \"image-registry-6ccb666b9d-s5gmb\" (UID: \"9d88d4a3-c264-4f45-844a-6a2d39e1a1aa\") " pod="openshift-image-registry/image-registry-6ccb666b9d-s5gmb"
Jul 23 23:14:09.947709 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.947335    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tx28\" (UniqueName: \"kubernetes.io/projected/4bdbd7d3-94a5-4f5a-8bec-5da77e14deab-kube-api-access-6tx28\") pod \"insights-runtime-extractor-z8bsv\" (UID: \"4bdbd7d3-94a5-4f5a-8bec-5da77e14deab\") " pod="openshift-insights/insights-runtime-extractor-z8bsv"
Jul 23 23:14:09.947956 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.947836    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9d88d4a3-c264-4f45-844a-6a2d39e1a1aa-registry-certificates\") pod \"image-registry-6ccb666b9d-s5gmb\" (UID: \"9d88d4a3-c264-4f45-844a-6a2d39e1a1aa\") " pod="openshift-image-registry/image-registry-6ccb666b9d-s5gmb"
Jul 23 23:14:09.948067 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.947969    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ee636a48-3311-4ae5-a214-5c69fb0f4fc0-config-volume\") pod \"dns-default-96p7w\" (UID: \"ee636a48-3311-4ae5-a214-5c69fb0f4fc0\") " pod="openshift-dns/dns-default-96p7w"
Jul 23 23:14:09.948067 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.948012    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9d88d4a3-c264-4f45-844a-6a2d39e1a1aa-ca-trust-extracted\") pod \"image-registry-6ccb666b9d-s5gmb\" (UID: \"9d88d4a3-c264-4f45-844a-6a2d39e1a1aa\") " pod="openshift-image-registry/image-registry-6ccb666b9d-s5gmb"
Jul 23 23:14:09.948960 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.948934    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9d88d4a3-c264-4f45-844a-6a2d39e1a1aa-trusted-ca\") pod \"image-registry-6ccb666b9d-s5gmb\" (UID: \"9d88d4a3-c264-4f45-844a-6a2d39e1a1aa\") " pod="openshift-image-registry/image-registry-6ccb666b9d-s5gmb"
Jul 23 23:14:09.951233 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.951208    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee636a48-3311-4ae5-a214-5c69fb0f4fc0-metrics-tls\") pod \"dns-default-96p7w\" (UID: \"ee636a48-3311-4ae5-a214-5c69fb0f4fc0\") " pod="openshift-dns/dns-default-96p7w"
Jul 23 23:14:09.951331 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.951236    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9d88d4a3-c264-4f45-844a-6a2d39e1a1aa-image-registry-private-configuration\") pod \"image-registry-6ccb666b9d-s5gmb\" (UID: \"9d88d4a3-c264-4f45-844a-6a2d39e1a1aa\") " pod="openshift-image-registry/image-registry-6ccb666b9d-s5gmb"
Jul 23 23:14:09.951331 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.951283    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9d88d4a3-c264-4f45-844a-6a2d39e1a1aa-installation-pull-secrets\") pod \"image-registry-6ccb666b9d-s5gmb\" (UID: \"9d88d4a3-c264-4f45-844a-6a2d39e1a1aa\") " pod="openshift-image-registry/image-registry-6ccb666b9d-s5gmb"
Jul 23 23:14:09.951448 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.951342    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c2143110-0125-4cee-ac64-ac116965e143-cert\") pod \"ingress-canary-tqqp6\" (UID: \"c2143110-0125-4cee-ac64-ac116965e143\") " pod="openshift-ingress-canary/ingress-canary-tqqp6"
Jul 23 23:14:09.951448 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.951351    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9d88d4a3-c264-4f45-844a-6a2d39e1a1aa-registry-tls\") pod \"image-registry-6ccb666b9d-s5gmb\" (UID: \"9d88d4a3-c264-4f45-844a-6a2d39e1a1aa\") " pod="openshift-image-registry/image-registry-6ccb666b9d-s5gmb"
Jul 23 23:14:09.973402 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.973367    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2jc8\" (UniqueName: \"kubernetes.io/projected/ee636a48-3311-4ae5-a214-5c69fb0f4fc0-kube-api-access-n2jc8\") pod \"dns-default-96p7w\" (UID: \"ee636a48-3311-4ae5-a214-5c69fb0f4fc0\") " pod="openshift-dns/dns-default-96p7w"
Jul 23 23:14:09.973532 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.973367    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tswfr\" (UniqueName: \"kubernetes.io/projected/9d88d4a3-c264-4f45-844a-6a2d39e1a1aa-kube-api-access-tswfr\") pod \"image-registry-6ccb666b9d-s5gmb\" (UID: \"9d88d4a3-c264-4f45-844a-6a2d39e1a1aa\") " pod="openshift-image-registry/image-registry-6ccb666b9d-s5gmb"
Jul 23 23:14:09.974129 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.974109    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9d88d4a3-c264-4f45-844a-6a2d39e1a1aa-bound-sa-token\") pod \"image-registry-6ccb666b9d-s5gmb\" (UID: \"9d88d4a3-c264-4f45-844a-6a2d39e1a1aa\") " pod="openshift-image-registry/image-registry-6ccb666b9d-s5gmb"
Jul 23 23:14:09.978360 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.978340    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jq4cp\" (UniqueName: \"kubernetes.io/projected/c2143110-0125-4cee-ac64-ac116965e143-kube-api-access-jq4cp\") pod \"ingress-canary-tqqp6\" (UID: \"c2143110-0125-4cee-ac64-ac116965e143\") " pod="openshift-ingress-canary/ingress-canary-tqqp6"
Jul 23 23:14:09.986828 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:09.986807    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7787698577-zkc2m"]
Jul 23 23:14:10.010275 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.010247    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7787698577-zkc2m"]
Jul 23 23:14:10.010382 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.010352    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7787698577-zkc2m"
Jul 23 23:14:10.013013 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.012993    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 23 23:14:10.013138 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.013122    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 23 23:14:10.013349 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.013189    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 23 23:14:10.013349 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.013263    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 23 23:14:10.013349 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.013276    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 23 23:14:10.013574 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.013546    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 23 23:14:10.013574 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.013561    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 23 23:14:10.013680 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.013577    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-njw8z\""
Jul 23 23:14:10.015673 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.015649    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6ccb666b9d-s5gmb"
Jul 23 23:14:10.024248 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.024228    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tqqp6"
Jul 23 23:14:10.038631 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.038605    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-96p7w"
Jul 23 23:14:10.047895 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.047870    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4bdbd7d3-94a5-4f5a-8bec-5da77e14deab-data-volume\") pod \"insights-runtime-extractor-z8bsv\" (UID: \"4bdbd7d3-94a5-4f5a-8bec-5da77e14deab\") " pod="openshift-insights/insights-runtime-extractor-z8bsv"
Jul 23 23:14:10.048011 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.047926    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4bdbd7d3-94a5-4f5a-8bec-5da77e14deab-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z8bsv\" (UID: \"4bdbd7d3-94a5-4f5a-8bec-5da77e14deab\") " pod="openshift-insights/insights-runtime-extractor-z8bsv"
Jul 23 23:14:10.048011 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.047962    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tx28\" (UniqueName: \"kubernetes.io/projected/4bdbd7d3-94a5-4f5a-8bec-5da77e14deab-kube-api-access-6tx28\") pod \"insights-runtime-extractor-z8bsv\" (UID: \"4bdbd7d3-94a5-4f5a-8bec-5da77e14deab\") " pod="openshift-insights/insights-runtime-extractor-z8bsv"
Jul 23 23:14:10.048135 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.048032    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4bdbd7d3-94a5-4f5a-8bec-5da77e14deab-crio-socket\") pod \"insights-runtime-extractor-z8bsv\" (UID: \"4bdbd7d3-94a5-4f5a-8bec-5da77e14deab\") " pod="openshift-insights/insights-runtime-extractor-z8bsv"
Jul 23 23:14:10.048135 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.048062    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4bdbd7d3-94a5-4f5a-8bec-5da77e14deab-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z8bsv\" (UID: \"4bdbd7d3-94a5-4f5a-8bec-5da77e14deab\") " pod="openshift-insights/insights-runtime-extractor-z8bsv"
Jul 23 23:14:10.048234 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.048169    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4bdbd7d3-94a5-4f5a-8bec-5da77e14deab-crio-socket\") pod \"insights-runtime-extractor-z8bsv\" (UID: \"4bdbd7d3-94a5-4f5a-8bec-5da77e14deab\") " pod="openshift-insights/insights-runtime-extractor-z8bsv"
Jul 23 23:14:10.048234 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.048215    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4bdbd7d3-94a5-4f5a-8bec-5da77e14deab-data-volume\") pod \"insights-runtime-extractor-z8bsv\" (UID: \"4bdbd7d3-94a5-4f5a-8bec-5da77e14deab\") " pod="openshift-insights/insights-runtime-extractor-z8bsv"
Jul 23 23:14:10.050155 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.050121    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4bdbd7d3-94a5-4f5a-8bec-5da77e14deab-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z8bsv\" (UID: \"4bdbd7d3-94a5-4f5a-8bec-5da77e14deab\") " pod="openshift-insights/insights-runtime-extractor-z8bsv"
Jul 23 23:14:10.074609 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.074526    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tx28\" (UniqueName: \"kubernetes.io/projected/4bdbd7d3-94a5-4f5a-8bec-5da77e14deab-kube-api-access-6tx28\") pod \"insights-runtime-extractor-z8bsv\" (UID: \"4bdbd7d3-94a5-4f5a-8bec-5da77e14deab\") " pod="openshift-insights/insights-runtime-extractor-z8bsv"
Jul 23 23:14:10.074769 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.074663    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4bdbd7d3-94a5-4f5a-8bec-5da77e14deab-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z8bsv\" (UID: \"4bdbd7d3-94a5-4f5a-8bec-5da77e14deab\") " pod="openshift-insights/insights-runtime-extractor-z8bsv"
Jul 23 23:14:10.132849 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.132512    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-z8bsv"
Jul 23 23:14:10.149397 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.149331    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/64fd4c42-3b3a-4b99-872a-82078bd89291-console-oauth-config\") pod \"console-7787698577-zkc2m\" (UID: \"64fd4c42-3b3a-4b99-872a-82078bd89291\") " pod="openshift-console/console-7787698577-zkc2m"
Jul 23 23:14:10.149397 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.149378    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/64fd4c42-3b3a-4b99-872a-82078bd89291-service-ca\") pod \"console-7787698577-zkc2m\" (UID: \"64fd4c42-3b3a-4b99-872a-82078bd89291\") " pod="openshift-console/console-7787698577-zkc2m"
Jul 23 23:14:10.149575 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.149424    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/64fd4c42-3b3a-4b99-872a-82078bd89291-console-serving-cert\") pod \"console-7787698577-zkc2m\" (UID: \"64fd4c42-3b3a-4b99-872a-82078bd89291\") " pod="openshift-console/console-7787698577-zkc2m"
Jul 23 23:14:10.149575 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.149473    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgrh9\" (UniqueName: \"kubernetes.io/projected/64fd4c42-3b3a-4b99-872a-82078bd89291-kube-api-access-dgrh9\") pod \"console-7787698577-zkc2m\" (UID: \"64fd4c42-3b3a-4b99-872a-82078bd89291\") " pod="openshift-console/console-7787698577-zkc2m"
Jul 23 23:14:10.149575 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.149531    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/64fd4c42-3b3a-4b99-872a-82078bd89291-oauth-serving-cert\") pod \"console-7787698577-zkc2m\" (UID: \"64fd4c42-3b3a-4b99-872a-82078bd89291\") " pod="openshift-console/console-7787698577-zkc2m"
Jul 23 23:14:10.149575 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.149558    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/64fd4c42-3b3a-4b99-872a-82078bd89291-console-config\") pod \"console-7787698577-zkc2m\" (UID: \"64fd4c42-3b3a-4b99-872a-82078bd89291\") " pod="openshift-console/console-7787698577-zkc2m"
Jul 23 23:14:10.209402 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.209317    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tqqp6"]
Jul 23 23:14:10.212117 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.212057    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6ccb666b9d-s5gmb"]
Jul 23 23:14:10.213542 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:14:10.213503    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2143110_0125_4cee_ac64_ac116965e143.slice/crio-e2855feab2bf3cec7c5efe9c169d2e51dc6fa63e9e60f8c0307535f6fa637c4f WatchSource:0}: Error finding container e2855feab2bf3cec7c5efe9c169d2e51dc6fa63e9e60f8c0307535f6fa637c4f: Status 404 returned error can't find the container with id e2855feab2bf3cec7c5efe9c169d2e51dc6fa63e9e60f8c0307535f6fa637c4f
Jul 23 23:14:10.216457 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:14:10.216394    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d88d4a3_c264_4f45_844a_6a2d39e1a1aa.slice/crio-9fe1bcf64cb3ad0803911606df19dbecd1b2f92f9274b5fe5eb6f40392944b64 WatchSource:0}: Error finding container 9fe1bcf64cb3ad0803911606df19dbecd1b2f92f9274b5fe5eb6f40392944b64: Status 404 returned error can't find the container with id 9fe1bcf64cb3ad0803911606df19dbecd1b2f92f9274b5fe5eb6f40392944b64
Jul 23 23:14:10.221891 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.221869    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-96p7w"]
Jul 23 23:14:10.250486 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.250455    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/64fd4c42-3b3a-4b99-872a-82078bd89291-console-serving-cert\") pod \"console-7787698577-zkc2m\" (UID: \"64fd4c42-3b3a-4b99-872a-82078bd89291\") " pod="openshift-console/console-7787698577-zkc2m"
Jul 23 23:14:10.250607 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.250526    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgrh9\" (UniqueName: \"kubernetes.io/projected/64fd4c42-3b3a-4b99-872a-82078bd89291-kube-api-access-dgrh9\") pod \"console-7787698577-zkc2m\" (UID: \"64fd4c42-3b3a-4b99-872a-82078bd89291\") " pod="openshift-console/console-7787698577-zkc2m"
Jul 23 23:14:10.250607 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.250556    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/64fd4c42-3b3a-4b99-872a-82078bd89291-oauth-serving-cert\") pod \"console-7787698577-zkc2m\" (UID: \"64fd4c42-3b3a-4b99-872a-82078bd89291\") " pod="openshift-console/console-7787698577-zkc2m"
Jul 23 23:14:10.250607 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.250581    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/64fd4c42-3b3a-4b99-872a-82078bd89291-console-config\") pod \"console-7787698577-zkc2m\" (UID: \"64fd4c42-3b3a-4b99-872a-82078bd89291\") " pod="openshift-console/console-7787698577-zkc2m"
Jul 23 23:14:10.250796 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.250761    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/64fd4c42-3b3a-4b99-872a-82078bd89291-console-oauth-config\") pod \"console-7787698577-zkc2m\" (UID: \"64fd4c42-3b3a-4b99-872a-82078bd89291\") " pod="openshift-console/console-7787698577-zkc2m"
Jul 23 23:14:10.251134 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.250883    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/64fd4c42-3b3a-4b99-872a-82078bd89291-service-ca\") pod \"console-7787698577-zkc2m\" (UID: \"64fd4c42-3b3a-4b99-872a-82078bd89291\") " pod="openshift-console/console-7787698577-zkc2m"
Jul 23 23:14:10.251547 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.251501    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/64fd4c42-3b3a-4b99-872a-82078bd89291-console-config\") pod \"console-7787698577-zkc2m\" (UID: \"64fd4c42-3b3a-4b99-872a-82078bd89291\") " pod="openshift-console/console-7787698577-zkc2m"
Jul 23 23:14:10.251647 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.251593    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/64fd4c42-3b3a-4b99-872a-82078bd89291-oauth-serving-cert\") pod \"console-7787698577-zkc2m\" (UID: \"64fd4c42-3b3a-4b99-872a-82078bd89291\") " pod="openshift-console/console-7787698577-zkc2m"
Jul 23 23:14:10.251739 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.251715    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/64fd4c42-3b3a-4b99-872a-82078bd89291-service-ca\") pod \"console-7787698577-zkc2m\" (UID: \"64fd4c42-3b3a-4b99-872a-82078bd89291\") " pod="openshift-console/console-7787698577-zkc2m"
Jul 23 23:14:10.253552 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.253527    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/64fd4c42-3b3a-4b99-872a-82078bd89291-console-serving-cert\") pod \"console-7787698577-zkc2m\" (UID: \"64fd4c42-3b3a-4b99-872a-82078bd89291\") " pod="openshift-console/console-7787698577-zkc2m"
Jul 23 23:14:10.254641 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.254614    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/64fd4c42-3b3a-4b99-872a-82078bd89291-console-oauth-config\") pod \"console-7787698577-zkc2m\" (UID: \"64fd4c42-3b3a-4b99-872a-82078bd89291\") " pod="openshift-console/console-7787698577-zkc2m"
Jul 23 23:14:10.259026 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.259006    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgrh9\" (UniqueName: \"kubernetes.io/projected/64fd4c42-3b3a-4b99-872a-82078bd89291-kube-api-access-dgrh9\") pod \"console-7787698577-zkc2m\" (UID: \"64fd4c42-3b3a-4b99-872a-82078bd89291\") " pod="openshift-console/console-7787698577-zkc2m"
Jul 23 23:14:10.296104 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.296074    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-z8bsv"]
Jul 23 23:14:10.299157 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:14:10.299134    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bdbd7d3_94a5_4f5a_8bec_5da77e14deab.slice/crio-beefa45b9ea33c3b8669839ac8c48760dbebb038500323b8122f5843fb3ac6f5 WatchSource:0}: Error finding container beefa45b9ea33c3b8669839ac8c48760dbebb038500323b8122f5843fb3ac6f5: Status 404 returned error can't find the container with id beefa45b9ea33c3b8669839ac8c48760dbebb038500323b8122f5843fb3ac6f5
Jul 23 23:14:10.320937 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.320914    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7787698577-zkc2m"
Jul 23 23:14:10.455098 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:10.455066    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7787698577-zkc2m"]
Jul 23 23:14:10.458841 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:14:10.458810    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64fd4c42_3b3a_4b99_872a_82078bd89291.slice/crio-6c8215eb64e35418773dcb031ab3b450a0ef360cc6e7b6639c0e64bd5f1a8e99 WatchSource:0}: Error finding container 6c8215eb64e35418773dcb031ab3b450a0ef360cc6e7b6639c0e64bd5f1a8e99: Status 404 returned error can't find the container with id 6c8215eb64e35418773dcb031ab3b450a0ef360cc6e7b6639c0e64bd5f1a8e99
Jul 23 23:14:11.110877 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:11.110839    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7787698577-zkc2m" event={"ID":"64fd4c42-3b3a-4b99-872a-82078bd89291","Type":"ContainerStarted","Data":"6c8215eb64e35418773dcb031ab3b450a0ef360cc6e7b6639c0e64bd5f1a8e99"}
Jul 23 23:14:11.113621 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:11.113554    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tqqp6" event={"ID":"c2143110-0125-4cee-ac64-ac116965e143","Type":"ContainerStarted","Data":"e2855feab2bf3cec7c5efe9c169d2e51dc6fa63e9e60f8c0307535f6fa637c4f"}
Jul 23 23:14:11.116308 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:11.116216    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-96p7w" event={"ID":"ee636a48-3311-4ae5-a214-5c69fb0f4fc0","Type":"ContainerStarted","Data":"ea460c2c9a1f3a34c56444f680b479de764c53794b3a4295d313eb6f3c144d55"}
Jul 23 23:14:11.119021 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:11.118942    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-z8bsv" event={"ID":"4bdbd7d3-94a5-4f5a-8bec-5da77e14deab","Type":"ContainerStarted","Data":"beefa45b9ea33c3b8669839ac8c48760dbebb038500323b8122f5843fb3ac6f5"}
Jul 23 23:14:11.122007 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:11.121314    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6ccb666b9d-s5gmb" event={"ID":"9d88d4a3-c264-4f45-844a-6a2d39e1a1aa","Type":"ContainerStarted","Data":"500cd6bed48df2b0178ceff221db98167ea1b665577e60189cbe44770b7457e2"}
Jul 23 23:14:11.122007 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:11.121362    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6ccb666b9d-s5gmb" event={"ID":"9d88d4a3-c264-4f45-844a-6a2d39e1a1aa","Type":"ContainerStarted","Data":"9fe1bcf64cb3ad0803911606df19dbecd1b2f92f9274b5fe5eb6f40392944b64"}
Jul 23 23:14:11.122007 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:11.121590    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6ccb666b9d-s5gmb"
Jul 23 23:14:11.143587 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:11.143503    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6ccb666b9d-s5gmb" podStartSLOduration=7.143480853 podStartE2EDuration="7.143480853s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:14:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:14:11.141688068 +0000 UTC m=+52.885417118" watchObservedRunningTime="2026-07-23 23:14:11.143480853 +0000 UTC m=+52.887209904"
Jul 23 23:14:18.148046 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:18.147991    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-z8bsv" event={"ID":"4bdbd7d3-94a5-4f5a-8bec-5da77e14deab","Type":"ContainerStarted","Data":"e8d77f97dbaa9fe8ca075fdd534ab3b37af219ba103a11f54224cabc6f33a31f"}
Jul 23 23:14:18.149464 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:18.149431    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7787698577-zkc2m" event={"ID":"64fd4c42-3b3a-4b99-872a-82078bd89291","Type":"ContainerStarted","Data":"1f6e1feea715cdded8389272d5f309f904cebdded11bf13ab699ce055e951c65"}
Jul 23 23:14:18.150928 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:18.150905    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tqqp6" event={"ID":"c2143110-0125-4cee-ac64-ac116965e143","Type":"ContainerStarted","Data":"4f408eb19f20eb19d5bba8967d9688e6c2a592016bd6c6b9ca6b2b421a6f6691"}
Jul 23 23:14:18.152744 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:18.152723    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-96p7w" event={"ID":"ee636a48-3311-4ae5-a214-5c69fb0f4fc0","Type":"ContainerStarted","Data":"d96dd1509742623dc67db7638441082b38199badb51d48af8e531909f8dc72e6"}
Jul 23 23:14:18.152837 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:18.152764    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-96p7w" event={"ID":"ee636a48-3311-4ae5-a214-5c69fb0f4fc0","Type":"ContainerStarted","Data":"dd870daf7fb7d438d54bd55e964d8d8e79150fc6e627fc919bbc43fbc400fc94"}
Jul 23 23:14:18.152885 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:18.152855    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-96p7w"
Jul 23 23:14:18.171616 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:18.171570    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7787698577-zkc2m" podStartSLOduration=2.113226193 podStartE2EDuration="9.171555415s" totalImagesPullingTime="7.058329222s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:14:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:14:18.170180147 +0000 UTC m=+59.913909197" watchObservedRunningTime="2026-07-23 23:14:18.171555415 +0000 UTC m=+59.915284457"
Jul 23 23:14:18.189836 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:18.189783    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-tqqp6" podStartSLOduration=1.8870772310000001 podStartE2EDuration="9.189767576s" totalImagesPullingTime="7.302690345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:14:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:14:18.185363196 +0000 UTC m=+59.929092258" watchObservedRunningTime="2026-07-23 23:14:18.189767576 +0000 UTC m=+59.933496624"
Jul 23 23:14:18.202672 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:18.202622    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-96p7w" podStartSLOduration=1.911000376 podStartE2EDuration="9.202602083s" totalImagesPullingTime="7.291601707s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:14:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:14:18.201997887 +0000 UTC m=+59.945726935" watchObservedRunningTime="2026-07-23 23:14:18.202602083 +0000 UTC m=+59.946331132"
Jul 23 23:14:20.060364 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:20.060330    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-55rnx"
Jul 23 23:14:20.321240 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:20.321142    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7787698577-zkc2m"
Jul 23 23:14:20.321240 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:20.321237    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7787698577-zkc2m"
Jul 23 23:14:20.322625 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:20.322590    2548 patch_prober.go:28] interesting pod/console-7787698577-zkc2m container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.10:8443/health\": dial tcp 10.133.0.10:8443: connect: connection refused" start-of-body=
Jul 23 23:14:20.322757 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:20.322649    2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-7787698577-zkc2m" podUID="64fd4c42-3b3a-4b99-872a-82078bd89291" containerName="console" probeResult="failure" output="Get \"https://10.133.0.10:8443/health\": dial tcp 10.133.0.10:8443: connect: connection refused"
Jul 23 23:14:22.167445 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:22.167391    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-z8bsv" event={"ID":"4bdbd7d3-94a5-4f5a-8bec-5da77e14deab","Type":"ContainerStarted","Data":"bbeaec3872295f78a3cf043a3410806873dc6bc1b0890a450ee1fdaef43c749d"}
Jul 23 23:14:23.353594 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:23.353550    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f98e0ab3-2bd3-44a8-9132-6f3521075ba5-original-pull-secret\") pod \"global-pull-secret-syncer-95x9c\" (UID: \"f98e0ab3-2bd3-44a8-9132-6f3521075ba5\") " pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:14:23.356384 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:23.356359    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 23 23:14:23.368237 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:23.368215    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f98e0ab3-2bd3-44a8-9132-6f3521075ba5-original-pull-secret\") pod \"global-pull-secret-syncer-95x9c\" (UID: \"f98e0ab3-2bd3-44a8-9132-6f3521075ba5\") " pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:14:23.454345 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:23.454308    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5037079f-e554-410b-973a-adef25cd2493-metrics-certs\") pod \"network-metrics-daemon-fkdcf\" (UID: \"5037079f-e554-410b-973a-adef25cd2493\") " pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:14:23.456891 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:23.456872    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 23 23:14:23.467261 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:23.467239    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5037079f-e554-410b-973a-adef25cd2493-metrics-certs\") pod \"network-metrics-daemon-fkdcf\" (UID: \"5037079f-e554-410b-973a-adef25cd2493\") " pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:14:23.555555 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:23.555518    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwqtd\" (UniqueName: \"kubernetes.io/projected/4fa58538-875d-4416-954e-289429d66399-kube-api-access-mwqtd\") pod \"network-check-target-fbnfl\" (UID: \"4fa58538-875d-4416-954e-289429d66399\") " pod="openshift-network-diagnostics/network-check-target-fbnfl"
Jul 23 23:14:23.557986 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:23.557935    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-86k6d\""
Jul 23 23:14:23.558278 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:23.558260    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 23 23:14:23.559985 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:23.559963    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95x9c"
Jul 23 23:14:23.566288 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:23.566265    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fkdcf"
Jul 23 23:14:23.568609 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:23.568471    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 23 23:14:23.579887 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:23.579865    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwqtd\" (UniqueName: \"kubernetes.io/projected/4fa58538-875d-4416-954e-289429d66399-kube-api-access-mwqtd\") pod \"network-check-target-fbnfl\" (UID: \"4fa58538-875d-4416-954e-289429d66399\") " pod="openshift-network-diagnostics/network-check-target-fbnfl"
Jul 23 23:14:23.852222 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:23.852187    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rl54t\""
Jul 23 23:14:23.860591 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:23.860554    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fbnfl"
Jul 23 23:14:24.810679 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:24.810644    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-95x9c"]
Jul 23 23:14:24.814521 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:14:24.814228    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf98e0ab3_2bd3_44a8_9132_6f3521075ba5.slice/crio-e0fc5957edfadd652226915cab0d8ca09e152d4adceea142e29a65d2f6131a3e WatchSource:0}: Error finding container e0fc5957edfadd652226915cab0d8ca09e152d4adceea142e29a65d2f6131a3e: Status 404 returned error can't find the container with id e0fc5957edfadd652226915cab0d8ca09e152d4adceea142e29a65d2f6131a3e
Jul 23 23:14:24.829141 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:24.829113    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fbnfl"]
Jul 23 23:14:24.832567 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:14:24.832541    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fa58538_875d_4416_954e_289429d66399.slice/crio-bcbf919e899e357f4b94dead5b4a08bea37b129f0999588f48b41ac1daa73891 WatchSource:0}: Error finding container bcbf919e899e357f4b94dead5b4a08bea37b129f0999588f48b41ac1daa73891: Status 404 returned error can't find the container with id bcbf919e899e357f4b94dead5b4a08bea37b129f0999588f48b41ac1daa73891
Jul 23 23:14:24.845497 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:24.845470    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fkdcf"]
Jul 23 23:14:24.848588 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:14:24.848564    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5037079f_e554_410b_973a_adef25cd2493.slice/crio-b90953283b731703b1797f47da4a6bd08cae12c2791ddabc775bf7eb0bfb14b8 WatchSource:0}: Error finding container b90953283b731703b1797f47da4a6bd08cae12c2791ddabc775bf7eb0bfb14b8: Status 404 returned error can't find the container with id b90953283b731703b1797f47da4a6bd08cae12c2791ddabc775bf7eb0bfb14b8
Jul 23 23:14:25.176398 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:25.176359    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-95x9c" event={"ID":"f98e0ab3-2bd3-44a8-9132-6f3521075ba5","Type":"ContainerStarted","Data":"e0fc5957edfadd652226915cab0d8ca09e152d4adceea142e29a65d2f6131a3e"}
Jul 23 23:14:25.177380 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:25.177359    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fbnfl" event={"ID":"4fa58538-875d-4416-954e-289429d66399","Type":"ContainerStarted","Data":"bcbf919e899e357f4b94dead5b4a08bea37b129f0999588f48b41ac1daa73891"}
Jul 23 23:14:25.179127 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:25.179095    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-z8bsv" event={"ID":"4bdbd7d3-94a5-4f5a-8bec-5da77e14deab","Type":"ContainerStarted","Data":"e9d031e165c8ad448f4503a413dd704597d354d4fc03afc343858e1116552582"}
Jul 23 23:14:25.180164 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:25.180143    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fkdcf" event={"ID":"5037079f-e554-410b-973a-adef25cd2493","Type":"ContainerStarted","Data":"b90953283b731703b1797f47da4a6bd08cae12c2791ddabc775bf7eb0bfb14b8"}
Jul 23 23:14:25.196752 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:25.196704    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-z8bsv" podStartSLOduration=1.884320797 podStartE2EDuration="16.196687123s" totalImagesPullingTime="14.312366326s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 23:14:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:14:25.195662332 +0000 UTC m=+66.939391391" watchObservedRunningTime="2026-07-23 23:14:25.196687123 +0000 UTC m=+66.940416171"
Jul 23 23:14:27.190132 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:27.190027    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fkdcf" event={"ID":"5037079f-e554-410b-973a-adef25cd2493","Type":"ContainerStarted","Data":"de54463064c1f8edb07874dca90dc3264677c030e9bc7da0e044b02f8f592859"}
Jul 23 23:14:27.190132 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:27.190097    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fkdcf" event={"ID":"5037079f-e554-410b-973a-adef25cd2493","Type":"ContainerStarted","Data":"b8a5e0740f71c7eeff0bbfac114ced137076308d60c15e88d3ca0eb01244352c"}
Jul 23 23:14:27.207427 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:27.206668    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-fkdcf" podStartSLOduration=67.997453276 podStartE2EDuration="1m9.206648161s" totalImagesPullingTime="1.209194885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:13:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:14:27.204421831 +0000 UTC m=+68.948150881" watchObservedRunningTime="2026-07-23 23:14:27.206648161 +0000 UTC m=+68.950377211"
Jul 23 23:14:28.157536 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:28.157507    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-96p7w"
Jul 23 23:14:30.019881 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:30.019847    2548 patch_prober.go:28] interesting pod/image-registry-6ccb666b9d-s5gmb container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 23 23:14:30.020274 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:30.019902    2548 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6ccb666b9d-s5gmb" podUID="9d88d4a3-c264-4f45-844a-6a2d39e1a1aa" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 23 23:14:30.202679 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:30.202635    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-95x9c" event={"ID":"f98e0ab3-2bd3-44a8-9132-6f3521075ba5","Type":"ContainerStarted","Data":"437b7f3c1c07f37a9783264b8f00a6766c76bb1d2c34f5e4868dd0823b65ea9e"}
Jul 23 23:14:30.204087 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:30.204061    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fbnfl" event={"ID":"4fa58538-875d-4416-954e-289429d66399","Type":"ContainerStarted","Data":"a7f268259da5a056c13f0d8bd5f04e056c3afe6a3ab5b11f12eba59a8ea00aca"}
Jul 23 23:14:30.204210 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:30.204188    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-fbnfl"
Jul 23 23:14:30.217767 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:30.217724    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-95x9c" podStartSLOduration=67.517948564 podStartE2EDuration="1m12.217710648s" totalImagesPullingTime="4.699762084s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 23:13:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:14:30.216712792 +0000 UTC m=+71.960441841" watchObservedRunningTime="2026-07-23 23:14:30.217710648 +0000 UTC m=+71.961439696"
Jul 23 23:14:30.230508 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:30.230458    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-fbnfl" podStartSLOduration=67.554080256 podStartE2EDuration="1m12.230444175s" totalImagesPullingTime="4.676363919s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:13:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:14:30.229995999 +0000 UTC m=+71.973725049" watchObservedRunningTime="2026-07-23 23:14:30.230444175 +0000 UTC m=+71.974173250"
Jul 23 23:14:30.321967 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:30.321888    2548 patch_prober.go:28] interesting pod/console-7787698577-zkc2m container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.10:8443/health\": dial tcp 10.133.0.10:8443: connect: connection refused" start-of-body=
Jul 23 23:14:30.321967 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:30.321939    2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-7787698577-zkc2m" podUID="64fd4c42-3b3a-4b99-872a-82078bd89291" containerName="console" probeResult="failure" output="Get \"https://10.133.0.10:8443/health\": dial tcp 10.133.0.10:8443: connect: connection refused"
Jul 23 23:14:32.045011 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:32.044960    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mbgws"]
Jul 23 23:14:32.048162 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:32.048145    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mbgws"
Jul 23 23:14:32.050725 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:32.050703    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 23 23:14:32.050859 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:32.050798    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 23 23:14:32.050922 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:32.050742    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 23 23:14:32.050922 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:32.050751    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 23 23:14:32.051049 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:32.050708    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 23 23:14:32.051816 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:32.051795    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 23 23:14:32.051917 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:32.051818    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-52bjn\""
Jul 23 23:14:32.122096 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:32.122060    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a83f48b0-a080-4412-b457-4d06b0e963e8-metrics-client-ca\") pod \"node-exporter-mbgws\" (UID: \"a83f48b0-a080-4412-b457-4d06b0e963e8\") " pod="openshift-monitoring/node-exporter-mbgws"
Jul 23 23:14:32.122096 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:32.122097    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a83f48b0-a080-4412-b457-4d06b0e963e8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mbgws\" (UID: \"a83f48b0-a080-4412-b457-4d06b0e963e8\") " pod="openshift-monitoring/node-exporter-mbgws"
Jul 23 23:14:32.122370 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:32.122113    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6p6zc\" (UniqueName: \"kubernetes.io/projected/a83f48b0-a080-4412-b457-4d06b0e963e8-kube-api-access-6p6zc\") pod \"node-exporter-mbgws\" (UID: \"a83f48b0-a080-4412-b457-4d06b0e963e8\") " pod="openshift-monitoring/node-exporter-mbgws"
Jul 23 23:14:32.122370 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:32.122138    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a83f48b0-a080-4412-b457-4d06b0e963e8-node-exporter-tls\") pod \"node-exporter-mbgws\" (UID: \"a83f48b0-a080-4412-b457-4d06b0e963e8\") " pod="openshift-monitoring/node-exporter-mbgws"
Jul 23 23:14:32.122370 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:32.122253    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a83f48b0-a080-4412-b457-4d06b0e963e8-node-exporter-accelerators-collector-config\") pod \"node-exporter-mbgws\" (UID: \"a83f48b0-a080-4412-b457-4d06b0e963e8\") " pod="openshift-monitoring/node-exporter-mbgws"
Jul 23 23:14:32.122370 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:32.122287    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a83f48b0-a080-4412-b457-4d06b0e963e8-node-exporter-wtmp\") pod \"node-exporter-mbgws\" (UID: \"a83f48b0-a080-4412-b457-4d06b0e963e8\") " pod="openshift-monitoring/node-exporter-mbgws"
Jul 23 23:14:32.122370 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:32.122318    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a83f48b0-a080-4412-b457-4d06b0e963e8-node-exporter-textfile\") pod \"node-exporter-mbgws\" (UID: \"a83f48b0-a080-4412-b457-4d06b0e963e8\") " pod="openshift-monitoring/node-exporter-mbgws"
Jul 23 23:14:32.122370 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:32.122355    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a83f48b0-a080-4412-b457-4d06b0e963e8-sys\") pod \"node-exporter-mbgws\" (UID: \"a83f48b0-a080-4412-b457-4d06b0e963e8\") " pod="openshift-monitoring/node-exporter-mbgws"
Jul 23 23:14:32.122574 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:32.122383    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a83f48b0-a080-4412-b457-4d06b0e963e8-root\") pod \"node-exporter-mbgws\" (UID: \"a83f48b0-a080-4412-b457-4d06b0e963e8\") " pod="openshift-monitoring/node-exporter-mbgws"
Jul 23 23:14:32.130042 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:32.130001    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6ccb666b9d-s5gmb"
Jul 23 23:14:32.222767 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:32.222735    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a83f48b0-a080-4412-b457-4d06b0e963e8-node-exporter-accelerators-collector-config\") pod \"node-exporter-mbgws\" (UID: \"a83f48b0-a080-4412-b457-4d06b0e963e8\") " pod="openshift-monitoring/node-exporter-mbgws"
Jul 23 23:14:32.222767 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:32.222770    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a83f48b0-a080-4412-b457-4d06b0e963e8-node-exporter-wtmp\") pod \"node-exporter-mbgws\" (UID: \"a83f48b0-a080-4412-b457-4d06b0e963e8\") " pod="openshift-monitoring/node-exporter-mbgws"
Jul 23 23:14:32.223068 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:32.222823    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a83f48b0-a080-4412-b457-4d06b0e963e8-node-exporter-textfile\") pod \"node-exporter-mbgws\" (UID: \"a83f48b0-a080-4412-b457-4d06b0e963e8\") " pod="openshift-monitoring/node-exporter-mbgws"
Jul 23 23:14:32.223068 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:32.222941    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a83f48b0-a080-4412-b457-4d06b0e963e8-sys\") pod \"node-exporter-mbgws\" (UID: \"a83f48b0-a080-4412-b457-4d06b0e963e8\") " pod="openshift-monitoring/node-exporter-mbgws"
Jul 23 23:14:32.223068 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:32.223024    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a83f48b0-a080-4412-b457-4d06b0e963e8-root\") pod \"node-exporter-mbgws\" (UID: \"a83f48b0-a080-4412-b457-4d06b0e963e8\") " pod="openshift-monitoring/node-exporter-mbgws"
Jul 23 23:14:32.223068 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:32.222948    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a83f48b0-a080-4412-b457-4d06b0e963e8-node-exporter-wtmp\") pod \"node-exporter-mbgws\" (UID: \"a83f48b0-a080-4412-b457-4d06b0e963e8\") " pod="openshift-monitoring/node-exporter-mbgws"
Jul 23 23:14:32.223278 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:32.223117    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a83f48b0-a080-4412-b457-4d06b0e963e8-metrics-client-ca\") pod \"node-exporter-mbgws\" (UID: \"a83f48b0-a080-4412-b457-4d06b0e963e8\") " pod="openshift-monitoring/node-exporter-mbgws"
Jul 23 23:14:32.223278 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:32.223118    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a83f48b0-a080-4412-b457-4d06b0e963e8-sys\") pod \"node-exporter-mbgws\" (UID: \"a83f48b0-a080-4412-b457-4d06b0e963e8\") " pod="openshift-monitoring/node-exporter-mbgws"
Jul 23 23:14:32.223278 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:32.223118    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a83f48b0-a080-4412-b457-4d06b0e963e8-root\") pod \"node-exporter-mbgws\" (UID: \"a83f48b0-a080-4412-b457-4d06b0e963e8\") " pod="openshift-monitoring/node-exporter-mbgws"
Jul 23 23:14:32.223278 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:32.223189    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a83f48b0-a080-4412-b457-4d06b0e963e8-node-exporter-textfile\") pod \"node-exporter-mbgws\" (UID: \"a83f48b0-a080-4412-b457-4d06b0e963e8\") " pod="openshift-monitoring/node-exporter-mbgws"
Jul 23 23:14:32.223278 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:32.223237    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a83f48b0-a080-4412-b457-4d06b0e963e8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mbgws\" (UID: \"a83f48b0-a080-4412-b457-4d06b0e963e8\") " pod="openshift-monitoring/node-exporter-mbgws"
Jul 23 23:14:32.223278 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:32.223265    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6p6zc\" (UniqueName: \"kubernetes.io/projected/a83f48b0-a080-4412-b457-4d06b0e963e8-kube-api-access-6p6zc\") pod \"node-exporter-mbgws\" (UID: \"a83f48b0-a080-4412-b457-4d06b0e963e8\") " pod="openshift-monitoring/node-exporter-mbgws"
Jul 23 23:14:32.223602 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:32.223318    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a83f48b0-a080-4412-b457-4d06b0e963e8-node-exporter-tls\") pod \"node-exporter-mbgws\" (UID: \"a83f48b0-a080-4412-b457-4d06b0e963e8\") " pod="openshift-monitoring/node-exporter-mbgws"
Jul 23 23:14:32.223602 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:14:32.223398    2548 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 23 23:14:32.223602 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:14:32.223449    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a83f48b0-a080-4412-b457-4d06b0e963e8-node-exporter-tls podName:a83f48b0-a080-4412-b457-4d06b0e963e8 nodeName:}" failed. No retries permitted until 2026-07-23 23:14:32.723435179 +0000 UTC m=+74.467164209 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/a83f48b0-a080-4412-b457-4d06b0e963e8-node-exporter-tls") pod "node-exporter-mbgws" (UID: "a83f48b0-a080-4412-b457-4d06b0e963e8") : secret "node-exporter-tls" not found
Jul 23 23:14:32.223602 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:32.223475    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a83f48b0-a080-4412-b457-4d06b0e963e8-node-exporter-accelerators-collector-config\") pod \"node-exporter-mbgws\" (UID: \"a83f48b0-a080-4412-b457-4d06b0e963e8\") " pod="openshift-monitoring/node-exporter-mbgws"
Jul 23 23:14:32.223791 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:32.223666    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a83f48b0-a080-4412-b457-4d06b0e963e8-metrics-client-ca\") pod \"node-exporter-mbgws\" (UID: \"a83f48b0-a080-4412-b457-4d06b0e963e8\") " pod="openshift-monitoring/node-exporter-mbgws"
Jul 23 23:14:32.225445 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:32.225427    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a83f48b0-a080-4412-b457-4d06b0e963e8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mbgws\" (UID: \"a83f48b0-a080-4412-b457-4d06b0e963e8\") " pod="openshift-monitoring/node-exporter-mbgws"
Jul 23 23:14:32.232701 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:32.232673    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6p6zc\" (UniqueName: \"kubernetes.io/projected/a83f48b0-a080-4412-b457-4d06b0e963e8-kube-api-access-6p6zc\") pod \"node-exporter-mbgws\" (UID: \"a83f48b0-a080-4412-b457-4d06b0e963e8\") " pod="openshift-monitoring/node-exporter-mbgws"
Jul 23 23:14:32.726761 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:32.726727    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a83f48b0-a080-4412-b457-4d06b0e963e8-node-exporter-tls\") pod \"node-exporter-mbgws\" (UID: \"a83f48b0-a080-4412-b457-4d06b0e963e8\") " pod="openshift-monitoring/node-exporter-mbgws"
Jul 23 23:14:32.729039 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:32.729020    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a83f48b0-a080-4412-b457-4d06b0e963e8-node-exporter-tls\") pod \"node-exporter-mbgws\" (UID: \"a83f48b0-a080-4412-b457-4d06b0e963e8\") " pod="openshift-monitoring/node-exporter-mbgws"
Jul 23 23:14:32.957244 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:32.957196    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mbgws"
Jul 23 23:14:32.970114 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:14:32.970082    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda83f48b0_a080_4412_b457_4d06b0e963e8.slice/crio-51f3ff331eed7b651876b8d920e20f8879a3eb47a5160e986b8fa8d3d82b7626 WatchSource:0}: Error finding container 51f3ff331eed7b651876b8d920e20f8879a3eb47a5160e986b8fa8d3d82b7626: Status 404 returned error can't find the container with id 51f3ff331eed7b651876b8d920e20f8879a3eb47a5160e986b8fa8d3d82b7626
Jul 23 23:14:33.079132 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.079048    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 23 23:14:33.083710 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.083685    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.086203 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.086179    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 23 23:14:33.086459 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.086440    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 23 23:14:33.087309 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.086785    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 23 23:14:33.087309 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.086817    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 23 23:14:33.087309 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.086832    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 23 23:14:33.087309 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.086844    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 23 23:14:33.087309 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.086850    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 23 23:14:33.087309 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.086866    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-45xff\""
Jul 23 23:14:33.087309 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.086900    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 23 23:14:33.087309 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.087251    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 23 23:14:33.097172 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.097151    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 23 23:14:33.130243 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.130210    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/790669a6-8ee8-4a03-9a9c-a701fccb615f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.130408 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.130246    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/790669a6-8ee8-4a03-9a9c-a701fccb615f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.130408 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.130280    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/790669a6-8ee8-4a03-9a9c-a701fccb615f-config-out\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.130408 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.130303    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/790669a6-8ee8-4a03-9a9c-a701fccb615f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.130408 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.130326    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/790669a6-8ee8-4a03-9a9c-a701fccb615f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.130408 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.130354    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/790669a6-8ee8-4a03-9a9c-a701fccb615f-web-config\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.130660 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.130409    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/790669a6-8ee8-4a03-9a9c-a701fccb615f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.130660 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.130431    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twsbw\" (UniqueName: \"kubernetes.io/projected/790669a6-8ee8-4a03-9a9c-a701fccb615f-kube-api-access-twsbw\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.130660 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.130471    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/790669a6-8ee8-4a03-9a9c-a701fccb615f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.130660 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.130522    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/790669a6-8ee8-4a03-9a9c-a701fccb615f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.130660 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.130551    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/790669a6-8ee8-4a03-9a9c-a701fccb615f-config-volume\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.130660 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.130575    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/790669a6-8ee8-4a03-9a9c-a701fccb615f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.130660 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.130601    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/790669a6-8ee8-4a03-9a9c-a701fccb615f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.218016 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.217963    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mbgws" event={"ID":"a83f48b0-a080-4412-b457-4d06b0e963e8","Type":"ContainerStarted","Data":"51f3ff331eed7b651876b8d920e20f8879a3eb47a5160e986b8fa8d3d82b7626"}
Jul 23 23:14:33.231386 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.231362    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/790669a6-8ee8-4a03-9a9c-a701fccb615f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.231522 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.231390    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twsbw\" (UniqueName: \"kubernetes.io/projected/790669a6-8ee8-4a03-9a9c-a701fccb615f-kube-api-access-twsbw\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.231522 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.231426    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/790669a6-8ee8-4a03-9a9c-a701fccb615f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.231522 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.231460    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/790669a6-8ee8-4a03-9a9c-a701fccb615f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.231522 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.231490    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/790669a6-8ee8-4a03-9a9c-a701fccb615f-config-volume\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.231522 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.231512    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/790669a6-8ee8-4a03-9a9c-a701fccb615f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.231788 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.231539    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/790669a6-8ee8-4a03-9a9c-a701fccb615f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.231788 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.231698    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/790669a6-8ee8-4a03-9a9c-a701fccb615f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.231788 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.231731    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/790669a6-8ee8-4a03-9a9c-a701fccb615f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.231788 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.231773    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/790669a6-8ee8-4a03-9a9c-a701fccb615f-config-out\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.232019 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.231800    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/790669a6-8ee8-4a03-9a9c-a701fccb615f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.232019 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.231827    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/790669a6-8ee8-4a03-9a9c-a701fccb615f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.232019 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.231859    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/790669a6-8ee8-4a03-9a9c-a701fccb615f-web-config\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.232447 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.232426    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/790669a6-8ee8-4a03-9a9c-a701fccb615f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.233187 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.233167    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/790669a6-8ee8-4a03-9a9c-a701fccb615f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.233475 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:14:33.233453    2548 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 23 23:14:33.233608 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:14:33.233594    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/790669a6-8ee8-4a03-9a9c-a701fccb615f-alertmanager-trusted-ca-bundle podName:790669a6-8ee8-4a03-9a9c-a701fccb615f nodeName:}" failed. No retries permitted until 2026-07-23 23:14:33.73357126 +0000 UTC m=+75.477300307 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/790669a6-8ee8-4a03-9a9c-a701fccb615f-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "790669a6-8ee8-4a03-9a9c-a701fccb615f") : configmap references non-existent config key: ca-bundle.crt
Jul 23 23:14:33.233731 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:14:33.233719    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/790669a6-8ee8-4a03-9a9c-a701fccb615f-secret-alertmanager-main-tls podName:790669a6-8ee8-4a03-9a9c-a701fccb615f nodeName:}" failed. No retries permitted until 2026-07-23 23:14:33.733706275 +0000 UTC m=+75.477435319 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/790669a6-8ee8-4a03-9a9c-a701fccb615f-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "790669a6-8ee8-4a03-9a9c-a701fccb615f") : secret "alertmanager-main-tls" not found
Jul 23 23:14:33.234666 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.234643    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/790669a6-8ee8-4a03-9a9c-a701fccb615f-config-volume\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.235282 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.235094    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/790669a6-8ee8-4a03-9a9c-a701fccb615f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.235282 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.235110    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/790669a6-8ee8-4a03-9a9c-a701fccb615f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.235282 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.235158    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/790669a6-8ee8-4a03-9a9c-a701fccb615f-config-out\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.235282 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.235238    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/790669a6-8ee8-4a03-9a9c-a701fccb615f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.235282 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.235261    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/790669a6-8ee8-4a03-9a9c-a701fccb615f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.236000 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.235961    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/790669a6-8ee8-4a03-9a9c-a701fccb615f-web-config\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.236059 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.236045    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/790669a6-8ee8-4a03-9a9c-a701fccb615f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.239074 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.239054    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twsbw\" (UniqueName: \"kubernetes.io/projected/790669a6-8ee8-4a03-9a9c-a701fccb615f-kube-api-access-twsbw\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.736621 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.736584    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/790669a6-8ee8-4a03-9a9c-a701fccb615f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.736621 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.736628    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/790669a6-8ee8-4a03-9a9c-a701fccb615f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.737494 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.737471    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/790669a6-8ee8-4a03-9a9c-a701fccb615f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.739359 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.739339    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/790669a6-8ee8-4a03-9a9c-a701fccb615f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"790669a6-8ee8-4a03-9a9c-a701fccb615f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:33.993905 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:33.993811    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 23 23:14:34.124578 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:34.124548    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 23 23:14:34.127076 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:14:34.127047    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod790669a6_8ee8_4a03_9a9c_a701fccb615f.slice/crio-d49128e51c530b9685bf3b38aa92bbc6fed4a050090ba51b9c3ab2080dd32871 WatchSource:0}: Error finding container d49128e51c530b9685bf3b38aa92bbc6fed4a050090ba51b9c3ab2080dd32871: Status 404 returned error can't find the container with id d49128e51c530b9685bf3b38aa92bbc6fed4a050090ba51b9c3ab2080dd32871
Jul 23 23:14:34.222967 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:34.222933    2548 generic.go:356] "Generic (PLEG): container finished" podID="a83f48b0-a080-4412-b457-4d06b0e963e8" containerID="8021e73ddf0c092682ca6c034d868fcc1f2114fd2dcab41d7db6a27ed7ad5365" exitCode=0
Jul 23 23:14:34.223175 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:34.223028    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mbgws" event={"ID":"a83f48b0-a080-4412-b457-4d06b0e963e8","Type":"ContainerDied","Data":"8021e73ddf0c092682ca6c034d868fcc1f2114fd2dcab41d7db6a27ed7ad5365"}
Jul 23 23:14:34.224239 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:34.224211    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"790669a6-8ee8-4a03-9a9c-a701fccb615f","Type":"ContainerStarted","Data":"d49128e51c530b9685bf3b38aa92bbc6fed4a050090ba51b9c3ab2080dd32871"}
Jul 23 23:14:34.706683 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:34.706645    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7787698577-zkc2m"]
Jul 23 23:14:35.006402 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.006315    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-54cdb95f56-7gnrn"]
Jul 23 23:14:35.010538 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.010507    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-54cdb95f56-7gnrn"
Jul 23 23:14:35.013181 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.013154    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 23 23:14:35.013299 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.013241    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 23 23:14:35.013405 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.013387    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-24mt70vhkfs1v\""
Jul 23 23:14:35.013453 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.013388    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 23 23:14:35.013755 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.013567    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 23 23:14:35.013755 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.013661    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 23 23:14:35.013930 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.013782    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-sblpj\""
Jul 23 23:14:35.021839 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.021754    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-54cdb95f56-7gnrn"]
Jul 23 23:14:35.047087 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.045755    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1-metrics-client-ca\") pod \"thanos-querier-54cdb95f56-7gnrn\" (UID: \"ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1\") " pod="openshift-monitoring/thanos-querier-54cdb95f56-7gnrn"
Jul 23 23:14:35.047087 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.045801    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-54cdb95f56-7gnrn\" (UID: \"ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1\") " pod="openshift-monitoring/thanos-querier-54cdb95f56-7gnrn"
Jul 23 23:14:35.047087 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.045835    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-54cdb95f56-7gnrn\" (UID: \"ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1\") " pod="openshift-monitoring/thanos-querier-54cdb95f56-7gnrn"
Jul 23 23:14:35.047087 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.045863    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1-secret-grpc-tls\") pod \"thanos-querier-54cdb95f56-7gnrn\" (UID: \"ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1\") " pod="openshift-monitoring/thanos-querier-54cdb95f56-7gnrn"
Jul 23 23:14:35.047087 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.045890    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-54cdb95f56-7gnrn\" (UID: \"ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1\") " pod="openshift-monitoring/thanos-querier-54cdb95f56-7gnrn"
Jul 23 23:14:35.047087 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.045945    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6xkf\" (UniqueName: \"kubernetes.io/projected/ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1-kube-api-access-x6xkf\") pod \"thanos-querier-54cdb95f56-7gnrn\" (UID: \"ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1\") " pod="openshift-monitoring/thanos-querier-54cdb95f56-7gnrn"
Jul 23 23:14:35.047087 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.046004    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1-secret-thanos-querier-tls\") pod \"thanos-querier-54cdb95f56-7gnrn\" (UID: \"ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1\") " pod="openshift-monitoring/thanos-querier-54cdb95f56-7gnrn"
Jul 23 23:14:35.047087 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.046031    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-54cdb95f56-7gnrn\" (UID: \"ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1\") " pod="openshift-monitoring/thanos-querier-54cdb95f56-7gnrn"
Jul 23 23:14:35.147315 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.147292    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6xkf\" (UniqueName: \"kubernetes.io/projected/ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1-kube-api-access-x6xkf\") pod \"thanos-querier-54cdb95f56-7gnrn\" (UID: \"ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1\") " pod="openshift-monitoring/thanos-querier-54cdb95f56-7gnrn"
Jul 23 23:14:35.147719 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.147331    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1-secret-thanos-querier-tls\") pod \"thanos-querier-54cdb95f56-7gnrn\" (UID: \"ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1\") " pod="openshift-monitoring/thanos-querier-54cdb95f56-7gnrn"
Jul 23 23:14:35.147719 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.147348    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-54cdb95f56-7gnrn\" (UID: \"ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1\") " pod="openshift-monitoring/thanos-querier-54cdb95f56-7gnrn"
Jul 23 23:14:35.147719 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.147376    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1-metrics-client-ca\") pod \"thanos-querier-54cdb95f56-7gnrn\" (UID: \"ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1\") " pod="openshift-monitoring/thanos-querier-54cdb95f56-7gnrn"
Jul 23 23:14:35.147719 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.147403    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-54cdb95f56-7gnrn\" (UID: \"ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1\") " pod="openshift-monitoring/thanos-querier-54cdb95f56-7gnrn"
Jul 23 23:14:35.147719 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.147563    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-54cdb95f56-7gnrn\" (UID: \"ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1\") " pod="openshift-monitoring/thanos-querier-54cdb95f56-7gnrn"
Jul 23 23:14:35.147719 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.147594    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1-secret-grpc-tls\") pod \"thanos-querier-54cdb95f56-7gnrn\" (UID: \"ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1\") " pod="openshift-monitoring/thanos-querier-54cdb95f56-7gnrn"
Jul 23 23:14:35.147719 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.147623    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-54cdb95f56-7gnrn\" (UID: \"ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1\") " pod="openshift-monitoring/thanos-querier-54cdb95f56-7gnrn"
Jul 23 23:14:35.148677 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.148645    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1-metrics-client-ca\") pod \"thanos-querier-54cdb95f56-7gnrn\" (UID: \"ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1\") " pod="openshift-monitoring/thanos-querier-54cdb95f56-7gnrn"
Jul 23 23:14:35.150652 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.150451    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-54cdb95f56-7gnrn\" (UID: \"ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1\") " pod="openshift-monitoring/thanos-querier-54cdb95f56-7gnrn"
Jul 23 23:14:35.150652 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.150598    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-54cdb95f56-7gnrn\" (UID: \"ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1\") " pod="openshift-monitoring/thanos-querier-54cdb95f56-7gnrn"
Jul 23 23:14:35.150811 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.150735    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1-secret-grpc-tls\") pod \"thanos-querier-54cdb95f56-7gnrn\" (UID: \"ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1\") " pod="openshift-monitoring/thanos-querier-54cdb95f56-7gnrn"
Jul 23 23:14:35.150873 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.150859    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-54cdb95f56-7gnrn\" (UID: \"ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1\") " pod="openshift-monitoring/thanos-querier-54cdb95f56-7gnrn"
Jul 23 23:14:35.150934 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.150879    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1-secret-thanos-querier-tls\") pod \"thanos-querier-54cdb95f56-7gnrn\" (UID: \"ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1\") " pod="openshift-monitoring/thanos-querier-54cdb95f56-7gnrn"
Jul 23 23:14:35.151084 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.151065    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-54cdb95f56-7gnrn\" (UID: \"ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1\") " pod="openshift-monitoring/thanos-querier-54cdb95f56-7gnrn"
Jul 23 23:14:35.154919 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.154902    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6xkf\" (UniqueName: \"kubernetes.io/projected/ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1-kube-api-access-x6xkf\") pod \"thanos-querier-54cdb95f56-7gnrn\" (UID: \"ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1\") " pod="openshift-monitoring/thanos-querier-54cdb95f56-7gnrn"
Jul 23 23:14:35.229659 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.229629    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mbgws" event={"ID":"a83f48b0-a080-4412-b457-4d06b0e963e8","Type":"ContainerStarted","Data":"5b21c30ff025edc3d20a83e442738d07fada6d3bd9d41dea1d12dc177e90a0ac"}
Jul 23 23:14:35.229800 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.229691    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mbgws" event={"ID":"a83f48b0-a080-4412-b457-4d06b0e963e8","Type":"ContainerStarted","Data":"1b62693c89f53ea433aa6d3a421aa3ba6812c2b0a58b0e64801f22680b7237be"}
Jul 23 23:14:35.231281 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.231256    2548 generic.go:356] "Generic (PLEG): container finished" podID="790669a6-8ee8-4a03-9a9c-a701fccb615f" containerID="55f296185fa3544bd3aef2a10a7d3fc2cc0386858cdc13ac5826f9cfd415c359" exitCode=0
Jul 23 23:14:35.231405 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.231313    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"790669a6-8ee8-4a03-9a9c-a701fccb615f","Type":"ContainerDied","Data":"55f296185fa3544bd3aef2a10a7d3fc2cc0386858cdc13ac5826f9cfd415c359"}
Jul 23 23:14:35.262003 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.261907    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mbgws" podStartSLOduration=2.306482164 podStartE2EDuration="3.26189377s" totalImagesPullingTime="891.996324ms" totalInitContainerRuntime="63.415282ms" isStatefulPod=true podCreationTimestamp="2026-07-23 23:14:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:14:35.260703452 +0000 UTC m=+77.004432658" watchObservedRunningTime="2026-07-23 23:14:35.26189377 +0000 UTC m=+77.005622843"
Jul 23 23:14:35.320841 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.320814    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-54cdb95f56-7gnrn"
Jul 23 23:14:35.445736 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:35.445679    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-54cdb95f56-7gnrn"]
Jul 23 23:14:35.448107 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:14:35.448084    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea87dce4_8d4f_4378_8f3c_eb7bba3b21f1.slice/crio-41d65eda1626faa000753c50c0cbfe1b89d57cad318d21b56106500a204812ec WatchSource:0}: Error finding container 41d65eda1626faa000753c50c0cbfe1b89d57cad318d21b56106500a204812ec: Status 404 returned error can't find the container with id 41d65eda1626faa000753c50c0cbfe1b89d57cad318d21b56106500a204812ec
Jul 23 23:14:36.236482 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:36.236431    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-54cdb95f56-7gnrn" event={"ID":"ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1","Type":"ContainerStarted","Data":"41d65eda1626faa000753c50c0cbfe1b89d57cad318d21b56106500a204812ec"}
Jul 23 23:14:36.821322 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:36.821296    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-mpdl4"]
Jul 23 23:14:36.824542 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:36.824517    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mpdl4"
Jul 23 23:14:36.827567 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:36.827383    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 23 23:14:36.827567 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:36.827486    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-sghdj\""
Jul 23 23:14:36.833306 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:36.833249    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-mpdl4"]
Jul 23 23:14:36.864046 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:36.864007    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/59f469d8-44e2-4ad9-91d9-a3355df31183-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-mpdl4\" (UID: \"59f469d8-44e2-4ad9-91d9-a3355df31183\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mpdl4"
Jul 23 23:14:36.864176 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:36.864087    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smv8r\" (UniqueName: \"kubernetes.io/projected/59f469d8-44e2-4ad9-91d9-a3355df31183-kube-api-access-smv8r\") pod \"monitoring-plugin-68b7fc9574-mpdl4\" (UID: \"59f469d8-44e2-4ad9-91d9-a3355df31183\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mpdl4"
Jul 23 23:14:36.965173 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:36.965144    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smv8r\" (UniqueName: \"kubernetes.io/projected/59f469d8-44e2-4ad9-91d9-a3355df31183-kube-api-access-smv8r\") pod \"monitoring-plugin-68b7fc9574-mpdl4\" (UID: \"59f469d8-44e2-4ad9-91d9-a3355df31183\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mpdl4"
Jul 23 23:14:36.965313 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:36.965226    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/59f469d8-44e2-4ad9-91d9-a3355df31183-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-mpdl4\" (UID: \"59f469d8-44e2-4ad9-91d9-a3355df31183\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mpdl4"
Jul 23 23:14:36.965388 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:14:36.965343    2548 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 23 23:14:36.965437 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:14:36.965411    2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59f469d8-44e2-4ad9-91d9-a3355df31183-monitoring-plugin-cert podName:59f469d8-44e2-4ad9-91d9-a3355df31183 nodeName:}" failed. No retries permitted until 2026-07-23 23:14:37.465388005 +0000 UTC m=+79.209117036 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/59f469d8-44e2-4ad9-91d9-a3355df31183-monitoring-plugin-cert") pod "monitoring-plugin-68b7fc9574-mpdl4" (UID: "59f469d8-44e2-4ad9-91d9-a3355df31183") : secret "monitoring-plugin-cert" not found
Jul 23 23:14:36.974604 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:36.974543    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smv8r\" (UniqueName: \"kubernetes.io/projected/59f469d8-44e2-4ad9-91d9-a3355df31183-kube-api-access-smv8r\") pod \"monitoring-plugin-68b7fc9574-mpdl4\" (UID: \"59f469d8-44e2-4ad9-91d9-a3355df31183\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mpdl4"
Jul 23 23:14:37.244813 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:37.244731    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"790669a6-8ee8-4a03-9a9c-a701fccb615f","Type":"ContainerStarted","Data":"86fa72e86fa993e34568949f3f1566cf6d85fd98e82c1bf3bc76ae01d6651bde"}
Jul 23 23:14:37.245290 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:37.244810    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"790669a6-8ee8-4a03-9a9c-a701fccb615f","Type":"ContainerStarted","Data":"51d8c5c89600d91175dccd701565d3d9b8b5d3d311fe38eb48572990d431d3ad"}
Jul 23 23:14:37.245290 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:37.244859    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"790669a6-8ee8-4a03-9a9c-a701fccb615f","Type":"ContainerStarted","Data":"6cb3390927b0337a0f9b31d1567e296c0905eb0d94d63a9f97b73a6ab08b6d6c"}
Jul 23 23:14:37.245290 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:37.244892    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"790669a6-8ee8-4a03-9a9c-a701fccb615f","Type":"ContainerStarted","Data":"851254e51c578722a6dde2984350736f4665a9921cfceec18d004c62372b6d6e"}
Jul 23 23:14:37.245290 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:37.244930    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"790669a6-8ee8-4a03-9a9c-a701fccb615f","Type":"ContainerStarted","Data":"dab7430a48a6ef1ac25082c396d43146b817d2f871b49590f2d6f3dea7bbd719"}
Jul 23 23:14:37.469513 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:37.469475    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/59f469d8-44e2-4ad9-91d9-a3355df31183-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-mpdl4\" (UID: \"59f469d8-44e2-4ad9-91d9-a3355df31183\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mpdl4"
Jul 23 23:14:37.472200 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:37.472177    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/59f469d8-44e2-4ad9-91d9-a3355df31183-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-mpdl4\" (UID: \"59f469d8-44e2-4ad9-91d9-a3355df31183\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mpdl4"
Jul 23 23:14:37.737032 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:37.737007    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mpdl4"
Jul 23 23:14:37.882951 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:37.882915    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-mpdl4"]
Jul 23 23:14:37.886138 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:14:37.886072    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59f469d8_44e2_4ad9_91d9_a3355df31183.slice/crio-0fe87bfcb3a5d166e3219256aedc51995de3439ae3c84c0af925791924b6bcef WatchSource:0}: Error finding container 0fe87bfcb3a5d166e3219256aedc51995de3439ae3c84c0af925791924b6bcef: Status 404 returned error can't find the container with id 0fe87bfcb3a5d166e3219256aedc51995de3439ae3c84c0af925791924b6bcef
Jul 23 23:14:37.974669 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:37.974636    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-58cdcb6bc9-k4grx"]
Jul 23 23:14:37.978092 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:37.978067    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58cdcb6bc9-k4grx"
Jul 23 23:14:37.985928 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:37.985902    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 23 23:14:37.988715 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:37.988692    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58cdcb6bc9-k4grx"]
Jul 23 23:14:38.074888 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:38.074855    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49559220-6209-443a-bfaf-f5c61bdb9d5c-trusted-ca-bundle\") pod \"console-58cdcb6bc9-k4grx\" (UID: \"49559220-6209-443a-bfaf-f5c61bdb9d5c\") " pod="openshift-console/console-58cdcb6bc9-k4grx"
Jul 23 23:14:38.075077 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:38.074912    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/49559220-6209-443a-bfaf-f5c61bdb9d5c-console-config\") pod \"console-58cdcb6bc9-k4grx\" (UID: \"49559220-6209-443a-bfaf-f5c61bdb9d5c\") " pod="openshift-console/console-58cdcb6bc9-k4grx"
Jul 23 23:14:38.075077 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:38.074957    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/49559220-6209-443a-bfaf-f5c61bdb9d5c-console-oauth-config\") pod \"console-58cdcb6bc9-k4grx\" (UID: \"49559220-6209-443a-bfaf-f5c61bdb9d5c\") " pod="openshift-console/console-58cdcb6bc9-k4grx"
Jul 23 23:14:38.075077 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:38.074998    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/49559220-6209-443a-bfaf-f5c61bdb9d5c-service-ca\") pod \"console-58cdcb6bc9-k4grx\" (UID: \"49559220-6209-443a-bfaf-f5c61bdb9d5c\") " pod="openshift-console/console-58cdcb6bc9-k4grx"
Jul 23 23:14:38.075077 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:38.075036    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/49559220-6209-443a-bfaf-f5c61bdb9d5c-oauth-serving-cert\") pod \"console-58cdcb6bc9-k4grx\" (UID: \"49559220-6209-443a-bfaf-f5c61bdb9d5c\") " pod="openshift-console/console-58cdcb6bc9-k4grx"
Jul 23 23:14:38.075077 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:38.075052    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dh26v\" (UniqueName: \"kubernetes.io/projected/49559220-6209-443a-bfaf-f5c61bdb9d5c-kube-api-access-dh26v\") pod \"console-58cdcb6bc9-k4grx\" (UID: \"49559220-6209-443a-bfaf-f5c61bdb9d5c\") " pod="openshift-console/console-58cdcb6bc9-k4grx"
Jul 23 23:14:38.075077 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:38.075075    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/49559220-6209-443a-bfaf-f5c61bdb9d5c-console-serving-cert\") pod \"console-58cdcb6bc9-k4grx\" (UID: \"49559220-6209-443a-bfaf-f5c61bdb9d5c\") " pod="openshift-console/console-58cdcb6bc9-k4grx"
Jul 23 23:14:38.175897 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:38.175870    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/49559220-6209-443a-bfaf-f5c61bdb9d5c-console-config\") pod \"console-58cdcb6bc9-k4grx\" (UID: \"49559220-6209-443a-bfaf-f5c61bdb9d5c\") " pod="openshift-console/console-58cdcb6bc9-k4grx"
Jul 23 23:14:38.176029 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:38.175908    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/49559220-6209-443a-bfaf-f5c61bdb9d5c-console-oauth-config\") pod \"console-58cdcb6bc9-k4grx\" (UID: \"49559220-6209-443a-bfaf-f5c61bdb9d5c\") " pod="openshift-console/console-58cdcb6bc9-k4grx"
Jul 23 23:14:38.176029 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:38.175933    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/49559220-6209-443a-bfaf-f5c61bdb9d5c-service-ca\") pod \"console-58cdcb6bc9-k4grx\" (UID: \"49559220-6209-443a-bfaf-f5c61bdb9d5c\") " pod="openshift-console/console-58cdcb6bc9-k4grx"
Jul 23 23:14:38.176029 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:38.175991    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/49559220-6209-443a-bfaf-f5c61bdb9d5c-oauth-serving-cert\") pod \"console-58cdcb6bc9-k4grx\" (UID: \"49559220-6209-443a-bfaf-f5c61bdb9d5c\") " pod="openshift-console/console-58cdcb6bc9-k4grx"
Jul 23 23:14:38.176029 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:38.176019    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dh26v\" (UniqueName: \"kubernetes.io/projected/49559220-6209-443a-bfaf-f5c61bdb9d5c-kube-api-access-dh26v\") pod \"console-58cdcb6bc9-k4grx\" (UID: \"49559220-6209-443a-bfaf-f5c61bdb9d5c\") " pod="openshift-console/console-58cdcb6bc9-k4grx"
Jul 23 23:14:38.176203 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:38.176160    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/49559220-6209-443a-bfaf-f5c61bdb9d5c-console-serving-cert\") pod \"console-58cdcb6bc9-k4grx\" (UID: \"49559220-6209-443a-bfaf-f5c61bdb9d5c\") " pod="openshift-console/console-58cdcb6bc9-k4grx"
Jul 23 23:14:38.176273 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:38.176255    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49559220-6209-443a-bfaf-f5c61bdb9d5c-trusted-ca-bundle\") pod \"console-58cdcb6bc9-k4grx\" (UID: \"49559220-6209-443a-bfaf-f5c61bdb9d5c\") " pod="openshift-console/console-58cdcb6bc9-k4grx"
Jul 23 23:14:38.176780 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:38.176755    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/49559220-6209-443a-bfaf-f5c61bdb9d5c-console-config\") pod \"console-58cdcb6bc9-k4grx\" (UID: \"49559220-6209-443a-bfaf-f5c61bdb9d5c\") " pod="openshift-console/console-58cdcb6bc9-k4grx"
Jul 23 23:14:38.176883 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:38.176801    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/49559220-6209-443a-bfaf-f5c61bdb9d5c-service-ca\") pod \"console-58cdcb6bc9-k4grx\" (UID: \"49559220-6209-443a-bfaf-f5c61bdb9d5c\") " pod="openshift-console/console-58cdcb6bc9-k4grx"
Jul 23 23:14:38.176883 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:38.176830    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/49559220-6209-443a-bfaf-f5c61bdb9d5c-oauth-serving-cert\") pod \"console-58cdcb6bc9-k4grx\" (UID: \"49559220-6209-443a-bfaf-f5c61bdb9d5c\") " pod="openshift-console/console-58cdcb6bc9-k4grx"
Jul 23 23:14:38.177065 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:38.177048    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49559220-6209-443a-bfaf-f5c61bdb9d5c-trusted-ca-bundle\") pod \"console-58cdcb6bc9-k4grx\" (UID: \"49559220-6209-443a-bfaf-f5c61bdb9d5c\") " pod="openshift-console/console-58cdcb6bc9-k4grx"
Jul 23 23:14:38.178197 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:38.178182    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/49559220-6209-443a-bfaf-f5c61bdb9d5c-console-oauth-config\") pod \"console-58cdcb6bc9-k4grx\" (UID: \"49559220-6209-443a-bfaf-f5c61bdb9d5c\") " pod="openshift-console/console-58cdcb6bc9-k4grx"
Jul 23 23:14:38.178436 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:38.178417    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/49559220-6209-443a-bfaf-f5c61bdb9d5c-console-serving-cert\") pod \"console-58cdcb6bc9-k4grx\" (UID: \"49559220-6209-443a-bfaf-f5c61bdb9d5c\") " pod="openshift-console/console-58cdcb6bc9-k4grx"
Jul 23 23:14:38.184758 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:38.184740    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dh26v\" (UniqueName: \"kubernetes.io/projected/49559220-6209-443a-bfaf-f5c61bdb9d5c-kube-api-access-dh26v\") pod \"console-58cdcb6bc9-k4grx\" (UID: \"49559220-6209-443a-bfaf-f5c61bdb9d5c\") " pod="openshift-console/console-58cdcb6bc9-k4grx"
Jul 23 23:14:38.250111 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:38.250079    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mpdl4" event={"ID":"59f469d8-44e2-4ad9-91d9-a3355df31183","Type":"ContainerStarted","Data":"0fe87bfcb3a5d166e3219256aedc51995de3439ae3c84c0af925791924b6bcef"}
Jul 23 23:14:38.253120 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:38.253093    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"790669a6-8ee8-4a03-9a9c-a701fccb615f","Type":"ContainerStarted","Data":"c65a6e7779d7a1aa2321b7efa6842bf4ac155029f4d1d4b072f16be40c7d4f17"}
Jul 23 23:14:38.254996 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:38.254959    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-54cdb95f56-7gnrn" event={"ID":"ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1","Type":"ContainerStarted","Data":"1fb60007d8b31b6b0a4a28c6ca090b6a4016faefbf3465f4828ec8bbf543d5ac"}
Jul 23 23:14:38.255072 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:38.255028    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-54cdb95f56-7gnrn" event={"ID":"ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1","Type":"ContainerStarted","Data":"7799bee4811d7c15e6afb5ecc6849a593f076c5b6f10e003c0efa1009f781e2e"}
Jul 23 23:14:38.255072 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:38.255056    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-54cdb95f56-7gnrn" event={"ID":"ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1","Type":"ContainerStarted","Data":"1f5d80f9ec4507bca88a104cd794fe6062ad98e6e83ac10c850a4df9d5ef1c70"}
Jul 23 23:14:38.283087 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:38.283041    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=1.800067334 podStartE2EDuration="5.283026185s" totalImagesPullingTime="3.463264943s" totalInitContainerRuntime="19.693908ms" isStatefulPod=false podCreationTimestamp="2026-07-23 23:14:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:14:38.278058024 +0000 UTC m=+80.021787071" watchObservedRunningTime="2026-07-23 23:14:38.283026185 +0000 UTC m=+80.026755234"
Jul 23 23:14:38.288594 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:38.288570    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58cdcb6bc9-k4grx"
Jul 23 23:14:38.441452 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:38.441423    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58cdcb6bc9-k4grx"]
Jul 23 23:14:38.445564 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:14:38.445538    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49559220_6209_443a_bfaf_f5c61bdb9d5c.slice/crio-d2d6ad51e129db2be6b9ffeb17cd3a3190311f4b70cc04711d53903b3de70924 WatchSource:0}: Error finding container d2d6ad51e129db2be6b9ffeb17cd3a3190311f4b70cc04711d53903b3de70924: Status 404 returned error can't find the container with id d2d6ad51e129db2be6b9ffeb17cd3a3190311f4b70cc04711d53903b3de70924
Jul 23 23:14:39.260785 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:39.260735    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58cdcb6bc9-k4grx" event={"ID":"49559220-6209-443a-bfaf-f5c61bdb9d5c","Type":"ContainerStarted","Data":"eff84fa657a01d908f030a17c5c8c0970534224b77ce89b346ff5f9f71340d9a"}
Jul 23 23:14:39.261270 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:39.260801    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58cdcb6bc9-k4grx" event={"ID":"49559220-6209-443a-bfaf-f5c61bdb9d5c","Type":"ContainerStarted","Data":"d2d6ad51e129db2be6b9ffeb17cd3a3190311f4b70cc04711d53903b3de70924"}
Jul 23 23:14:39.263730 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:39.263706    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-54cdb95f56-7gnrn" event={"ID":"ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1","Type":"ContainerStarted","Data":"352efe0439d36116553cc302306598677e172aed75ba66c4c2a1e57b40121894"}
Jul 23 23:14:39.263846 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:39.263767    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-54cdb95f56-7gnrn" event={"ID":"ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1","Type":"ContainerStarted","Data":"20c97a1d4b137981a665df1c6f72c6c4433aa55a33112b27b8cc54b5bfeb4d90"}
Jul 23 23:14:39.263846 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:39.263794    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-54cdb95f56-7gnrn" event={"ID":"ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1","Type":"ContainerStarted","Data":"5acd17d7186931e4ab4ffb3ba3cfa82b8705d914bca37664f5bbdbc6d7bdec38"}
Jul 23 23:14:39.277124 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:39.277080    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-58cdcb6bc9-k4grx" podStartSLOduration=2.277066251 podStartE2EDuration="2.277066251s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:14:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:14:39.276109897 +0000 UTC m=+81.019838946" watchObservedRunningTime="2026-07-23 23:14:39.277066251 +0000 UTC m=+81.020795301"
Jul 23 23:14:39.295746 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:39.295695    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-54cdb95f56-7gnrn" podStartSLOduration=2.459020867 podStartE2EDuration="5.295680341s" totalImagesPullingTime="2.836659474s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:14:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:14:39.294486822 +0000 UTC m=+81.038215870" watchObservedRunningTime="2026-07-23 23:14:39.295680341 +0000 UTC m=+81.039409389"
Jul 23 23:14:40.268855 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:40.268815    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mpdl4" event={"ID":"59f469d8-44e2-4ad9-91d9-a3355df31183","Type":"ContainerStarted","Data":"42cc9bf5cb28e9e9dbc006def34b4b7c9169696d0452a7e470199729394218af"}
Jul 23 23:14:40.269345 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:40.269072    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-54cdb95f56-7gnrn"
Jul 23 23:14:40.269345 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:40.269129    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mpdl4"
Jul 23 23:14:40.273759 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:40.273735    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mpdl4"
Jul 23 23:14:40.284432 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:40.284388    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mpdl4" podStartSLOduration=2.501568565 podStartE2EDuration="4.284373124s" totalImagesPullingTime="1.782804559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:14:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:14:40.28363207 +0000 UTC m=+82.027361145" watchObservedRunningTime="2026-07-23 23:14:40.284373124 +0000 UTC m=+82.028102173"
Jul 23 23:14:45.582396 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:45.582354    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58cdcb6bc9-k4grx"]
Jul 23 23:14:45.612020 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:45.611969    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-55c5469fbd-pj4cs"]
Jul 23 23:14:45.669037 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:45.668997    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55c5469fbd-pj4cs"]
Jul 23 23:14:45.669194 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:45.669106    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55c5469fbd-pj4cs"
Jul 23 23:14:45.746450 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:45.746419    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lt568\" (UniqueName: \"kubernetes.io/projected/68554255-21fb-45b5-8f13-d9d91307e565-kube-api-access-lt568\") pod \"console-55c5469fbd-pj4cs\" (UID: \"68554255-21fb-45b5-8f13-d9d91307e565\") " pod="openshift-console/console-55c5469fbd-pj4cs"
Jul 23 23:14:45.746585 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:45.746457    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/68554255-21fb-45b5-8f13-d9d91307e565-oauth-serving-cert\") pod \"console-55c5469fbd-pj4cs\" (UID: \"68554255-21fb-45b5-8f13-d9d91307e565\") " pod="openshift-console/console-55c5469fbd-pj4cs"
Jul 23 23:14:45.746585 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:45.746482    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/68554255-21fb-45b5-8f13-d9d91307e565-console-config\") pod \"console-55c5469fbd-pj4cs\" (UID: \"68554255-21fb-45b5-8f13-d9d91307e565\") " pod="openshift-console/console-55c5469fbd-pj4cs"
Jul 23 23:14:45.746585 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:45.746516    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/68554255-21fb-45b5-8f13-d9d91307e565-service-ca\") pod \"console-55c5469fbd-pj4cs\" (UID: \"68554255-21fb-45b5-8f13-d9d91307e565\") " pod="openshift-console/console-55c5469fbd-pj4cs"
Jul 23 23:14:45.746585 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:45.746574    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68554255-21fb-45b5-8f13-d9d91307e565-trusted-ca-bundle\") pod \"console-55c5469fbd-pj4cs\" (UID: \"68554255-21fb-45b5-8f13-d9d91307e565\") " pod="openshift-console/console-55c5469fbd-pj4cs"
Jul 23 23:14:45.746748 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:45.746615    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/68554255-21fb-45b5-8f13-d9d91307e565-console-serving-cert\") pod \"console-55c5469fbd-pj4cs\" (UID: \"68554255-21fb-45b5-8f13-d9d91307e565\") " pod="openshift-console/console-55c5469fbd-pj4cs"
Jul 23 23:14:45.746748 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:45.746660    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/68554255-21fb-45b5-8f13-d9d91307e565-console-oauth-config\") pod \"console-55c5469fbd-pj4cs\" (UID: \"68554255-21fb-45b5-8f13-d9d91307e565\") " pod="openshift-console/console-55c5469fbd-pj4cs"
Jul 23 23:14:45.847766 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:45.847689    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/68554255-21fb-45b5-8f13-d9d91307e565-console-serving-cert\") pod \"console-55c5469fbd-pj4cs\" (UID: \"68554255-21fb-45b5-8f13-d9d91307e565\") " pod="openshift-console/console-55c5469fbd-pj4cs"
Jul 23 23:14:45.847766 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:45.847736    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/68554255-21fb-45b5-8f13-d9d91307e565-console-oauth-config\") pod \"console-55c5469fbd-pj4cs\" (UID: \"68554255-21fb-45b5-8f13-d9d91307e565\") " pod="openshift-console/console-55c5469fbd-pj4cs"
Jul 23 23:14:45.847766 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:45.847761    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lt568\" (UniqueName: \"kubernetes.io/projected/68554255-21fb-45b5-8f13-d9d91307e565-kube-api-access-lt568\") pod \"console-55c5469fbd-pj4cs\" (UID: \"68554255-21fb-45b5-8f13-d9d91307e565\") " pod="openshift-console/console-55c5469fbd-pj4cs"
Jul 23 23:14:45.847965 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:45.847879    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/68554255-21fb-45b5-8f13-d9d91307e565-oauth-serving-cert\") pod \"console-55c5469fbd-pj4cs\" (UID: \"68554255-21fb-45b5-8f13-d9d91307e565\") " pod="openshift-console/console-55c5469fbd-pj4cs"
Jul 23 23:14:45.847965 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:45.847929    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/68554255-21fb-45b5-8f13-d9d91307e565-console-config\") pod \"console-55c5469fbd-pj4cs\" (UID: \"68554255-21fb-45b5-8f13-d9d91307e565\") " pod="openshift-console/console-55c5469fbd-pj4cs"
Jul 23 23:14:45.847965 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:45.847954    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/68554255-21fb-45b5-8f13-d9d91307e565-service-ca\") pod \"console-55c5469fbd-pj4cs\" (UID: \"68554255-21fb-45b5-8f13-d9d91307e565\") " pod="openshift-console/console-55c5469fbd-pj4cs"
Jul 23 23:14:45.848134 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:45.848011    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68554255-21fb-45b5-8f13-d9d91307e565-trusted-ca-bundle\") pod \"console-55c5469fbd-pj4cs\" (UID: \"68554255-21fb-45b5-8f13-d9d91307e565\") " pod="openshift-console/console-55c5469fbd-pj4cs"
Jul 23 23:14:45.848679 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:45.848656    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/68554255-21fb-45b5-8f13-d9d91307e565-service-ca\") pod \"console-55c5469fbd-pj4cs\" (UID: \"68554255-21fb-45b5-8f13-d9d91307e565\") " pod="openshift-console/console-55c5469fbd-pj4cs"
Jul 23 23:14:45.848766 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:45.848696    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/68554255-21fb-45b5-8f13-d9d91307e565-oauth-serving-cert\") pod \"console-55c5469fbd-pj4cs\" (UID: \"68554255-21fb-45b5-8f13-d9d91307e565\") " pod="openshift-console/console-55c5469fbd-pj4cs"
Jul 23 23:14:45.848766 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:45.848740    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/68554255-21fb-45b5-8f13-d9d91307e565-console-config\") pod \"console-55c5469fbd-pj4cs\" (UID: \"68554255-21fb-45b5-8f13-d9d91307e565\") " pod="openshift-console/console-55c5469fbd-pj4cs"
Jul 23 23:14:45.848865 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:45.848850    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68554255-21fb-45b5-8f13-d9d91307e565-trusted-ca-bundle\") pod \"console-55c5469fbd-pj4cs\" (UID: \"68554255-21fb-45b5-8f13-d9d91307e565\") " pod="openshift-console/console-55c5469fbd-pj4cs"
Jul 23 23:14:45.850202 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:45.850181    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/68554255-21fb-45b5-8f13-d9d91307e565-console-oauth-config\") pod \"console-55c5469fbd-pj4cs\" (UID: \"68554255-21fb-45b5-8f13-d9d91307e565\") " pod="openshift-console/console-55c5469fbd-pj4cs"
Jul 23 23:14:45.850449 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:45.850430    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/68554255-21fb-45b5-8f13-d9d91307e565-console-serving-cert\") pod \"console-55c5469fbd-pj4cs\" (UID: \"68554255-21fb-45b5-8f13-d9d91307e565\") " pod="openshift-console/console-55c5469fbd-pj4cs"
Jul 23 23:14:45.855357 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:45.855339    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lt568\" (UniqueName: \"kubernetes.io/projected/68554255-21fb-45b5-8f13-d9d91307e565-kube-api-access-lt568\") pod \"console-55c5469fbd-pj4cs\" (UID: \"68554255-21fb-45b5-8f13-d9d91307e565\") " pod="openshift-console/console-55c5469fbd-pj4cs"
Jul 23 23:14:45.978537 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:45.978502    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55c5469fbd-pj4cs"
Jul 23 23:14:46.136821 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:46.136741    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55c5469fbd-pj4cs"]
Jul 23 23:14:46.139542 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:14:46.139510    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68554255_21fb_45b5_8f13_d9d91307e565.slice/crio-7e967020c128c3384c1d819448b597ac3825d94e1f423062782921de3aca794c WatchSource:0}: Error finding container 7e967020c128c3384c1d819448b597ac3825d94e1f423062782921de3aca794c: Status 404 returned error can't find the container with id 7e967020c128c3384c1d819448b597ac3825d94e1f423062782921de3aca794c
Jul 23 23:14:46.279490 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:46.279461    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-54cdb95f56-7gnrn"
Jul 23 23:14:46.291727 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:46.291699    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55c5469fbd-pj4cs" event={"ID":"68554255-21fb-45b5-8f13-d9d91307e565","Type":"ContainerStarted","Data":"1c6f0ccd9fe9f5648803081d36b83a1df0cc5ee4580c76c2f2ca7b0a4c6d00c2"}
Jul 23 23:14:46.291853 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:46.291749    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55c5469fbd-pj4cs" event={"ID":"68554255-21fb-45b5-8f13-d9d91307e565","Type":"ContainerStarted","Data":"7e967020c128c3384c1d819448b597ac3825d94e1f423062782921de3aca794c"}
Jul 23 23:14:46.319496 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:46.319445    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-55c5469fbd-pj4cs" podStartSLOduration=1.3194213160000001 podStartE2EDuration="1.319421316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:14:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:14:46.318568209 +0000 UTC m=+88.062297260" watchObservedRunningTime="2026-07-23 23:14:46.319421316 +0000 UTC m=+88.063150364"
Jul 23 23:14:48.288794 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:48.288753    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-58cdcb6bc9-k4grx"
Jul 23 23:14:55.979228 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:55.979185    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-55c5469fbd-pj4cs"
Jul 23 23:14:55.979647 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:55.979276    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-55c5469fbd-pj4cs"
Jul 23 23:14:55.984045 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:55.984022    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-55c5469fbd-pj4cs"
Jul 23 23:14:56.332918 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:56.332836    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-55c5469fbd-pj4cs"
Jul 23 23:14:59.727555 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:14:59.727490    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7787698577-zkc2m" podUID="64fd4c42-3b3a-4b99-872a-82078bd89291" containerName="console" containerID="cri-o://1f6e1feea715cdded8389272d5f309f904cebdded11bf13ab699ce055e951c65" gracePeriod=15
Jul 23 23:15:00.345364 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:00.345332    2548 generic.go:356] "Generic (PLEG): container finished" podID="64fd4c42-3b3a-4b99-872a-82078bd89291" containerID="1f6e1feea715cdded8389272d5f309f904cebdded11bf13ab699ce055e951c65" exitCode=0
Jul 23 23:15:00.345520 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:00.345381    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7787698577-zkc2m" event={"ID":"64fd4c42-3b3a-4b99-872a-82078bd89291","Type":"ContainerDied","Data":"1f6e1feea715cdded8389272d5f309f904cebdded11bf13ab699ce055e951c65"}
Jul 23 23:15:00.498791 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:00.498769    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7787698577-zkc2m"
Jul 23 23:15:00.572587 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:00.572543    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/64fd4c42-3b3a-4b99-872a-82078bd89291-console-serving-cert\") pod \"64fd4c42-3b3a-4b99-872a-82078bd89291\" (UID: \"64fd4c42-3b3a-4b99-872a-82078bd89291\") "
Jul 23 23:15:00.572782 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:00.572610    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/64fd4c42-3b3a-4b99-872a-82078bd89291-console-oauth-config\") pod \"64fd4c42-3b3a-4b99-872a-82078bd89291\" (UID: \"64fd4c42-3b3a-4b99-872a-82078bd89291\") "
Jul 23 23:15:00.572782 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:00.572642    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgrh9\" (UniqueName: \"kubernetes.io/projected/64fd4c42-3b3a-4b99-872a-82078bd89291-kube-api-access-dgrh9\") pod \"64fd4c42-3b3a-4b99-872a-82078bd89291\" (UID: \"64fd4c42-3b3a-4b99-872a-82078bd89291\") "
Jul 23 23:15:00.572782 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:00.572662    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/64fd4c42-3b3a-4b99-872a-82078bd89291-console-config\") pod \"64fd4c42-3b3a-4b99-872a-82078bd89291\" (UID: \"64fd4c42-3b3a-4b99-872a-82078bd89291\") "
Jul 23 23:15:00.572782 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:00.572695    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/64fd4c42-3b3a-4b99-872a-82078bd89291-service-ca\") pod \"64fd4c42-3b3a-4b99-872a-82078bd89291\" (UID: \"64fd4c42-3b3a-4b99-872a-82078bd89291\") "
Jul 23 23:15:00.572962 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:00.572791    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/64fd4c42-3b3a-4b99-872a-82078bd89291-oauth-serving-cert\") pod \"64fd4c42-3b3a-4b99-872a-82078bd89291\" (UID: \"64fd4c42-3b3a-4b99-872a-82078bd89291\") "
Jul 23 23:15:00.573208 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:00.573183    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64fd4c42-3b3a-4b99-872a-82078bd89291-service-ca" (OuterVolumeSpecName: "service-ca") pod "64fd4c42-3b3a-4b99-872a-82078bd89291" (UID: "64fd4c42-3b3a-4b99-872a-82078bd89291"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:15:00.573341 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:00.573315    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64fd4c42-3b3a-4b99-872a-82078bd89291-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "64fd4c42-3b3a-4b99-872a-82078bd89291" (UID: "64fd4c42-3b3a-4b99-872a-82078bd89291"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:15:00.573416 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:00.573358    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64fd4c42-3b3a-4b99-872a-82078bd89291-console-config" (OuterVolumeSpecName: "console-config") pod "64fd4c42-3b3a-4b99-872a-82078bd89291" (UID: "64fd4c42-3b3a-4b99-872a-82078bd89291"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:15:00.575105 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:00.575073    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64fd4c42-3b3a-4b99-872a-82078bd89291-kube-api-access-dgrh9" (OuterVolumeSpecName: "kube-api-access-dgrh9") pod "64fd4c42-3b3a-4b99-872a-82078bd89291" (UID: "64fd4c42-3b3a-4b99-872a-82078bd89291"). InnerVolumeSpecName "kube-api-access-dgrh9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:15:00.575227 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:00.575104    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64fd4c42-3b3a-4b99-872a-82078bd89291-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "64fd4c42-3b3a-4b99-872a-82078bd89291" (UID: "64fd4c42-3b3a-4b99-872a-82078bd89291"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:15:00.575347 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:00.575331    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64fd4c42-3b3a-4b99-872a-82078bd89291-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "64fd4c42-3b3a-4b99-872a-82078bd89291" (UID: "64fd4c42-3b3a-4b99-872a-82078bd89291"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:15:00.674354 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:00.674259    2548 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/64fd4c42-3b3a-4b99-872a-82078bd89291-service-ca\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:15:00.674354 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:00.674292    2548 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/64fd4c42-3b3a-4b99-872a-82078bd89291-oauth-serving-cert\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:15:00.674354 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:00.674303    2548 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/64fd4c42-3b3a-4b99-872a-82078bd89291-console-serving-cert\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:15:00.674354 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:00.674312    2548 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/64fd4c42-3b3a-4b99-872a-82078bd89291-console-oauth-config\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:15:00.674354 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:00.674321    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgrh9\" (UniqueName: \"kubernetes.io/projected/64fd4c42-3b3a-4b99-872a-82078bd89291-kube-api-access-dgrh9\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:15:00.674354 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:00.674330    2548 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/64fd4c42-3b3a-4b99-872a-82078bd89291-console-config\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:15:01.209753 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:01.209725    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-fbnfl"
Jul 23 23:15:01.350121 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:01.350090    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7787698577-zkc2m" event={"ID":"64fd4c42-3b3a-4b99-872a-82078bd89291","Type":"ContainerDied","Data":"6c8215eb64e35418773dcb031ab3b450a0ef360cc6e7b6639c0e64bd5f1a8e99"}
Jul 23 23:15:01.350287 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:01.350144    2548 scope.go:117] "RemoveContainer" containerID="1f6e1feea715cdded8389272d5f309f904cebdded11bf13ab699ce055e951c65"
Jul 23 23:15:01.350287 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:01.350100    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7787698577-zkc2m"
Jul 23 23:15:01.366910 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:01.366882    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7787698577-zkc2m"]
Jul 23 23:15:01.370676 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:01.370655    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7787698577-zkc2m"]
Jul 23 23:15:02.746428 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:02.746389    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64fd4c42-3b3a-4b99-872a-82078bd89291" path="/var/lib/kubelet/pods/64fd4c42-3b3a-4b99-872a-82078bd89291/volumes"
Jul 23 23:15:10.601153 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:10.601087    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-58cdcb6bc9-k4grx" podUID="49559220-6209-443a-bfaf-f5c61bdb9d5c" containerName="console" containerID="cri-o://eff84fa657a01d908f030a17c5c8c0970534224b77ce89b346ff5f9f71340d9a" gracePeriod=15
Jul 23 23:15:10.842569 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:10.842542    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58cdcb6bc9-k4grx"
Jul 23 23:15:10.952643 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:10.952610    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/49559220-6209-443a-bfaf-f5c61bdb9d5c-oauth-serving-cert\") pod \"49559220-6209-443a-bfaf-f5c61bdb9d5c\" (UID: \"49559220-6209-443a-bfaf-f5c61bdb9d5c\") "
Jul 23 23:15:10.952816 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:10.952661    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/49559220-6209-443a-bfaf-f5c61bdb9d5c-console-oauth-config\") pod \"49559220-6209-443a-bfaf-f5c61bdb9d5c\" (UID: \"49559220-6209-443a-bfaf-f5c61bdb9d5c\") "
Jul 23 23:15:10.952816 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:10.952691    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/49559220-6209-443a-bfaf-f5c61bdb9d5c-console-config\") pod \"49559220-6209-443a-bfaf-f5c61bdb9d5c\" (UID: \"49559220-6209-443a-bfaf-f5c61bdb9d5c\") "
Jul 23 23:15:10.952816 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:10.952720    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49559220-6209-443a-bfaf-f5c61bdb9d5c-trusted-ca-bundle\") pod \"49559220-6209-443a-bfaf-f5c61bdb9d5c\" (UID: \"49559220-6209-443a-bfaf-f5c61bdb9d5c\") "
Jul 23 23:15:10.952816 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:10.952756    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dh26v\" (UniqueName: \"kubernetes.io/projected/49559220-6209-443a-bfaf-f5c61bdb9d5c-kube-api-access-dh26v\") pod \"49559220-6209-443a-bfaf-f5c61bdb9d5c\" (UID: \"49559220-6209-443a-bfaf-f5c61bdb9d5c\") "
Jul 23 23:15:10.952816 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:10.952814    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/49559220-6209-443a-bfaf-f5c61bdb9d5c-service-ca\") pod \"49559220-6209-443a-bfaf-f5c61bdb9d5c\" (UID: \"49559220-6209-443a-bfaf-f5c61bdb9d5c\") "
Jul 23 23:15:10.953095 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:10.952842    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/49559220-6209-443a-bfaf-f5c61bdb9d5c-console-serving-cert\") pod \"49559220-6209-443a-bfaf-f5c61bdb9d5c\" (UID: \"49559220-6209-443a-bfaf-f5c61bdb9d5c\") "
Jul 23 23:15:10.953151 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:10.953134    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49559220-6209-443a-bfaf-f5c61bdb9d5c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "49559220-6209-443a-bfaf-f5c61bdb9d5c" (UID: "49559220-6209-443a-bfaf-f5c61bdb9d5c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:15:10.953306 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:10.953279    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49559220-6209-443a-bfaf-f5c61bdb9d5c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "49559220-6209-443a-bfaf-f5c61bdb9d5c" (UID: "49559220-6209-443a-bfaf-f5c61bdb9d5c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:15:10.953408 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:10.953356    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49559220-6209-443a-bfaf-f5c61bdb9d5c-console-config" (OuterVolumeSpecName: "console-config") pod "49559220-6209-443a-bfaf-f5c61bdb9d5c" (UID: "49559220-6209-443a-bfaf-f5c61bdb9d5c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:15:10.953408 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:10.953367    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49559220-6209-443a-bfaf-f5c61bdb9d5c-service-ca" (OuterVolumeSpecName: "service-ca") pod "49559220-6209-443a-bfaf-f5c61bdb9d5c" (UID: "49559220-6209-443a-bfaf-f5c61bdb9d5c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:15:10.955046 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:10.954939    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49559220-6209-443a-bfaf-f5c61bdb9d5c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "49559220-6209-443a-bfaf-f5c61bdb9d5c" (UID: "49559220-6209-443a-bfaf-f5c61bdb9d5c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:15:10.955046 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:10.954963    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49559220-6209-443a-bfaf-f5c61bdb9d5c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "49559220-6209-443a-bfaf-f5c61bdb9d5c" (UID: "49559220-6209-443a-bfaf-f5c61bdb9d5c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:15:10.955046 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:10.954995    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49559220-6209-443a-bfaf-f5c61bdb9d5c-kube-api-access-dh26v" (OuterVolumeSpecName: "kube-api-access-dh26v") pod "49559220-6209-443a-bfaf-f5c61bdb9d5c" (UID: "49559220-6209-443a-bfaf-f5c61bdb9d5c"). InnerVolumeSpecName "kube-api-access-dh26v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:15:11.053778 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:11.053730    2548 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/49559220-6209-443a-bfaf-f5c61bdb9d5c-service-ca\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:15:11.053778 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:11.053775    2548 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/49559220-6209-443a-bfaf-f5c61bdb9d5c-console-serving-cert\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:15:11.053778 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:11.053785    2548 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/49559220-6209-443a-bfaf-f5c61bdb9d5c-oauth-serving-cert\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:15:11.053778 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:11.053796    2548 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/49559220-6209-443a-bfaf-f5c61bdb9d5c-console-oauth-config\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:15:11.054086 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:11.053804    2548 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/49559220-6209-443a-bfaf-f5c61bdb9d5c-console-config\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:15:11.054086 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:11.053812    2548 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49559220-6209-443a-bfaf-f5c61bdb9d5c-trusted-ca-bundle\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:15:11.054086 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:11.053821    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dh26v\" (UniqueName: \"kubernetes.io/projected/49559220-6209-443a-bfaf-f5c61bdb9d5c-kube-api-access-dh26v\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:15:11.387199 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:11.387109    2548 generic.go:356] "Generic (PLEG): container finished" podID="49559220-6209-443a-bfaf-f5c61bdb9d5c" containerID="eff84fa657a01d908f030a17c5c8c0970534224b77ce89b346ff5f9f71340d9a" exitCode=0
Jul 23 23:15:11.387199 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:11.387159    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58cdcb6bc9-k4grx"
Jul 23 23:15:11.387414 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:11.387193    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58cdcb6bc9-k4grx" event={"ID":"49559220-6209-443a-bfaf-f5c61bdb9d5c","Type":"ContainerDied","Data":"eff84fa657a01d908f030a17c5c8c0970534224b77ce89b346ff5f9f71340d9a"}
Jul 23 23:15:11.387414 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:11.387244    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58cdcb6bc9-k4grx" event={"ID":"49559220-6209-443a-bfaf-f5c61bdb9d5c","Type":"ContainerDied","Data":"d2d6ad51e129db2be6b9ffeb17cd3a3190311f4b70cc04711d53903b3de70924"}
Jul 23 23:15:11.387414 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:11.387264    2548 scope.go:117] "RemoveContainer" containerID="eff84fa657a01d908f030a17c5c8c0970534224b77ce89b346ff5f9f71340d9a"
Jul 23 23:15:11.395991 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:11.395962    2548 scope.go:117] "RemoveContainer" containerID="eff84fa657a01d908f030a17c5c8c0970534224b77ce89b346ff5f9f71340d9a"
Jul 23 23:15:11.396261 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:15:11.396242    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eff84fa657a01d908f030a17c5c8c0970534224b77ce89b346ff5f9f71340d9a\": container with ID starting with eff84fa657a01d908f030a17c5c8c0970534224b77ce89b346ff5f9f71340d9a not found: ID does not exist" containerID="eff84fa657a01d908f030a17c5c8c0970534224b77ce89b346ff5f9f71340d9a"
Jul 23 23:15:11.396320 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:11.396269    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eff84fa657a01d908f030a17c5c8c0970534224b77ce89b346ff5f9f71340d9a"} err="failed to get container status \"eff84fa657a01d908f030a17c5c8c0970534224b77ce89b346ff5f9f71340d9a\": rpc error: code = NotFound desc = could not find container \"eff84fa657a01d908f030a17c5c8c0970534224b77ce89b346ff5f9f71340d9a\": container with ID starting with eff84fa657a01d908f030a17c5c8c0970534224b77ce89b346ff5f9f71340d9a not found: ID does not exist"
Jul 23 23:15:11.409363 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:11.409333    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58cdcb6bc9-k4grx"]
Jul 23 23:15:11.414024 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:11.413993    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-58cdcb6bc9-k4grx"]
Jul 23 23:15:12.746120 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:12.746089    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49559220-6209-443a-bfaf-f5c61bdb9d5c" path="/var/lib/kubelet/pods/49559220-6209-443a-bfaf-f5c61bdb9d5c/volumes"
Jul 23 23:15:52.436458 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:52.436423    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-859d9575c6-g5jmh"]
Jul 23 23:15:52.437073 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:52.436854    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64fd4c42-3b3a-4b99-872a-82078bd89291" containerName="console"
Jul 23 23:15:52.437073 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:52.436873    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="64fd4c42-3b3a-4b99-872a-82078bd89291" containerName="console"
Jul 23 23:15:52.437073 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:52.436893    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49559220-6209-443a-bfaf-f5c61bdb9d5c" containerName="console"
Jul 23 23:15:52.437073 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:52.436901    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="49559220-6209-443a-bfaf-f5c61bdb9d5c" containerName="console"
Jul 23 23:15:52.437073 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:52.437006    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="64fd4c42-3b3a-4b99-872a-82078bd89291" containerName="console"
Jul 23 23:15:52.437073 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:52.437021    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="49559220-6209-443a-bfaf-f5c61bdb9d5c" containerName="console"
Jul 23 23:15:52.439633 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:52.439612    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-859d9575c6-g5jmh"
Jul 23 23:15:52.449853 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:52.449831    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-859d9575c6-g5jmh"]
Jul 23 23:15:52.589482 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:52.589440    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15a6e91d-e650-4d7c-a059-10b2c3b3c636-oauth-serving-cert\") pod \"console-859d9575c6-g5jmh\" (UID: \"15a6e91d-e650-4d7c-a059-10b2c3b3c636\") " pod="openshift-console/console-859d9575c6-g5jmh"
Jul 23 23:15:52.589673 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:52.589521    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15a6e91d-e650-4d7c-a059-10b2c3b3c636-console-serving-cert\") pod \"console-859d9575c6-g5jmh\" (UID: \"15a6e91d-e650-4d7c-a059-10b2c3b3c636\") " pod="openshift-console/console-859d9575c6-g5jmh"
Jul 23 23:15:52.589673 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:52.589582    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15a6e91d-e650-4d7c-a059-10b2c3b3c636-trusted-ca-bundle\") pod \"console-859d9575c6-g5jmh\" (UID: \"15a6e91d-e650-4d7c-a059-10b2c3b3c636\") " pod="openshift-console/console-859d9575c6-g5jmh"
Jul 23 23:15:52.589673 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:52.589622    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15a6e91d-e650-4d7c-a059-10b2c3b3c636-console-oauth-config\") pod \"console-859d9575c6-g5jmh\" (UID: \"15a6e91d-e650-4d7c-a059-10b2c3b3c636\") " pod="openshift-console/console-859d9575c6-g5jmh"
Jul 23 23:15:52.589673 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:52.589640    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cw2f5\" (UniqueName: \"kubernetes.io/projected/15a6e91d-e650-4d7c-a059-10b2c3b3c636-kube-api-access-cw2f5\") pod \"console-859d9575c6-g5jmh\" (UID: \"15a6e91d-e650-4d7c-a059-10b2c3b3c636\") " pod="openshift-console/console-859d9575c6-g5jmh"
Jul 23 23:15:52.589825 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:52.589734    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15a6e91d-e650-4d7c-a059-10b2c3b3c636-service-ca\") pod \"console-859d9575c6-g5jmh\" (UID: \"15a6e91d-e650-4d7c-a059-10b2c3b3c636\") " pod="openshift-console/console-859d9575c6-g5jmh"
Jul 23 23:15:52.589825 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:52.589762    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15a6e91d-e650-4d7c-a059-10b2c3b3c636-console-config\") pod \"console-859d9575c6-g5jmh\" (UID: \"15a6e91d-e650-4d7c-a059-10b2c3b3c636\") " pod="openshift-console/console-859d9575c6-g5jmh"
Jul 23 23:15:52.690936 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:52.690853    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15a6e91d-e650-4d7c-a059-10b2c3b3c636-console-config\") pod \"console-859d9575c6-g5jmh\" (UID: \"15a6e91d-e650-4d7c-a059-10b2c3b3c636\") " pod="openshift-console/console-859d9575c6-g5jmh"
Jul 23 23:15:52.690936 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:52.690893    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15a6e91d-e650-4d7c-a059-10b2c3b3c636-oauth-serving-cert\") pod \"console-859d9575c6-g5jmh\" (UID: \"15a6e91d-e650-4d7c-a059-10b2c3b3c636\") " pod="openshift-console/console-859d9575c6-g5jmh"
Jul 23 23:15:52.691135 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:52.690947    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15a6e91d-e650-4d7c-a059-10b2c3b3c636-console-serving-cert\") pod \"console-859d9575c6-g5jmh\" (UID: \"15a6e91d-e650-4d7c-a059-10b2c3b3c636\") " pod="openshift-console/console-859d9575c6-g5jmh"
Jul 23 23:15:52.691135 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:52.691001    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15a6e91d-e650-4d7c-a059-10b2c3b3c636-trusted-ca-bundle\") pod \"console-859d9575c6-g5jmh\" (UID: \"15a6e91d-e650-4d7c-a059-10b2c3b3c636\") " pod="openshift-console/console-859d9575c6-g5jmh"
Jul 23 23:15:52.691135 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:52.691040    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15a6e91d-e650-4d7c-a059-10b2c3b3c636-console-oauth-config\") pod \"console-859d9575c6-g5jmh\" (UID: \"15a6e91d-e650-4d7c-a059-10b2c3b3c636\") " pod="openshift-console/console-859d9575c6-g5jmh"
Jul 23 23:15:52.691135 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:52.691067    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cw2f5\" (UniqueName: \"kubernetes.io/projected/15a6e91d-e650-4d7c-a059-10b2c3b3c636-kube-api-access-cw2f5\") pod \"console-859d9575c6-g5jmh\" (UID: \"15a6e91d-e650-4d7c-a059-10b2c3b3c636\") " pod="openshift-console/console-859d9575c6-g5jmh"
Jul 23 23:15:52.691135 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:52.691103    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15a6e91d-e650-4d7c-a059-10b2c3b3c636-service-ca\") pod \"console-859d9575c6-g5jmh\" (UID: \"15a6e91d-e650-4d7c-a059-10b2c3b3c636\") " pod="openshift-console/console-859d9575c6-g5jmh"
Jul 23 23:15:52.691732 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:52.691699    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15a6e91d-e650-4d7c-a059-10b2c3b3c636-console-config\") pod \"console-859d9575c6-g5jmh\" (UID: \"15a6e91d-e650-4d7c-a059-10b2c3b3c636\") " pod="openshift-console/console-859d9575c6-g5jmh"
Jul 23 23:15:52.691855 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:52.691774    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15a6e91d-e650-4d7c-a059-10b2c3b3c636-oauth-serving-cert\") pod \"console-859d9575c6-g5jmh\" (UID: \"15a6e91d-e650-4d7c-a059-10b2c3b3c636\") " pod="openshift-console/console-859d9575c6-g5jmh"
Jul 23 23:15:52.691855 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:52.691840    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15a6e91d-e650-4d7c-a059-10b2c3b3c636-service-ca\") pod \"console-859d9575c6-g5jmh\" (UID: \"15a6e91d-e650-4d7c-a059-10b2c3b3c636\") " pod="openshift-console/console-859d9575c6-g5jmh"
Jul 23 23:15:52.691968 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:52.691862    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15a6e91d-e650-4d7c-a059-10b2c3b3c636-trusted-ca-bundle\") pod \"console-859d9575c6-g5jmh\" (UID: \"15a6e91d-e650-4d7c-a059-10b2c3b3c636\") " pod="openshift-console/console-859d9575c6-g5jmh"
Jul 23 23:15:52.693496 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:52.693468    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15a6e91d-e650-4d7c-a059-10b2c3b3c636-console-serving-cert\") pod \"console-859d9575c6-g5jmh\" (UID: \"15a6e91d-e650-4d7c-a059-10b2c3b3c636\") " pod="openshift-console/console-859d9575c6-g5jmh"
Jul 23 23:15:52.693582 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:52.693509    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15a6e91d-e650-4d7c-a059-10b2c3b3c636-console-oauth-config\") pod \"console-859d9575c6-g5jmh\" (UID: \"15a6e91d-e650-4d7c-a059-10b2c3b3c636\") " pod="openshift-console/console-859d9575c6-g5jmh"
Jul 23 23:15:52.699021 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:52.699000    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cw2f5\" (UniqueName: \"kubernetes.io/projected/15a6e91d-e650-4d7c-a059-10b2c3b3c636-kube-api-access-cw2f5\") pod \"console-859d9575c6-g5jmh\" (UID: \"15a6e91d-e650-4d7c-a059-10b2c3b3c636\") " pod="openshift-console/console-859d9575c6-g5jmh"
Jul 23 23:15:52.752466 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:52.752202    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-859d9575c6-g5jmh"
Jul 23 23:15:52.889341 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:52.889240    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-859d9575c6-g5jmh"]
Jul 23 23:15:52.892117 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:15:52.892087    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15a6e91d_e650_4d7c_a059_10b2c3b3c636.slice/crio-d6bc2f28d2e630dfc86831a0d6312511d907c9bd892fce814951998d13ac7850 WatchSource:0}: Error finding container d6bc2f28d2e630dfc86831a0d6312511d907c9bd892fce814951998d13ac7850: Status 404 returned error can't find the container with id d6bc2f28d2e630dfc86831a0d6312511d907c9bd892fce814951998d13ac7850
Jul 23 23:15:53.535422 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:53.535385    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-859d9575c6-g5jmh" event={"ID":"15a6e91d-e650-4d7c-a059-10b2c3b3c636","Type":"ContainerStarted","Data":"161801e5b89725ada46e2384b39c0dc0d4edce6cfc2cd332548b391c190b16fb"}
Jul 23 23:15:53.535806 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:53.535435    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-859d9575c6-g5jmh" event={"ID":"15a6e91d-e650-4d7c-a059-10b2c3b3c636","Type":"ContainerStarted","Data":"d6bc2f28d2e630dfc86831a0d6312511d907c9bd892fce814951998d13ac7850"}
Jul 23 23:15:53.552951 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:15:53.552901    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-859d9575c6-g5jmh" podStartSLOduration=1.552884385 podStartE2EDuration="1.552884385s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:15:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:15:53.551793249 +0000 UTC m=+155.295522298" watchObservedRunningTime="2026-07-23 23:15:53.552884385 +0000 UTC m=+155.296613433"
Jul 23 23:16:02.753280 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:02.753246    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-859d9575c6-g5jmh"
Jul 23 23:16:02.753738 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:02.753297    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-859d9575c6-g5jmh"
Jul 23 23:16:02.758110 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:02.758089    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-859d9575c6-g5jmh"
Jul 23 23:16:03.575871 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:03.575841    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-859d9575c6-g5jmh"
Jul 23 23:16:03.615948 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:03.615911    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55c5469fbd-pj4cs"]
Jul 23 23:16:28.635936 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:28.635892    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-55c5469fbd-pj4cs" podUID="68554255-21fb-45b5-8f13-d9d91307e565" containerName="console" containerID="cri-o://1c6f0ccd9fe9f5648803081d36b83a1df0cc5ee4580c76c2f2ca7b0a4c6d00c2" gracePeriod=15
Jul 23 23:16:28.874239 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:28.874213    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55c5469fbd-pj4cs"
Jul 23 23:16:28.982723 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:28.982683    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/68554255-21fb-45b5-8f13-d9d91307e565-console-config\") pod \"68554255-21fb-45b5-8f13-d9d91307e565\" (UID: \"68554255-21fb-45b5-8f13-d9d91307e565\") "
Jul 23 23:16:28.982907 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:28.982748    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/68554255-21fb-45b5-8f13-d9d91307e565-service-ca\") pod \"68554255-21fb-45b5-8f13-d9d91307e565\" (UID: \"68554255-21fb-45b5-8f13-d9d91307e565\") "
Jul 23 23:16:28.982907 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:28.982765    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/68554255-21fb-45b5-8f13-d9d91307e565-console-oauth-config\") pod \"68554255-21fb-45b5-8f13-d9d91307e565\" (UID: \"68554255-21fb-45b5-8f13-d9d91307e565\") "
Jul 23 23:16:28.982907 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:28.982795    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/68554255-21fb-45b5-8f13-d9d91307e565-oauth-serving-cert\") pod \"68554255-21fb-45b5-8f13-d9d91307e565\" (UID: \"68554255-21fb-45b5-8f13-d9d91307e565\") "
Jul 23 23:16:28.983095 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:28.983004    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lt568\" (UniqueName: \"kubernetes.io/projected/68554255-21fb-45b5-8f13-d9d91307e565-kube-api-access-lt568\") pod \"68554255-21fb-45b5-8f13-d9d91307e565\" (UID: \"68554255-21fb-45b5-8f13-d9d91307e565\") "
Jul 23 23:16:28.983095 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:28.983059    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68554255-21fb-45b5-8f13-d9d91307e565-trusted-ca-bundle\") pod \"68554255-21fb-45b5-8f13-d9d91307e565\" (UID: \"68554255-21fb-45b5-8f13-d9d91307e565\") "
Jul 23 23:16:28.983095 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:28.983091    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/68554255-21fb-45b5-8f13-d9d91307e565-console-serving-cert\") pod \"68554255-21fb-45b5-8f13-d9d91307e565\" (UID: \"68554255-21fb-45b5-8f13-d9d91307e565\") "
Jul 23 23:16:28.983236 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:28.983178    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68554255-21fb-45b5-8f13-d9d91307e565-console-config" (OuterVolumeSpecName: "console-config") pod "68554255-21fb-45b5-8f13-d9d91307e565" (UID: "68554255-21fb-45b5-8f13-d9d91307e565"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:16:28.983356 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:28.983324    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68554255-21fb-45b5-8f13-d9d91307e565-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "68554255-21fb-45b5-8f13-d9d91307e565" (UID: "68554255-21fb-45b5-8f13-d9d91307e565"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:16:28.983489 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:28.983363    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68554255-21fb-45b5-8f13-d9d91307e565-service-ca" (OuterVolumeSpecName: "service-ca") pod "68554255-21fb-45b5-8f13-d9d91307e565" (UID: "68554255-21fb-45b5-8f13-d9d91307e565"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:16:28.983552 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:28.983474    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68554255-21fb-45b5-8f13-d9d91307e565-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "68554255-21fb-45b5-8f13-d9d91307e565" (UID: "68554255-21fb-45b5-8f13-d9d91307e565"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:16:28.983552 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:28.983507    2548 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/68554255-21fb-45b5-8f13-d9d91307e565-console-config\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:16:28.983552 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:28.983526    2548 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/68554255-21fb-45b5-8f13-d9d91307e565-service-ca\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:16:28.983552 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:28.983536    2548 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/68554255-21fb-45b5-8f13-d9d91307e565-oauth-serving-cert\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:16:28.984961 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:28.984939    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68554255-21fb-45b5-8f13-d9d91307e565-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "68554255-21fb-45b5-8f13-d9d91307e565" (UID: "68554255-21fb-45b5-8f13-d9d91307e565"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:16:28.985145 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:28.985123    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68554255-21fb-45b5-8f13-d9d91307e565-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "68554255-21fb-45b5-8f13-d9d91307e565" (UID: "68554255-21fb-45b5-8f13-d9d91307e565"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:16:28.985194 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:28.985123    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68554255-21fb-45b5-8f13-d9d91307e565-kube-api-access-lt568" (OuterVolumeSpecName: "kube-api-access-lt568") pod "68554255-21fb-45b5-8f13-d9d91307e565" (UID: "68554255-21fb-45b5-8f13-d9d91307e565"). InnerVolumeSpecName "kube-api-access-lt568". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:16:29.084130 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:29.084091    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lt568\" (UniqueName: \"kubernetes.io/projected/68554255-21fb-45b5-8f13-d9d91307e565-kube-api-access-lt568\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:16:29.084130 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:29.084122    2548 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68554255-21fb-45b5-8f13-d9d91307e565-trusted-ca-bundle\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:16:29.084130 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:29.084131    2548 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/68554255-21fb-45b5-8f13-d9d91307e565-console-serving-cert\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:16:29.084130 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:29.084140    2548 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/68554255-21fb-45b5-8f13-d9d91307e565-console-oauth-config\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:16:29.660186 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:29.660148    2548 generic.go:356] "Generic (PLEG): container finished" podID="68554255-21fb-45b5-8f13-d9d91307e565" containerID="1c6f0ccd9fe9f5648803081d36b83a1df0cc5ee4580c76c2f2ca7b0a4c6d00c2" exitCode=0
Jul 23 23:16:29.660594 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:29.660199    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55c5469fbd-pj4cs"
Jul 23 23:16:29.660594 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:29.660242    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55c5469fbd-pj4cs" event={"ID":"68554255-21fb-45b5-8f13-d9d91307e565","Type":"ContainerDied","Data":"1c6f0ccd9fe9f5648803081d36b83a1df0cc5ee4580c76c2f2ca7b0a4c6d00c2"}
Jul 23 23:16:29.660594 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:29.660288    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55c5469fbd-pj4cs" event={"ID":"68554255-21fb-45b5-8f13-d9d91307e565","Type":"ContainerDied","Data":"7e967020c128c3384c1d819448b597ac3825d94e1f423062782921de3aca794c"}
Jul 23 23:16:29.660594 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:29.660307    2548 scope.go:117] "RemoveContainer" containerID="1c6f0ccd9fe9f5648803081d36b83a1df0cc5ee4580c76c2f2ca7b0a4c6d00c2"
Jul 23 23:16:29.668834 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:29.668818    2548 scope.go:117] "RemoveContainer" containerID="1c6f0ccd9fe9f5648803081d36b83a1df0cc5ee4580c76c2f2ca7b0a4c6d00c2"
Jul 23 23:16:29.669094 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:16:29.669070    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c6f0ccd9fe9f5648803081d36b83a1df0cc5ee4580c76c2f2ca7b0a4c6d00c2\": container with ID starting with 1c6f0ccd9fe9f5648803081d36b83a1df0cc5ee4580c76c2f2ca7b0a4c6d00c2 not found: ID does not exist" containerID="1c6f0ccd9fe9f5648803081d36b83a1df0cc5ee4580c76c2f2ca7b0a4c6d00c2"
Jul 23 23:16:29.669183 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:29.669099    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c6f0ccd9fe9f5648803081d36b83a1df0cc5ee4580c76c2f2ca7b0a4c6d00c2"} err="failed to get container status \"1c6f0ccd9fe9f5648803081d36b83a1df0cc5ee4580c76c2f2ca7b0a4c6d00c2\": rpc error: code = NotFound desc = could not find container \"1c6f0ccd9fe9f5648803081d36b83a1df0cc5ee4580c76c2f2ca7b0a4c6d00c2\": container with ID starting with 1c6f0ccd9fe9f5648803081d36b83a1df0cc5ee4580c76c2f2ca7b0a4c6d00c2 not found: ID does not exist"
Jul 23 23:16:29.680258 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:29.680231    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55c5469fbd-pj4cs"]
Jul 23 23:16:29.684875 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:29.684853    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-55c5469fbd-pj4cs"]
Jul 23 23:16:30.746194 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:30.746160    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68554255-21fb-45b5-8f13-d9d91307e565" path="/var/lib/kubelet/pods/68554255-21fb-45b5-8f13-d9d91307e565/volumes"
Jul 23 23:16:33.202589 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:33.202557    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jthtn"]
Jul 23 23:16:33.202989 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:33.202862    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68554255-21fb-45b5-8f13-d9d91307e565" containerName="console"
Jul 23 23:16:33.202989 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:33.202875    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="68554255-21fb-45b5-8f13-d9d91307e565" containerName="console"
Jul 23 23:16:33.202989 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:33.202915    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="68554255-21fb-45b5-8f13-d9d91307e565" containerName="console"
Jul 23 23:16:33.207322 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:33.207305    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jthtn"
Jul 23 23:16:33.210075 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:33.210054    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 23 23:16:33.211110 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:33.211092    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 23 23:16:33.211110 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:33.211115    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ph65g\""
Jul 23 23:16:33.215053 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:33.215031    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jthtn"]
Jul 23 23:16:33.322242 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:33.322201    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c36db154-a7b3-4590-8abc-8af193e6cf8b-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jthtn\" (UID: \"c36db154-a7b3-4590-8abc-8af193e6cf8b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jthtn"
Jul 23 23:16:33.322431 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:33.322257    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c36db154-a7b3-4590-8abc-8af193e6cf8b-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jthtn\" (UID: \"c36db154-a7b3-4590-8abc-8af193e6cf8b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jthtn"
Jul 23 23:16:33.322431 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:33.322322    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7gh5\" (UniqueName: \"kubernetes.io/projected/c36db154-a7b3-4590-8abc-8af193e6cf8b-kube-api-access-x7gh5\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jthtn\" (UID: \"c36db154-a7b3-4590-8abc-8af193e6cf8b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jthtn"
Jul 23 23:16:33.422987 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:33.422946    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c36db154-a7b3-4590-8abc-8af193e6cf8b-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jthtn\" (UID: \"c36db154-a7b3-4590-8abc-8af193e6cf8b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jthtn"
Jul 23 23:16:33.423179 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:33.423016    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7gh5\" (UniqueName: \"kubernetes.io/projected/c36db154-a7b3-4590-8abc-8af193e6cf8b-kube-api-access-x7gh5\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jthtn\" (UID: \"c36db154-a7b3-4590-8abc-8af193e6cf8b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jthtn"
Jul 23 23:16:33.423179 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:33.423073    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c36db154-a7b3-4590-8abc-8af193e6cf8b-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jthtn\" (UID: \"c36db154-a7b3-4590-8abc-8af193e6cf8b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jthtn"
Jul 23 23:16:33.423358 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:33.423320    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c36db154-a7b3-4590-8abc-8af193e6cf8b-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jthtn\" (UID: \"c36db154-a7b3-4590-8abc-8af193e6cf8b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jthtn"
Jul 23 23:16:33.423422 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:33.423388    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c36db154-a7b3-4590-8abc-8af193e6cf8b-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jthtn\" (UID: \"c36db154-a7b3-4590-8abc-8af193e6cf8b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jthtn"
Jul 23 23:16:33.430812 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:33.430782    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7gh5\" (UniqueName: \"kubernetes.io/projected/c36db154-a7b3-4590-8abc-8af193e6cf8b-kube-api-access-x7gh5\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jthtn\" (UID: \"c36db154-a7b3-4590-8abc-8af193e6cf8b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jthtn"
Jul 23 23:16:33.517206 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:33.517131    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jthtn"
Jul 23 23:16:33.644100 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:33.644042    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jthtn"]
Jul 23 23:16:33.646465 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:16:33.646439    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc36db154_a7b3_4590_8abc_8af193e6cf8b.slice/crio-1b4597c4d3bf961f3a884560a3c113f2bfc97c297d69efc502d5ab0fb180ef6e WatchSource:0}: Error finding container 1b4597c4d3bf961f3a884560a3c113f2bfc97c297d69efc502d5ab0fb180ef6e: Status 404 returned error can't find the container with id 1b4597c4d3bf961f3a884560a3c113f2bfc97c297d69efc502d5ab0fb180ef6e
Jul 23 23:16:33.676251 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:33.676225    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jthtn" event={"ID":"c36db154-a7b3-4590-8abc-8af193e6cf8b","Type":"ContainerStarted","Data":"1b4597c4d3bf961f3a884560a3c113f2bfc97c297d69efc502d5ab0fb180ef6e"}
Jul 23 23:16:39.728568 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:39.724443    2548 generic.go:356] "Generic (PLEG): container finished" podID="c36db154-a7b3-4590-8abc-8af193e6cf8b" containerID="b62638f92251f0c2f3848a7ac4f357496e28a153b3209e92e6679bb2893933d9" exitCode=0
Jul 23 23:16:39.728568 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:39.724493    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jthtn" event={"ID":"c36db154-a7b3-4590-8abc-8af193e6cf8b","Type":"ContainerDied","Data":"b62638f92251f0c2f3848a7ac4f357496e28a153b3209e92e6679bb2893933d9"}
Jul 23 23:16:42.737507 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:42.737469    2548 generic.go:356] "Generic (PLEG): container finished" podID="c36db154-a7b3-4590-8abc-8af193e6cf8b" containerID="27123710b1db823ec4101981e4e29b8cd691f5e0f85d232a04f9832f40df5e3a" exitCode=0
Jul 23 23:16:42.737886 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:42.737530    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jthtn" event={"ID":"c36db154-a7b3-4590-8abc-8af193e6cf8b","Type":"ContainerDied","Data":"27123710b1db823ec4101981e4e29b8cd691f5e0f85d232a04f9832f40df5e3a"}
Jul 23 23:16:49.770734 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:49.770635    2548 generic.go:356] "Generic (PLEG): container finished" podID="c36db154-a7b3-4590-8abc-8af193e6cf8b" containerID="9a85e2d65e374a6046020f950049382560ac6772c9af46547a60c161bce51a8a" exitCode=0
Jul 23 23:16:49.771096 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:49.770722    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jthtn" event={"ID":"c36db154-a7b3-4590-8abc-8af193e6cf8b","Type":"ContainerDied","Data":"9a85e2d65e374a6046020f950049382560ac6772c9af46547a60c161bce51a8a"}
Jul 23 23:16:50.899248 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:50.899224    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jthtn"
Jul 23 23:16:50.969043 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:50.969010    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c36db154-a7b3-4590-8abc-8af193e6cf8b-bundle\") pod \"c36db154-a7b3-4590-8abc-8af193e6cf8b\" (UID: \"c36db154-a7b3-4590-8abc-8af193e6cf8b\") "
Jul 23 23:16:50.969228 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:50.969063    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7gh5\" (UniqueName: \"kubernetes.io/projected/c36db154-a7b3-4590-8abc-8af193e6cf8b-kube-api-access-x7gh5\") pod \"c36db154-a7b3-4590-8abc-8af193e6cf8b\" (UID: \"c36db154-a7b3-4590-8abc-8af193e6cf8b\") "
Jul 23 23:16:50.969228 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:50.969083    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c36db154-a7b3-4590-8abc-8af193e6cf8b-util\") pod \"c36db154-a7b3-4590-8abc-8af193e6cf8b\" (UID: \"c36db154-a7b3-4590-8abc-8af193e6cf8b\") "
Jul 23 23:16:50.969723 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:50.969696    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c36db154-a7b3-4590-8abc-8af193e6cf8b-bundle" (OuterVolumeSpecName: "bundle") pod "c36db154-a7b3-4590-8abc-8af193e6cf8b" (UID: "c36db154-a7b3-4590-8abc-8af193e6cf8b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:16:50.971274 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:50.971249    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c36db154-a7b3-4590-8abc-8af193e6cf8b-kube-api-access-x7gh5" (OuterVolumeSpecName: "kube-api-access-x7gh5") pod "c36db154-a7b3-4590-8abc-8af193e6cf8b" (UID: "c36db154-a7b3-4590-8abc-8af193e6cf8b"). InnerVolumeSpecName "kube-api-access-x7gh5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:16:50.973546 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:50.973517    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c36db154-a7b3-4590-8abc-8af193e6cf8b-util" (OuterVolumeSpecName: "util") pod "c36db154-a7b3-4590-8abc-8af193e6cf8b" (UID: "c36db154-a7b3-4590-8abc-8af193e6cf8b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:16:51.070218 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:51.070134    2548 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c36db154-a7b3-4590-8abc-8af193e6cf8b-bundle\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:16:51.070218 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:51.070167    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7gh5\" (UniqueName: \"kubernetes.io/projected/c36db154-a7b3-4590-8abc-8af193e6cf8b-kube-api-access-x7gh5\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:16:51.070218 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:51.070177    2548 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c36db154-a7b3-4590-8abc-8af193e6cf8b-util\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:16:51.778433 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:51.778393    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jthtn" event={"ID":"c36db154-a7b3-4590-8abc-8af193e6cf8b","Type":"ContainerDied","Data":"1b4597c4d3bf961f3a884560a3c113f2bfc97c297d69efc502d5ab0fb180ef6e"}
Jul 23 23:16:51.778433 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:51.778425    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jthtn"
Jul 23 23:16:51.778641 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:51.778442    2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1b4597c4d3bf961f3a884560a3c113f2bfc97c297d69efc502d5ab0fb180ef6e"
Jul 23 23:16:56.125565 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:56.125526    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-85g97"]
Jul 23 23:16:56.126078 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:56.125902    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c36db154-a7b3-4590-8abc-8af193e6cf8b" containerName="util"
Jul 23 23:16:56.126078 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:56.125918    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="c36db154-a7b3-4590-8abc-8af193e6cf8b" containerName="util"
Jul 23 23:16:56.126078 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:56.125937    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c36db154-a7b3-4590-8abc-8af193e6cf8b" containerName="extract"
Jul 23 23:16:56.126078 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:56.125946    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="c36db154-a7b3-4590-8abc-8af193e6cf8b" containerName="extract"
Jul 23 23:16:56.126078 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:56.125959    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c36db154-a7b3-4590-8abc-8af193e6cf8b" containerName="pull"
Jul 23 23:16:56.126078 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:56.125967    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="c36db154-a7b3-4590-8abc-8af193e6cf8b" containerName="pull"
Jul 23 23:16:56.126078 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:56.126058    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="c36db154-a7b3-4590-8abc-8af193e6cf8b" containerName="extract"
Jul 23 23:16:56.152864 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:56.152827    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-85g97"]
Jul 23 23:16:56.153040 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:56.152939    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-85g97"
Jul 23 23:16:56.155809 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:56.155773    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 23 23:16:56.155939 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:56.155824    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 23 23:16:56.155939 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:56.155824    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-tspkd\""
Jul 23 23:16:56.203888 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:56.203847    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5td7q\" (UniqueName: \"kubernetes.io/projected/701b6502-5fbd-4236-ac08-a3a74cb8387b-kube-api-access-5td7q\") pod \"cert-manager-operator-controller-manager-55c444bf86-85g97\" (UID: \"701b6502-5fbd-4236-ac08-a3a74cb8387b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-85g97"
Jul 23 23:16:56.204065 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:56.203901    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/701b6502-5fbd-4236-ac08-a3a74cb8387b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-85g97\" (UID: \"701b6502-5fbd-4236-ac08-a3a74cb8387b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-85g97"
Jul 23 23:16:56.305159 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:56.305123    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5td7q\" (UniqueName: \"kubernetes.io/projected/701b6502-5fbd-4236-ac08-a3a74cb8387b-kube-api-access-5td7q\") pod \"cert-manager-operator-controller-manager-55c444bf86-85g97\" (UID: \"701b6502-5fbd-4236-ac08-a3a74cb8387b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-85g97"
Jul 23 23:16:56.305159 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:56.305167    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/701b6502-5fbd-4236-ac08-a3a74cb8387b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-85g97\" (UID: \"701b6502-5fbd-4236-ac08-a3a74cb8387b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-85g97"
Jul 23 23:16:56.305477 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:56.305463    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/701b6502-5fbd-4236-ac08-a3a74cb8387b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-85g97\" (UID: \"701b6502-5fbd-4236-ac08-a3a74cb8387b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-85g97"
Jul 23 23:16:56.313114 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:56.313091    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5td7q\" (UniqueName: \"kubernetes.io/projected/701b6502-5fbd-4236-ac08-a3a74cb8387b-kube-api-access-5td7q\") pod \"cert-manager-operator-controller-manager-55c444bf86-85g97\" (UID: \"701b6502-5fbd-4236-ac08-a3a74cb8387b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-85g97"
Jul 23 23:16:56.462411 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:56.462378    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-85g97"
Jul 23 23:16:56.620142 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:56.620052    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-85g97"]
Jul 23 23:16:56.623915 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:16:56.623887    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod701b6502_5fbd_4236_ac08_a3a74cb8387b.slice/crio-4a0dd6bead545cfb9e93c0d5e79dfafb18b35a6a73ba29a5897b80c36ef7d620 WatchSource:0}: Error finding container 4a0dd6bead545cfb9e93c0d5e79dfafb18b35a6a73ba29a5897b80c36ef7d620: Status 404 returned error can't find the container with id 4a0dd6bead545cfb9e93c0d5e79dfafb18b35a6a73ba29a5897b80c36ef7d620
Jul 23 23:16:56.796187 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:56.796099    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-85g97" event={"ID":"701b6502-5fbd-4236-ac08-a3a74cb8387b","Type":"ContainerStarted","Data":"4a0dd6bead545cfb9e93c0d5e79dfafb18b35a6a73ba29a5897b80c36ef7d620"}
Jul 23 23:16:59.810899 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:59.810865    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-85g97" event={"ID":"701b6502-5fbd-4236-ac08-a3a74cb8387b","Type":"ContainerStarted","Data":"6939ee95ac885de239bec9e22ec1941cb76de55ff09345b8d093829b100164a4"}
Jul 23 23:16:59.834470 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:16:59.834417    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-85g97" podStartSLOduration=1.468610958 podStartE2EDuration="3.834401629s" totalImagesPullingTime="2.365790671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:16:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:16:59.83219379 +0000 UTC m=+221.575922840" watchObservedRunningTime="2026-07-23 23:16:59.834401629 +0000 UTC m=+221.578130678"
Jul 23 23:17:00.533539 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:00.533502    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f886tm"]
Jul 23 23:17:00.555199 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:00.555168    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f886tm"]
Jul 23 23:17:00.555343 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:00.555291    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f886tm"
Jul 23 23:17:00.558007 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:00.557969    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 23 23:17:00.558007 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:00.557998    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ph65g\""
Jul 23 23:17:00.558177 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:00.557969    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 23 23:17:00.640835 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:00.640801    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a3c6921-39f1-4b4c-9beb-463618ed6521-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f886tm\" (UID: \"4a3c6921-39f1-4b4c-9beb-463618ed6521\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f886tm"
Jul 23 23:17:00.641045 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:00.640850    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92k5k\" (UniqueName: \"kubernetes.io/projected/4a3c6921-39f1-4b4c-9beb-463618ed6521-kube-api-access-92k5k\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f886tm\" (UID: \"4a3c6921-39f1-4b4c-9beb-463618ed6521\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f886tm"
Jul 23 23:17:00.641045 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:00.640928    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a3c6921-39f1-4b4c-9beb-463618ed6521-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f886tm\" (UID: \"4a3c6921-39f1-4b4c-9beb-463618ed6521\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f886tm"
Jul 23 23:17:00.741764 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:00.741728    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a3c6921-39f1-4b4c-9beb-463618ed6521-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f886tm\" (UID: \"4a3c6921-39f1-4b4c-9beb-463618ed6521\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f886tm"
Jul 23 23:17:00.741764 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:00.741766    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a3c6921-39f1-4b4c-9beb-463618ed6521-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f886tm\" (UID: \"4a3c6921-39f1-4b4c-9beb-463618ed6521\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f886tm"
Jul 23 23:17:00.741957 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:00.741800    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92k5k\" (UniqueName: \"kubernetes.io/projected/4a3c6921-39f1-4b4c-9beb-463618ed6521-kube-api-access-92k5k\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f886tm\" (UID: \"4a3c6921-39f1-4b4c-9beb-463618ed6521\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f886tm"
Jul 23 23:17:00.742267 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:00.742244    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a3c6921-39f1-4b4c-9beb-463618ed6521-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f886tm\" (UID: \"4a3c6921-39f1-4b4c-9beb-463618ed6521\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f886tm"
Jul 23 23:17:00.742313 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:00.742252    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a3c6921-39f1-4b4c-9beb-463618ed6521-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f886tm\" (UID: \"4a3c6921-39f1-4b4c-9beb-463618ed6521\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f886tm"
Jul 23 23:17:00.753471 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:00.753441    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92k5k\" (UniqueName: \"kubernetes.io/projected/4a3c6921-39f1-4b4c-9beb-463618ed6521-kube-api-access-92k5k\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f886tm\" (UID: \"4a3c6921-39f1-4b4c-9beb-463618ed6521\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f886tm"
Jul 23 23:17:00.864947 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:00.864860    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f886tm"
Jul 23 23:17:01.017503 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:01.017472    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f886tm"]
Jul 23 23:17:01.020737 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:17:01.020707    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a3c6921_39f1_4b4c_9beb_463618ed6521.slice/crio-c87b8d67bd02745dc8966a7d4eee927d66ab619bfa550a084a4056e0fd980ee7 WatchSource:0}: Error finding container c87b8d67bd02745dc8966a7d4eee927d66ab619bfa550a084a4056e0fd980ee7: Status 404 returned error can't find the container with id c87b8d67bd02745dc8966a7d4eee927d66ab619bfa550a084a4056e0fd980ee7
Jul 23 23:17:01.820048 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:01.820007    2548 generic.go:356] "Generic (PLEG): container finished" podID="4a3c6921-39f1-4b4c-9beb-463618ed6521" containerID="c3c40fbe1f999fa51b75237d7a4d938f4e4a4bd247a8c189605191c8b0626688" exitCode=0
Jul 23 23:17:01.820207 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:01.820100    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f886tm" event={"ID":"4a3c6921-39f1-4b4c-9beb-463618ed6521","Type":"ContainerDied","Data":"c3c40fbe1f999fa51b75237d7a4d938f4e4a4bd247a8c189605191c8b0626688"}
Jul 23 23:17:01.820207 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:01.820147    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f886tm" event={"ID":"4a3c6921-39f1-4b4c-9beb-463618ed6521","Type":"ContainerStarted","Data":"c87b8d67bd02745dc8966a7d4eee927d66ab619bfa550a084a4056e0fd980ee7"}
Jul 23 23:17:02.598943 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:02.598904    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-sntqf"]
Jul 23 23:17:02.613646 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:02.613618    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-sntqf"]
Jul 23 23:17:02.613809 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:02.613730    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-sntqf"
Jul 23 23:17:02.616158 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:02.616131    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 23 23:17:02.617427 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:02.617398    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 23 23:17:02.617561 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:02.617432    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-q8d7r\""
Jul 23 23:17:02.659197 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:02.659164    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czrd5\" (UniqueName: \"kubernetes.io/projected/148aaa98-c920-4b7a-a2d3-2c5ba6f67e7b-kube-api-access-czrd5\") pod \"cert-manager-cainjector-559cdd8c87-sntqf\" (UID: \"148aaa98-c920-4b7a-a2d3-2c5ba6f67e7b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sntqf"
Jul 23 23:17:02.659374 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:02.659208    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/148aaa98-c920-4b7a-a2d3-2c5ba6f67e7b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-sntqf\" (UID: \"148aaa98-c920-4b7a-a2d3-2c5ba6f67e7b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sntqf"
Jul 23 23:17:02.759933 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:02.759899    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czrd5\" (UniqueName: \"kubernetes.io/projected/148aaa98-c920-4b7a-a2d3-2c5ba6f67e7b-kube-api-access-czrd5\") pod \"cert-manager-cainjector-559cdd8c87-sntqf\" (UID: \"148aaa98-c920-4b7a-a2d3-2c5ba6f67e7b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sntqf"
Jul 23 23:17:02.760140 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:02.759951    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/148aaa98-c920-4b7a-a2d3-2c5ba6f67e7b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-sntqf\" (UID: \"148aaa98-c920-4b7a-a2d3-2c5ba6f67e7b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sntqf"
Jul 23 23:17:02.771037 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:02.770987    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/148aaa98-c920-4b7a-a2d3-2c5ba6f67e7b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-sntqf\" (UID: \"148aaa98-c920-4b7a-a2d3-2c5ba6f67e7b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sntqf"
Jul 23 23:17:02.771348 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:02.771316    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czrd5\" (UniqueName: \"kubernetes.io/projected/148aaa98-c920-4b7a-a2d3-2c5ba6f67e7b-kube-api-access-czrd5\") pod \"cert-manager-cainjector-559cdd8c87-sntqf\" (UID: \"148aaa98-c920-4b7a-a2d3-2c5ba6f67e7b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sntqf"
Jul 23 23:17:02.933122 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:02.933085    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-sntqf"
Jul 23 23:17:03.082622 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:03.082584    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-sntqf"]
Jul 23 23:17:03.675206 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:17:03.675172    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod148aaa98_c920_4b7a_a2d3_2c5ba6f67e7b.slice/crio-012176346c901d6d67ba0a9cfa9aa9f23e1c05ec098cf94885cd530f247b68a8 WatchSource:0}: Error finding container 012176346c901d6d67ba0a9cfa9aa9f23e1c05ec098cf94885cd530f247b68a8: Status 404 returned error can't find the container with id 012176346c901d6d67ba0a9cfa9aa9f23e1c05ec098cf94885cd530f247b68a8
Jul 23 23:17:03.830065 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:03.830025    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-sntqf" event={"ID":"148aaa98-c920-4b7a-a2d3-2c5ba6f67e7b","Type":"ContainerStarted","Data":"012176346c901d6d67ba0a9cfa9aa9f23e1c05ec098cf94885cd530f247b68a8"}
Jul 23 23:17:03.831908 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:03.831881    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f886tm" event={"ID":"4a3c6921-39f1-4b4c-9beb-463618ed6521","Type":"ContainerStarted","Data":"7c481f0e288632481196a64253563ec8cd1c7129bc2406663462ff10fef90f00"}
Jul 23 23:17:04.837330 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:04.837299    2548 generic.go:356] "Generic (PLEG): container finished" podID="4a3c6921-39f1-4b4c-9beb-463618ed6521" containerID="7c481f0e288632481196a64253563ec8cd1c7129bc2406663462ff10fef90f00" exitCode=0
Jul 23 23:17:04.837769 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:04.837333    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f886tm" event={"ID":"4a3c6921-39f1-4b4c-9beb-463618ed6521","Type":"ContainerDied","Data":"7c481f0e288632481196a64253563ec8cd1c7129bc2406663462ff10fef90f00"}
Jul 23 23:17:05.843380 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:05.843345    2548 generic.go:356] "Generic (PLEG): container finished" podID="4a3c6921-39f1-4b4c-9beb-463618ed6521" containerID="d2563ddd6f2b80079dc4cf3b208472b95e9443e6fc997276716d4876464992a3" exitCode=0
Jul 23 23:17:05.843757 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:05.843413    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f886tm" event={"ID":"4a3c6921-39f1-4b4c-9beb-463618ed6521","Type":"ContainerDied","Data":"d2563ddd6f2b80079dc4cf3b208472b95e9443e6fc997276716d4876464992a3"}
Jul 23 23:17:06.546348 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:06.546311    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-bd8vc"]
Jul 23 23:17:06.550067 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:06.550047    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-bd8vc"
Jul 23 23:17:06.552513 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:06.552495    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-qnwx8\""
Jul 23 23:17:06.557748 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:06.557725    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-bd8vc"]
Jul 23 23:17:06.593917 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:06.593896    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0543fa0d-39c5-4a75-a350-a9ac59f955ef-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-bd8vc\" (UID: \"0543fa0d-39c5-4a75-a350-a9ac59f955ef\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bd8vc"
Jul 23 23:17:06.594053 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:06.593944    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j7fm\" (UniqueName: \"kubernetes.io/projected/0543fa0d-39c5-4a75-a350-a9ac59f955ef-kube-api-access-9j7fm\") pod \"cert-manager-webhook-6486ffd997-bd8vc\" (UID: \"0543fa0d-39c5-4a75-a350-a9ac59f955ef\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bd8vc"
Jul 23 23:17:06.694758 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:06.694726    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0543fa0d-39c5-4a75-a350-a9ac59f955ef-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-bd8vc\" (UID: \"0543fa0d-39c5-4a75-a350-a9ac59f955ef\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bd8vc"
Jul 23 23:17:06.694888 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:06.694787    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j7fm\" (UniqueName: \"kubernetes.io/projected/0543fa0d-39c5-4a75-a350-a9ac59f955ef-kube-api-access-9j7fm\") pod \"cert-manager-webhook-6486ffd997-bd8vc\" (UID: \"0543fa0d-39c5-4a75-a350-a9ac59f955ef\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bd8vc"
Jul 23 23:17:06.702908 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:06.702875    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j7fm\" (UniqueName: \"kubernetes.io/projected/0543fa0d-39c5-4a75-a350-a9ac59f955ef-kube-api-access-9j7fm\") pod \"cert-manager-webhook-6486ffd997-bd8vc\" (UID: \"0543fa0d-39c5-4a75-a350-a9ac59f955ef\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bd8vc"
Jul 23 23:17:06.702908 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:06.702877    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0543fa0d-39c5-4a75-a350-a9ac59f955ef-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-bd8vc\" (UID: \"0543fa0d-39c5-4a75-a350-a9ac59f955ef\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bd8vc"
Jul 23 23:17:06.859412 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:06.859376    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-bd8vc"
Jul 23 23:17:06.980221 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:06.980197    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f886tm"
Jul 23 23:17:07.000992 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:07.000949    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-bd8vc"]
Jul 23 23:17:07.004247 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:17:07.004217    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0543fa0d_39c5_4a75_a350_a9ac59f955ef.slice/crio-c13c2e46b9d00d68ae2cc7e4a1ee8e573ec33bfcef67631b7a01d815bad53fee WatchSource:0}: Error finding container c13c2e46b9d00d68ae2cc7e4a1ee8e573ec33bfcef67631b7a01d815bad53fee: Status 404 returned error can't find the container with id c13c2e46b9d00d68ae2cc7e4a1ee8e573ec33bfcef67631b7a01d815bad53fee
Jul 23 23:17:07.097928 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:07.097898    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a3c6921-39f1-4b4c-9beb-463618ed6521-bundle\") pod \"4a3c6921-39f1-4b4c-9beb-463618ed6521\" (UID: \"4a3c6921-39f1-4b4c-9beb-463618ed6521\") "
Jul 23 23:17:07.098142 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:07.097963    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a3c6921-39f1-4b4c-9beb-463618ed6521-util\") pod \"4a3c6921-39f1-4b4c-9beb-463618ed6521\" (UID: \"4a3c6921-39f1-4b4c-9beb-463618ed6521\") "
Jul 23 23:17:07.098142 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:07.098041    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-92k5k\" (UniqueName: \"kubernetes.io/projected/4a3c6921-39f1-4b4c-9beb-463618ed6521-kube-api-access-92k5k\") pod \"4a3c6921-39f1-4b4c-9beb-463618ed6521\" (UID: \"4a3c6921-39f1-4b4c-9beb-463618ed6521\") "
Jul 23 23:17:07.098443 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:07.098376    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a3c6921-39f1-4b4c-9beb-463618ed6521-bundle" (OuterVolumeSpecName: "bundle") pod "4a3c6921-39f1-4b4c-9beb-463618ed6521" (UID: "4a3c6921-39f1-4b4c-9beb-463618ed6521"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:17:07.100256 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:07.100229    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a3c6921-39f1-4b4c-9beb-463618ed6521-kube-api-access-92k5k" (OuterVolumeSpecName: "kube-api-access-92k5k") pod "4a3c6921-39f1-4b4c-9beb-463618ed6521" (UID: "4a3c6921-39f1-4b4c-9beb-463618ed6521"). InnerVolumeSpecName "kube-api-access-92k5k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:17:07.102742 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:07.102718    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a3c6921-39f1-4b4c-9beb-463618ed6521-util" (OuterVolumeSpecName: "util") pod "4a3c6921-39f1-4b4c-9beb-463618ed6521" (UID: "4a3c6921-39f1-4b4c-9beb-463618ed6521"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:17:07.199599 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:07.199561    2548 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a3c6921-39f1-4b4c-9beb-463618ed6521-util\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:17:07.199599 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:07.199593    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92k5k\" (UniqueName: \"kubernetes.io/projected/4a3c6921-39f1-4b4c-9beb-463618ed6521-kube-api-access-92k5k\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:17:07.199599 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:07.199606    2548 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a3c6921-39f1-4b4c-9beb-463618ed6521-bundle\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:17:07.855051 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:07.855012    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-bd8vc" event={"ID":"0543fa0d-39c5-4a75-a350-a9ac59f955ef","Type":"ContainerStarted","Data":"c13c2e46b9d00d68ae2cc7e4a1ee8e573ec33bfcef67631b7a01d815bad53fee"}
Jul 23 23:17:07.857287 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:07.857249    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f886tm"
Jul 23 23:17:07.857287 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:07.857264    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f886tm" event={"ID":"4a3c6921-39f1-4b4c-9beb-463618ed6521","Type":"ContainerDied","Data":"c87b8d67bd02745dc8966a7d4eee927d66ab619bfa550a084a4056e0fd980ee7"}
Jul 23 23:17:07.857501 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:07.857310    2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c87b8d67bd02745dc8966a7d4eee927d66ab619bfa550a084a4056e0fd980ee7"
Jul 23 23:17:09.866694 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:09.866655    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-bd8vc" event={"ID":"0543fa0d-39c5-4a75-a350-a9ac59f955ef","Type":"ContainerStarted","Data":"99f2438c18f837463761e0d1afa9b1c6059cde8aa8f6eeb8589cfca22d44bbb3"}
Jul 23 23:17:09.867050 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:09.866747    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-bd8vc"
Jul 23 23:17:09.886075 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:09.886025    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-bd8vc" podStartSLOduration=1.121860165 podStartE2EDuration="3.886005106s" totalImagesPullingTime="2.764144941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:17:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:17:09.884892484 +0000 UTC m=+231.628621535" watchObservedRunningTime="2026-07-23 23:17:09.886005106 +0000 UTC m=+231.629734157"
Jul 23 23:17:11.876033 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:11.875996    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-sntqf" event={"ID":"148aaa98-c920-4b7a-a2d3-2c5ba6f67e7b","Type":"ContainerStarted","Data":"a25989c7751aff8b183f8bd7c796bdca793dddd33689f6cba7807747f9060edf"}
Jul 23 23:17:11.892082 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:11.892030    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-sntqf" podStartSLOduration=2.176328346 podStartE2EDuration="9.892014062s" totalImagesPullingTime="7.715685716s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:17:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:17:11.890746477 +0000 UTC m=+233.634475527" watchObservedRunningTime="2026-07-23 23:17:11.892014062 +0000 UTC m=+233.635743112"
Jul 23 23:17:15.873086 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:15.873057    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-bd8vc"
Jul 23 23:17:22.916096 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:22.916052    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpt7w7"]
Jul 23 23:17:22.916548 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:22.916455    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a3c6921-39f1-4b4c-9beb-463618ed6521" containerName="extract"
Jul 23 23:17:22.916548 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:22.916472    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a3c6921-39f1-4b4c-9beb-463618ed6521" containerName="extract"
Jul 23 23:17:22.916548 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:22.916489    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a3c6921-39f1-4b4c-9beb-463618ed6521" containerName="util"
Jul 23 23:17:22.916548 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:22.916498    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a3c6921-39f1-4b4c-9beb-463618ed6521" containerName="util"
Jul 23 23:17:22.916548 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:22.916508    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a3c6921-39f1-4b4c-9beb-463618ed6521" containerName="pull"
Jul 23 23:17:22.916548 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:22.916515    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a3c6921-39f1-4b4c-9beb-463618ed6521" containerName="pull"
Jul 23 23:17:22.916861 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:22.916640    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a3c6921-39f1-4b4c-9beb-463618ed6521" containerName="extract"
Jul 23 23:17:22.920943 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:22.920920    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpt7w7"
Jul 23 23:17:22.924518 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:22.924494    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 23 23:17:22.925618 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:22.925589    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ph65g\""
Jul 23 23:17:22.925618 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:22.925599    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 23 23:17:22.931108 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:22.931088    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpt7w7"]
Jul 23 23:17:23.034881 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:23.034850    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zv9kb\" (UniqueName: \"kubernetes.io/projected/35458fe9-24db-4520-89e1-b0475ac5b7e9-kube-api-access-zv9kb\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpt7w7\" (UID: \"35458fe9-24db-4520-89e1-b0475ac5b7e9\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpt7w7"
Jul 23 23:17:23.035103 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:23.034888    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35458fe9-24db-4520-89e1-b0475ac5b7e9-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpt7w7\" (UID: \"35458fe9-24db-4520-89e1-b0475ac5b7e9\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpt7w7"
Jul 23 23:17:23.035103 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:23.034910    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35458fe9-24db-4520-89e1-b0475ac5b7e9-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpt7w7\" (UID: \"35458fe9-24db-4520-89e1-b0475ac5b7e9\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpt7w7"
Jul 23 23:17:23.136378 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:23.136335    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zv9kb\" (UniqueName: \"kubernetes.io/projected/35458fe9-24db-4520-89e1-b0475ac5b7e9-kube-api-access-zv9kb\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpt7w7\" (UID: \"35458fe9-24db-4520-89e1-b0475ac5b7e9\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpt7w7"
Jul 23 23:17:23.136378 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:23.136379    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35458fe9-24db-4520-89e1-b0475ac5b7e9-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpt7w7\" (UID: \"35458fe9-24db-4520-89e1-b0475ac5b7e9\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpt7w7"
Jul 23 23:17:23.136645 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:23.136398    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35458fe9-24db-4520-89e1-b0475ac5b7e9-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpt7w7\" (UID: \"35458fe9-24db-4520-89e1-b0475ac5b7e9\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpt7w7"
Jul 23 23:17:23.136729 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:23.136706    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35458fe9-24db-4520-89e1-b0475ac5b7e9-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpt7w7\" (UID: \"35458fe9-24db-4520-89e1-b0475ac5b7e9\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpt7w7"
Jul 23 23:17:23.136729 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:23.136723    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35458fe9-24db-4520-89e1-b0475ac5b7e9-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpt7w7\" (UID: \"35458fe9-24db-4520-89e1-b0475ac5b7e9\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpt7w7"
Jul 23 23:17:23.144726 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:23.144705    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zv9kb\" (UniqueName: \"kubernetes.io/projected/35458fe9-24db-4520-89e1-b0475ac5b7e9-kube-api-access-zv9kb\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpt7w7\" (UID: \"35458fe9-24db-4520-89e1-b0475ac5b7e9\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpt7w7"
Jul 23 23:17:23.230798 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:23.230701    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpt7w7"
Jul 23 23:17:23.357776 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:23.357721    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpt7w7"]
Jul 23 23:17:23.359832 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:17:23.359808    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35458fe9_24db_4520_89e1_b0475ac5b7e9.slice/crio-a37cb9819661ed525a010ea0b236077e6dc0944c34d2c1365c7839cceb010061 WatchSource:0}: Error finding container a37cb9819661ed525a010ea0b236077e6dc0944c34d2c1365c7839cceb010061: Status 404 returned error can't find the container with id a37cb9819661ed525a010ea0b236077e6dc0944c34d2c1365c7839cceb010061
Jul 23 23:17:23.925971 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:23.925935    2548 generic.go:356] "Generic (PLEG): container finished" podID="35458fe9-24db-4520-89e1-b0475ac5b7e9" containerID="a381297afa3157cf5a02c34a51be9d42e96a49f5b3371ff6e315b02dca90092a" exitCode=0
Jul 23 23:17:23.926365 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:23.926030    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpt7w7" event={"ID":"35458fe9-24db-4520-89e1-b0475ac5b7e9","Type":"ContainerDied","Data":"a381297afa3157cf5a02c34a51be9d42e96a49f5b3371ff6e315b02dca90092a"}
Jul 23 23:17:23.926365 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:23.926084    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpt7w7" event={"ID":"35458fe9-24db-4520-89e1-b0475ac5b7e9","Type":"ContainerStarted","Data":"a37cb9819661ed525a010ea0b236077e6dc0944c34d2c1365c7839cceb010061"}
Jul 23 23:17:24.932500 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:24.932462    2548 generic.go:356] "Generic (PLEG): container finished" podID="35458fe9-24db-4520-89e1-b0475ac5b7e9" containerID="12b18f0ef4036250754d706eb79a7423d431803dd0271f7606b448d202a6e565" exitCode=0
Jul 23 23:17:24.932869 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:24.932508    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpt7w7" event={"ID":"35458fe9-24db-4520-89e1-b0475ac5b7e9","Type":"ContainerDied","Data":"12b18f0ef4036250754d706eb79a7423d431803dd0271f7606b448d202a6e565"}
Jul 23 23:17:25.938699 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:25.938662    2548 generic.go:356] "Generic (PLEG): container finished" podID="35458fe9-24db-4520-89e1-b0475ac5b7e9" containerID="812b4d3a8de65e6d46f6f90b6bd455a1925a6bfaf91651621d4e99d2b4a011c6" exitCode=0
Jul 23 23:17:25.939076 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:25.938743    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpt7w7" event={"ID":"35458fe9-24db-4520-89e1-b0475ac5b7e9","Type":"ContainerDied","Data":"812b4d3a8de65e6d46f6f90b6bd455a1925a6bfaf91651621d4e99d2b4a011c6"}
Jul 23 23:17:27.073285 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.073260    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpt7w7"
Jul 23 23:17:27.109026 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.108998    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-68dbcfb44d-d6rz4"]
Jul 23 23:17:27.109383 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.109365    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35458fe9-24db-4520-89e1-b0475ac5b7e9" containerName="extract"
Jul 23 23:17:27.109429 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.109389    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="35458fe9-24db-4520-89e1-b0475ac5b7e9" containerName="extract"
Jul 23 23:17:27.109429 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.109411    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35458fe9-24db-4520-89e1-b0475ac5b7e9" containerName="pull"
Jul 23 23:17:27.109429 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.109419    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="35458fe9-24db-4520-89e1-b0475ac5b7e9" containerName="pull"
Jul 23 23:17:27.109518 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.109438    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35458fe9-24db-4520-89e1-b0475ac5b7e9" containerName="util"
Jul 23 23:17:27.109518 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.109446    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="35458fe9-24db-4520-89e1-b0475ac5b7e9" containerName="util"
Jul 23 23:17:27.109581 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.109532    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="35458fe9-24db-4520-89e1-b0475ac5b7e9" containerName="extract"
Jul 23 23:17:27.112132 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.112117    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-68dbcfb44d-d6rz4"
Jul 23 23:17:27.114515 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.114499    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 23 23:17:27.114745 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.114721    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 23 23:17:27.114877 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.114857    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 23 23:17:27.114942 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.114912    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 23 23:17:27.115029 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.114950    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 23 23:17:27.115029 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.114952    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-6hxf7\""
Jul 23 23:17:27.120474 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.120453    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-68dbcfb44d-d6rz4"]
Jul 23 23:17:27.170493 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.170468    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35458fe9-24db-4520-89e1-b0475ac5b7e9-util\") pod \"35458fe9-24db-4520-89e1-b0475ac5b7e9\" (UID: \"35458fe9-24db-4520-89e1-b0475ac5b7e9\") "
Jul 23 23:17:27.170586 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.170513    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zv9kb\" (UniqueName: \"kubernetes.io/projected/35458fe9-24db-4520-89e1-b0475ac5b7e9-kube-api-access-zv9kb\") pod \"35458fe9-24db-4520-89e1-b0475ac5b7e9\" (UID: \"35458fe9-24db-4520-89e1-b0475ac5b7e9\") "
Jul 23 23:17:27.170586 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.170544    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35458fe9-24db-4520-89e1-b0475ac5b7e9-bundle\") pod \"35458fe9-24db-4520-89e1-b0475ac5b7e9\" (UID: \"35458fe9-24db-4520-89e1-b0475ac5b7e9\") "
Jul 23 23:17:27.171350 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.171329    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35458fe9-24db-4520-89e1-b0475ac5b7e9-bundle" (OuterVolumeSpecName: "bundle") pod "35458fe9-24db-4520-89e1-b0475ac5b7e9" (UID: "35458fe9-24db-4520-89e1-b0475ac5b7e9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:17:27.172548 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.172519    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35458fe9-24db-4520-89e1-b0475ac5b7e9-kube-api-access-zv9kb" (OuterVolumeSpecName: "kube-api-access-zv9kb") pod "35458fe9-24db-4520-89e1-b0475ac5b7e9" (UID: "35458fe9-24db-4520-89e1-b0475ac5b7e9"). InnerVolumeSpecName "kube-api-access-zv9kb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:17:27.176192 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.176168    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35458fe9-24db-4520-89e1-b0475ac5b7e9-util" (OuterVolumeSpecName: "util") pod "35458fe9-24db-4520-89e1-b0475ac5b7e9" (UID: "35458fe9-24db-4520-89e1-b0475ac5b7e9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:17:27.272163 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.272055    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ea8d4fb8-6b77-4e8b-9745-7c8bd9a4027f-manager-config\") pod \"lws-controller-manager-68dbcfb44d-d6rz4\" (UID: \"ea8d4fb8-6b77-4e8b-9745-7c8bd9a4027f\") " pod="openshift-lws-operator/lws-controller-manager-68dbcfb44d-d6rz4"
Jul 23 23:17:27.272163 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.272148    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ea8d4fb8-6b77-4e8b-9745-7c8bd9a4027f-cert\") pod \"lws-controller-manager-68dbcfb44d-d6rz4\" (UID: \"ea8d4fb8-6b77-4e8b-9745-7c8bd9a4027f\") " pod="openshift-lws-operator/lws-controller-manager-68dbcfb44d-d6rz4"
Jul 23 23:17:27.272368 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.272175    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ea8d4fb8-6b77-4e8b-9745-7c8bd9a4027f-metrics-cert\") pod \"lws-controller-manager-68dbcfb44d-d6rz4\" (UID: \"ea8d4fb8-6b77-4e8b-9745-7c8bd9a4027f\") " pod="openshift-lws-operator/lws-controller-manager-68dbcfb44d-d6rz4"
Jul 23 23:17:27.272368 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.272211    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqcs4\" (UniqueName: \"kubernetes.io/projected/ea8d4fb8-6b77-4e8b-9745-7c8bd9a4027f-kube-api-access-rqcs4\") pod \"lws-controller-manager-68dbcfb44d-d6rz4\" (UID: \"ea8d4fb8-6b77-4e8b-9745-7c8bd9a4027f\") " pod="openshift-lws-operator/lws-controller-manager-68dbcfb44d-d6rz4"
Jul 23 23:17:27.272368 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.272283    2548 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35458fe9-24db-4520-89e1-b0475ac5b7e9-util\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:17:27.272368 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.272298    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zv9kb\" (UniqueName: \"kubernetes.io/projected/35458fe9-24db-4520-89e1-b0475ac5b7e9-kube-api-access-zv9kb\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:17:27.272368 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.272310    2548 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35458fe9-24db-4520-89e1-b0475ac5b7e9-bundle\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:17:27.372877 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.372842    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ea8d4fb8-6b77-4e8b-9745-7c8bd9a4027f-cert\") pod \"lws-controller-manager-68dbcfb44d-d6rz4\" (UID: \"ea8d4fb8-6b77-4e8b-9745-7c8bd9a4027f\") " pod="openshift-lws-operator/lws-controller-manager-68dbcfb44d-d6rz4"
Jul 23 23:17:27.372877 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.372880    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ea8d4fb8-6b77-4e8b-9745-7c8bd9a4027f-metrics-cert\") pod \"lws-controller-manager-68dbcfb44d-d6rz4\" (UID: \"ea8d4fb8-6b77-4e8b-9745-7c8bd9a4027f\") " pod="openshift-lws-operator/lws-controller-manager-68dbcfb44d-d6rz4"
Jul 23 23:17:27.373151 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.372907    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqcs4\" (UniqueName: \"kubernetes.io/projected/ea8d4fb8-6b77-4e8b-9745-7c8bd9a4027f-kube-api-access-rqcs4\") pod \"lws-controller-manager-68dbcfb44d-d6rz4\" (UID: \"ea8d4fb8-6b77-4e8b-9745-7c8bd9a4027f\") " pod="openshift-lws-operator/lws-controller-manager-68dbcfb44d-d6rz4"
Jul 23 23:17:27.373151 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.372953    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ea8d4fb8-6b77-4e8b-9745-7c8bd9a4027f-manager-config\") pod \"lws-controller-manager-68dbcfb44d-d6rz4\" (UID: \"ea8d4fb8-6b77-4e8b-9745-7c8bd9a4027f\") " pod="openshift-lws-operator/lws-controller-manager-68dbcfb44d-d6rz4"
Jul 23 23:17:27.373707 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.373682    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ea8d4fb8-6b77-4e8b-9745-7c8bd9a4027f-manager-config\") pod \"lws-controller-manager-68dbcfb44d-d6rz4\" (UID: \"ea8d4fb8-6b77-4e8b-9745-7c8bd9a4027f\") " pod="openshift-lws-operator/lws-controller-manager-68dbcfb44d-d6rz4"
Jul 23 23:17:27.375494 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.375467    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ea8d4fb8-6b77-4e8b-9745-7c8bd9a4027f-metrics-cert\") pod \"lws-controller-manager-68dbcfb44d-d6rz4\" (UID: \"ea8d4fb8-6b77-4e8b-9745-7c8bd9a4027f\") " pod="openshift-lws-operator/lws-controller-manager-68dbcfb44d-d6rz4"
Jul 23 23:17:27.375567 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.375509    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ea8d4fb8-6b77-4e8b-9745-7c8bd9a4027f-cert\") pod \"lws-controller-manager-68dbcfb44d-d6rz4\" (UID: \"ea8d4fb8-6b77-4e8b-9745-7c8bd9a4027f\") " pod="openshift-lws-operator/lws-controller-manager-68dbcfb44d-d6rz4"
Jul 23 23:17:27.398638 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.398607    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqcs4\" (UniqueName: \"kubernetes.io/projected/ea8d4fb8-6b77-4e8b-9745-7c8bd9a4027f-kube-api-access-rqcs4\") pod \"lws-controller-manager-68dbcfb44d-d6rz4\" (UID: \"ea8d4fb8-6b77-4e8b-9745-7c8bd9a4027f\") " pod="openshift-lws-operator/lws-controller-manager-68dbcfb44d-d6rz4"
Jul 23 23:17:27.421435 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.421410    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-68dbcfb44d-d6rz4"
Jul 23 23:17:27.555508 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.555414    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-68dbcfb44d-d6rz4"]
Jul 23 23:17:27.557501 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:17:27.557474    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea8d4fb8_6b77_4e8b_9745_7c8bd9a4027f.slice/crio-9e877303a16bdc8b151adfba01451f13050d1cde952de57dd945d99efd37bdfe WatchSource:0}: Error finding container 9e877303a16bdc8b151adfba01451f13050d1cde952de57dd945d99efd37bdfe: Status 404 returned error can't find the container with id 9e877303a16bdc8b151adfba01451f13050d1cde952de57dd945d99efd37bdfe
Jul 23 23:17:27.949013 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.948985    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpt7w7"
Jul 23 23:17:27.949013 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.949003    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpt7w7" event={"ID":"35458fe9-24db-4520-89e1-b0475ac5b7e9","Type":"ContainerDied","Data":"a37cb9819661ed525a010ea0b236077e6dc0944c34d2c1365c7839cceb010061"}
Jul 23 23:17:27.949249 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.949045    2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a37cb9819661ed525a010ea0b236077e6dc0944c34d2c1365c7839cceb010061"
Jul 23 23:17:27.950190 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:27.950162    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-68dbcfb44d-d6rz4" event={"ID":"ea8d4fb8-6b77-4e8b-9745-7c8bd9a4027f","Type":"ContainerStarted","Data":"9e877303a16bdc8b151adfba01451f13050d1cde952de57dd945d99efd37bdfe"}
Jul 23 23:17:39.992006 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:39.991960    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-hzccw"]
Jul 23 23:17:39.999587 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:39.999565    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-hzccw"
Jul 23 23:17:40.004611 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:40.004587    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 23 23:17:40.004753 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:40.004613    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 23 23:17:40.004753 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:40.004640    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 23 23:17:40.005259 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:40.005241    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-bpjbc\""
Jul 23 23:17:40.005810 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:40.005791    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 23 23:17:40.017457 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:40.017431    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-hzccw"]
Jul 23 23:17:40.082616 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:40.082578    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c3744dc4-6e0c-4dfc-893c-1947479a7ff7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5fd4b8775f-hzccw\" (UID: \"c3744dc4-6e0c-4dfc-893c-1947479a7ff7\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-hzccw"
Jul 23 23:17:40.082787 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:40.082645    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c3744dc4-6e0c-4dfc-893c-1947479a7ff7-webhook-cert\") pod \"opendatahub-operator-controller-manager-5fd4b8775f-hzccw\" (UID: \"c3744dc4-6e0c-4dfc-893c-1947479a7ff7\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-hzccw"
Jul 23 23:17:40.082787 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:40.082680    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77ndm\" (UniqueName: \"kubernetes.io/projected/c3744dc4-6e0c-4dfc-893c-1947479a7ff7-kube-api-access-77ndm\") pod \"opendatahub-operator-controller-manager-5fd4b8775f-hzccw\" (UID: \"c3744dc4-6e0c-4dfc-893c-1947479a7ff7\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-hzccw"
Jul 23 23:17:40.082787 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:40.082708    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c3744dc4-6e0c-4dfc-893c-1947479a7ff7-manifests\") pod \"opendatahub-operator-controller-manager-5fd4b8775f-hzccw\" (UID: \"c3744dc4-6e0c-4dfc-893c-1947479a7ff7\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-hzccw"
Jul 23 23:17:40.082787 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:40.082732    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c3744dc4-6e0c-4dfc-893c-1947479a7ff7-tmp\") pod \"opendatahub-operator-controller-manager-5fd4b8775f-hzccw\" (UID: \"c3744dc4-6e0c-4dfc-893c-1947479a7ff7\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-hzccw"
Jul 23 23:17:40.184039 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:40.183995    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77ndm\" (UniqueName: \"kubernetes.io/projected/c3744dc4-6e0c-4dfc-893c-1947479a7ff7-kube-api-access-77ndm\") pod \"opendatahub-operator-controller-manager-5fd4b8775f-hzccw\" (UID: \"c3744dc4-6e0c-4dfc-893c-1947479a7ff7\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-hzccw"
Jul 23 23:17:40.184242 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:40.184061    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c3744dc4-6e0c-4dfc-893c-1947479a7ff7-manifests\") pod \"opendatahub-operator-controller-manager-5fd4b8775f-hzccw\" (UID: \"c3744dc4-6e0c-4dfc-893c-1947479a7ff7\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-hzccw"
Jul 23 23:17:40.184242 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:40.184100    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c3744dc4-6e0c-4dfc-893c-1947479a7ff7-tmp\") pod \"opendatahub-operator-controller-manager-5fd4b8775f-hzccw\" (UID: \"c3744dc4-6e0c-4dfc-893c-1947479a7ff7\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-hzccw"
Jul 23 23:17:40.184242 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:40.184131    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c3744dc4-6e0c-4dfc-893c-1947479a7ff7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5fd4b8775f-hzccw\" (UID: \"c3744dc4-6e0c-4dfc-893c-1947479a7ff7\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-hzccw"
Jul 23 23:17:40.184242 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:40.184222    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c3744dc4-6e0c-4dfc-893c-1947479a7ff7-webhook-cert\") pod \"opendatahub-operator-controller-manager-5fd4b8775f-hzccw\" (UID: \"c3744dc4-6e0c-4dfc-893c-1947479a7ff7\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-hzccw"
Jul 23 23:17:40.184531 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:40.184513    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c3744dc4-6e0c-4dfc-893c-1947479a7ff7-manifests\") pod \"opendatahub-operator-controller-manager-5fd4b8775f-hzccw\" (UID: \"c3744dc4-6e0c-4dfc-893c-1947479a7ff7\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-hzccw"
Jul 23 23:17:40.186502 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:40.186479    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c3744dc4-6e0c-4dfc-893c-1947479a7ff7-tmp\") pod \"opendatahub-operator-controller-manager-5fd4b8775f-hzccw\" (UID: \"c3744dc4-6e0c-4dfc-893c-1947479a7ff7\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-hzccw"
Jul 23 23:17:40.186604 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:40.186529    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c3744dc4-6e0c-4dfc-893c-1947479a7ff7-webhook-cert\") pod \"opendatahub-operator-controller-manager-5fd4b8775f-hzccw\" (UID: \"c3744dc4-6e0c-4dfc-893c-1947479a7ff7\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-hzccw"
Jul 23 23:17:40.186604 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:40.186576    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c3744dc4-6e0c-4dfc-893c-1947479a7ff7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5fd4b8775f-hzccw\" (UID: \"c3744dc4-6e0c-4dfc-893c-1947479a7ff7\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-hzccw"
Jul 23 23:17:40.193645 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:40.193623    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77ndm\" (UniqueName: \"kubernetes.io/projected/c3744dc4-6e0c-4dfc-893c-1947479a7ff7-kube-api-access-77ndm\") pod \"opendatahub-operator-controller-manager-5fd4b8775f-hzccw\" (UID: \"c3744dc4-6e0c-4dfc-893c-1947479a7ff7\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-hzccw"
Jul 23 23:17:40.310935 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:40.310849    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-hzccw"
Jul 23 23:17:40.453309 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:40.453278    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-hzccw"]
Jul 23 23:17:40.456851 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:17:40.456825    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3744dc4_6e0c_4dfc_893c_1947479a7ff7.slice/crio-0408002b5d020781a4b5fd2dd7d8c56ab7dc9771d165b416e893f3bb39f14db2 WatchSource:0}: Error finding container 0408002b5d020781a4b5fd2dd7d8c56ab7dc9771d165b416e893f3bb39f14db2: Status 404 returned error can't find the container with id 0408002b5d020781a4b5fd2dd7d8c56ab7dc9771d165b416e893f3bb39f14db2
Jul 23 23:17:41.006841 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:41.006808    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-hzccw" event={"ID":"c3744dc4-6e0c-4dfc-893c-1947479a7ff7","Type":"ContainerStarted","Data":"0408002b5d020781a4b5fd2dd7d8c56ab7dc9771d165b416e893f3bb39f14db2"}
Jul 23 23:17:42.014541 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:42.014240    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-68dbcfb44d-d6rz4" event={"ID":"ea8d4fb8-6b77-4e8b-9745-7c8bd9a4027f","Type":"ContainerStarted","Data":"cc9a52ca71f0869d2271634e414a100c09f9a327f70a3ad00051995e855e8fea"}
Jul 23 23:17:42.014541 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:42.014324    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-68dbcfb44d-d6rz4"
Jul 23 23:17:42.033182 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:42.033123    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-68dbcfb44d-d6rz4" podStartSLOduration=0.90827237 podStartE2EDuration="15.033104406s" totalImagesPullingTime="14.124832036s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:17:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:17:42.030506096 +0000 UTC m=+263.774235149" watchObservedRunningTime="2026-07-23 23:17:42.033104406 +0000 UTC m=+263.776833458"
Jul 23 23:17:44.029808 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:44.029770    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-hzccw" event={"ID":"c3744dc4-6e0c-4dfc-893c-1947479a7ff7","Type":"ContainerStarted","Data":"5c6c9f6daccbb6cc2ce0695a2e883e65a095d37949847a597884a2e50cb82e6b"}
Jul 23 23:17:45.035462 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:45.035365    2548 generic.go:356] "Generic (PLEG): container finished" podID="c3744dc4-6e0c-4dfc-893c-1947479a7ff7" containerID="5c6c9f6daccbb6cc2ce0695a2e883e65a095d37949847a597884a2e50cb82e6b" exitCode=0
Jul 23 23:17:45.035462 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:45.035423    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-hzccw" event={"ID":"c3744dc4-6e0c-4dfc-893c-1947479a7ff7","Type":"ContainerDied","Data":"5c6c9f6daccbb6cc2ce0695a2e883e65a095d37949847a597884a2e50cb82e6b"}
Jul 23 23:17:46.041904 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:46.041865    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-hzccw" event={"ID":"c3744dc4-6e0c-4dfc-893c-1947479a7ff7","Type":"ContainerStarted","Data":"8d75aa12988d6df75957a758a8f2bea9eb4faf1acdfe0f9d2ace1c2094885fc3"}
Jul 23 23:17:46.042371 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:46.042021    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-hzccw"
Jul 23 23:17:46.064007 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:46.063924    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-hzccw" podStartSLOduration=2.4144566 podStartE2EDuration="7.0639032s" totalImagesPullingTime="3.574613125s" totalInitContainerRuntime="1.074833475s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:17:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:17:46.063570722 +0000 UTC m=+267.807299775" watchObservedRunningTime="2026-07-23 23:17:46.0639032 +0000 UTC m=+267.807632247"
Jul 23 23:17:53.026957 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:53.026926    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-68dbcfb44d-d6rz4"
Jul 23 23:17:57.049134 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:57.049103    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-hzccw"
Jul 23 23:17:59.617350 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:59.617311    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-8b54995f6-sz5t4"]
Jul 23 23:17:59.625564 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:59.625546    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-8b54995f6-sz5t4"
Jul 23 23:17:59.630428 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:59.630405    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 23 23:17:59.630710 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:59.630686    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 23 23:17:59.630990 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:59.630958    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 23 23:17:59.631320 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:59.631305    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 23 23:17:59.632225 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:59.631867    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-kqgw6\""
Jul 23 23:17:59.636883 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:59.636862    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-8b54995f6-sz5t4"]
Jul 23 23:17:59.754415 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:59.754377    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d186cae4-08f8-465f-add7-507deb5699a2-tls-certs\") pod \"kube-auth-proxy-8b54995f6-sz5t4\" (UID: \"d186cae4-08f8-465f-add7-507deb5699a2\") " pod="openshift-ingress/kube-auth-proxy-8b54995f6-sz5t4"
Jul 23 23:17:59.754603 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:59.754529    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d186cae4-08f8-465f-add7-507deb5699a2-tmp\") pod \"kube-auth-proxy-8b54995f6-sz5t4\" (UID: \"d186cae4-08f8-465f-add7-507deb5699a2\") " pod="openshift-ingress/kube-auth-proxy-8b54995f6-sz5t4"
Jul 23 23:17:59.754603 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:59.754566    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtgvw\" (UniqueName: \"kubernetes.io/projected/d186cae4-08f8-465f-add7-507deb5699a2-kube-api-access-vtgvw\") pod \"kube-auth-proxy-8b54995f6-sz5t4\" (UID: \"d186cae4-08f8-465f-add7-507deb5699a2\") " pod="openshift-ingress/kube-auth-proxy-8b54995f6-sz5t4"
Jul 23 23:17:59.855093 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:59.855059    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d186cae4-08f8-465f-add7-507deb5699a2-tmp\") pod \"kube-auth-proxy-8b54995f6-sz5t4\" (UID: \"d186cae4-08f8-465f-add7-507deb5699a2\") " pod="openshift-ingress/kube-auth-proxy-8b54995f6-sz5t4"
Jul 23 23:17:59.855093 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:59.855096    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtgvw\" (UniqueName: \"kubernetes.io/projected/d186cae4-08f8-465f-add7-507deb5699a2-kube-api-access-vtgvw\") pod \"kube-auth-proxy-8b54995f6-sz5t4\" (UID: \"d186cae4-08f8-465f-add7-507deb5699a2\") " pod="openshift-ingress/kube-auth-proxy-8b54995f6-sz5t4"
Jul 23 23:17:59.855291 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:59.855146    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d186cae4-08f8-465f-add7-507deb5699a2-tls-certs\") pod \"kube-auth-proxy-8b54995f6-sz5t4\" (UID: \"d186cae4-08f8-465f-add7-507deb5699a2\") " pod="openshift-ingress/kube-auth-proxy-8b54995f6-sz5t4"
Jul 23 23:17:59.857508 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:59.857479    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d186cae4-08f8-465f-add7-507deb5699a2-tmp\") pod \"kube-auth-proxy-8b54995f6-sz5t4\" (UID: \"d186cae4-08f8-465f-add7-507deb5699a2\") " pod="openshift-ingress/kube-auth-proxy-8b54995f6-sz5t4"
Jul 23 23:17:59.857798 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:59.857777    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d186cae4-08f8-465f-add7-507deb5699a2-tls-certs\") pod \"kube-auth-proxy-8b54995f6-sz5t4\" (UID: \"d186cae4-08f8-465f-add7-507deb5699a2\") " pod="openshift-ingress/kube-auth-proxy-8b54995f6-sz5t4"
Jul 23 23:17:59.879022 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:59.878919    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtgvw\" (UniqueName: \"kubernetes.io/projected/d186cae4-08f8-465f-add7-507deb5699a2-kube-api-access-vtgvw\") pod \"kube-auth-proxy-8b54995f6-sz5t4\" (UID: \"d186cae4-08f8-465f-add7-507deb5699a2\") " pod="openshift-ingress/kube-auth-proxy-8b54995f6-sz5t4"
Jul 23 23:17:59.934773 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:17:59.934738    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-8b54995f6-sz5t4"
Jul 23 23:18:00.086370 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:18:00.086346    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-8b54995f6-sz5t4"]
Jul 23 23:18:00.089667 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:18:00.089633    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd186cae4_08f8_465f_add7_507deb5699a2.slice/crio-9ba3a63de323056a50db299526a316ceaea806b111adc2b4b23712bf05a5df55 WatchSource:0}: Error finding container 9ba3a63de323056a50db299526a316ceaea806b111adc2b4b23712bf05a5df55: Status 404 returned error can't find the container with id 9ba3a63de323056a50db299526a316ceaea806b111adc2b4b23712bf05a5df55
Jul 23 23:18:00.106224 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:18:00.106190    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-8b54995f6-sz5t4" event={"ID":"d186cae4-08f8-465f-add7-507deb5699a2","Type":"ContainerStarted","Data":"9ba3a63de323056a50db299526a316ceaea806b111adc2b4b23712bf05a5df55"}
Jul 23 23:18:03.122835 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:18:03.122793    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-8b54995f6-sz5t4" event={"ID":"d186cae4-08f8-465f-add7-507deb5699a2","Type":"ContainerStarted","Data":"5d125ff1f8bb9cd094e23adcad32ddbb0f8691626b55b30acf3c7d187fbe35bd"}
Jul 23 23:18:03.140235 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:18:03.140180    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-8b54995f6-sz5t4" podStartSLOduration=1.762356875 podStartE2EDuration="4.140162652s" totalImagesPullingTime="2.377805777s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:17:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:18:03.138716639 +0000 UTC m=+284.882445688" watchObservedRunningTime="2026-07-23 23:18:03.140162652 +0000 UTC m=+284.883891766"
Jul 23 23:18:09.995454 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:18:09.995416    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-sxwqj"]
Jul 23 23:18:10.000018 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:18:09.999995    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sxwqj"
Jul 23 23:18:10.003890 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:18:10.003863    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 23 23:18:10.003890 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:18:10.003880    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-jml5d\""
Jul 23 23:18:10.012371 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:18:10.012342    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-sxwqj"]
Jul 23 23:18:10.149897 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:18:10.149860    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2e7855e5-8651-4934-b6e4-43e4d8511ad3-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-sxwqj\" (UID: \"2e7855e5-8651-4934-b6e4-43e4d8511ad3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sxwqj"
Jul 23 23:18:10.150082 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:18:10.149918    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nvlj\" (UniqueName: \"kubernetes.io/projected/2e7855e5-8651-4934-b6e4-43e4d8511ad3-kube-api-access-7nvlj\") pod \"llmisvc-controller-manager-777c6bd7d5-sxwqj\" (UID: \"2e7855e5-8651-4934-b6e4-43e4d8511ad3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sxwqj"
Jul 23 23:18:10.251204 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:18:10.251107    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2e7855e5-8651-4934-b6e4-43e4d8511ad3-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-sxwqj\" (UID: \"2e7855e5-8651-4934-b6e4-43e4d8511ad3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sxwqj"
Jul 23 23:18:10.251204 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:18:10.251161    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nvlj\" (UniqueName: \"kubernetes.io/projected/2e7855e5-8651-4934-b6e4-43e4d8511ad3-kube-api-access-7nvlj\") pod \"llmisvc-controller-manager-777c6bd7d5-sxwqj\" (UID: \"2e7855e5-8651-4934-b6e4-43e4d8511ad3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sxwqj"
Jul 23 23:18:10.253764 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:18:10.253736    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2e7855e5-8651-4934-b6e4-43e4d8511ad3-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-sxwqj\" (UID: \"2e7855e5-8651-4934-b6e4-43e4d8511ad3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sxwqj"
Jul 23 23:18:10.262232 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:18:10.262205    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nvlj\" (UniqueName: \"kubernetes.io/projected/2e7855e5-8651-4934-b6e4-43e4d8511ad3-kube-api-access-7nvlj\") pod \"llmisvc-controller-manager-777c6bd7d5-sxwqj\" (UID: \"2e7855e5-8651-4934-b6e4-43e4d8511ad3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sxwqj"
Jul 23 23:18:10.310657 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:18:10.310620    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sxwqj"
Jul 23 23:18:10.440565 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:18:10.440540    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-sxwqj"]
Jul 23 23:18:10.443223 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:18:10.443190    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2e7855e5_8651_4934_b6e4_43e4d8511ad3.slice/crio-b34998bb33fe76e8e4a2a260c784ce818fc9177c966d61936764a5d3a603858d WatchSource:0}: Error finding container b34998bb33fe76e8e4a2a260c784ce818fc9177c966d61936764a5d3a603858d: Status 404 returned error can't find the container with id b34998bb33fe76e8e4a2a260c784ce818fc9177c966d61936764a5d3a603858d
Jul 23 23:18:11.159621 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:18:11.159587    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sxwqj" event={"ID":"2e7855e5-8651-4934-b6e4-43e4d8511ad3","Type":"ContainerStarted","Data":"b34998bb33fe76e8e4a2a260c784ce818fc9177c966d61936764a5d3a603858d"}
Jul 23 23:18:14.179371 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:18:14.179329    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sxwqj" event={"ID":"2e7855e5-8651-4934-b6e4-43e4d8511ad3","Type":"ContainerStarted","Data":"0c19c2e542401ff055d3d61fee87070a956fb91681f92ad28fcc4da11498da26"}
Jul 23 23:18:14.179836 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:18:14.179469    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sxwqj"
Jul 23 23:18:14.196370 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:18:14.196313    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sxwqj" podStartSLOduration=2.228350558 podStartE2EDuration="5.196295651s" totalImagesPullingTime="2.967945093s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:18:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:18:14.195906908 +0000 UTC m=+295.939635957" watchObservedRunningTime="2026-07-23 23:18:14.196295651 +0000 UTC m=+295.940024701"
Jul 23 23:18:18.584718 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:18:18.584642    2548 kubelet.go:1648] "Image garbage collection succeeded"
Jul 23 23:18:45.186620 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:18:45.186589    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sxwqj"
Jul 23 23:19:16.603356 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:16.603320    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nzckn"]
Jul 23 23:19:16.606732 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:16.606715    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nzckn"
Jul 23 23:19:16.609545 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:16.609518    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 23 23:19:16.609660 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:16.609527    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-rtq24\""
Jul 23 23:19:16.610611 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:16.610592    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 23 23:19:16.614889 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:16.614867    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nzckn"]
Jul 23 23:19:16.626918 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:16.626877    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dde480a2-3c65-4992-9070-febf7b7477a8-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nzckn\" (UID: \"dde480a2-3c65-4992-9070-febf7b7477a8\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nzckn"
Jul 23 23:19:16.627093 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:16.626921    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6j9ld\" (UniqueName: \"kubernetes.io/projected/dde480a2-3c65-4992-9070-febf7b7477a8-kube-api-access-6j9ld\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nzckn\" (UID: \"dde480a2-3c65-4992-9070-febf7b7477a8\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nzckn"
Jul 23 23:19:16.627093 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:16.627045    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dde480a2-3c65-4992-9070-febf7b7477a8-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nzckn\" (UID: \"dde480a2-3c65-4992-9070-febf7b7477a8\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nzckn"
Jul 23 23:19:16.727864 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:16.727834    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dde480a2-3c65-4992-9070-febf7b7477a8-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nzckn\" (UID: \"dde480a2-3c65-4992-9070-febf7b7477a8\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nzckn"
Jul 23 23:19:16.728049 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:16.727868    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6j9ld\" (UniqueName: \"kubernetes.io/projected/dde480a2-3c65-4992-9070-febf7b7477a8-kube-api-access-6j9ld\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nzckn\" (UID: \"dde480a2-3c65-4992-9070-febf7b7477a8\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nzckn"
Jul 23 23:19:16.728049 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:16.727924    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dde480a2-3c65-4992-9070-febf7b7477a8-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nzckn\" (UID: \"dde480a2-3c65-4992-9070-febf7b7477a8\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nzckn"
Jul 23 23:19:16.728215 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:16.728195    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dde480a2-3c65-4992-9070-febf7b7477a8-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nzckn\" (UID: \"dde480a2-3c65-4992-9070-febf7b7477a8\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nzckn"
Jul 23 23:19:16.728296 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:16.728243    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dde480a2-3c65-4992-9070-febf7b7477a8-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nzckn\" (UID: \"dde480a2-3c65-4992-9070-febf7b7477a8\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nzckn"
Jul 23 23:19:16.735861 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:16.735836    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6j9ld\" (UniqueName: \"kubernetes.io/projected/dde480a2-3c65-4992-9070-febf7b7477a8-kube-api-access-6j9ld\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nzckn\" (UID: \"dde480a2-3c65-4992-9070-febf7b7477a8\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nzckn"
Jul 23 23:19:16.917083 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:16.917051    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nzckn"
Jul 23 23:19:17.049153 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:17.049129    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nzckn"]
Jul 23 23:19:17.051142 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:19:17.051118    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddde480a2_3c65_4992_9070_febf7b7477a8.slice/crio-a9ce4a8a895e2d400ad2da4d1402ab76e53e383ea45ba9cea1a142235589e471 WatchSource:0}: Error finding container a9ce4a8a895e2d400ad2da4d1402ab76e53e383ea45ba9cea1a142235589e471: Status 404 returned error can't find the container with id a9ce4a8a895e2d400ad2da4d1402ab76e53e383ea45ba9cea1a142235589e471
Jul 23 23:19:17.402427 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:17.402393    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fhwbc"]
Jul 23 23:19:17.405757 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:17.405741    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fhwbc"
Jul 23 23:19:17.414293 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:17.414266    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fhwbc"]
Jul 23 23:19:17.433271 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:17.433246    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dcd3d034-aa51-460a-8274-7ca3ef0e7fd7-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fhwbc\" (UID: \"dcd3d034-aa51-460a-8274-7ca3ef0e7fd7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fhwbc"
Jul 23 23:19:17.433390 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:17.433302    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsb2z\" (UniqueName: \"kubernetes.io/projected/dcd3d034-aa51-460a-8274-7ca3ef0e7fd7-kube-api-access-nsb2z\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fhwbc\" (UID: \"dcd3d034-aa51-460a-8274-7ca3ef0e7fd7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fhwbc"
Jul 23 23:19:17.433390 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:17.433357    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dcd3d034-aa51-460a-8274-7ca3ef0e7fd7-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fhwbc\" (UID: \"dcd3d034-aa51-460a-8274-7ca3ef0e7fd7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fhwbc"
Jul 23 23:19:17.466401 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:17.466369    2548 generic.go:356] "Generic (PLEG): container finished" podID="dde480a2-3c65-4992-9070-febf7b7477a8" containerID="a2053a84cc68f844811162d9d630b5b5730e68748d42348b5452f6f756e4d610" exitCode=0
Jul 23 23:19:17.466512 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:17.466425    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nzckn" event={"ID":"dde480a2-3c65-4992-9070-febf7b7477a8","Type":"ContainerDied","Data":"a2053a84cc68f844811162d9d630b5b5730e68748d42348b5452f6f756e4d610"}
Jul 23 23:19:17.466512 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:17.466467    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nzckn" event={"ID":"dde480a2-3c65-4992-9070-febf7b7477a8","Type":"ContainerStarted","Data":"a9ce4a8a895e2d400ad2da4d1402ab76e53e383ea45ba9cea1a142235589e471"}
Jul 23 23:19:17.466944 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:17.466928    2548 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 23:19:17.534416 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:17.534383    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dcd3d034-aa51-460a-8274-7ca3ef0e7fd7-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fhwbc\" (UID: \"dcd3d034-aa51-460a-8274-7ca3ef0e7fd7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fhwbc"
Jul 23 23:19:17.534556 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:17.534432    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsb2z\" (UniqueName: \"kubernetes.io/projected/dcd3d034-aa51-460a-8274-7ca3ef0e7fd7-kube-api-access-nsb2z\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fhwbc\" (UID: \"dcd3d034-aa51-460a-8274-7ca3ef0e7fd7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fhwbc"
Jul 23 23:19:17.534556 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:17.534481    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dcd3d034-aa51-460a-8274-7ca3ef0e7fd7-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fhwbc\" (UID: \"dcd3d034-aa51-460a-8274-7ca3ef0e7fd7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fhwbc"
Jul 23 23:19:17.534819 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:17.534802    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dcd3d034-aa51-460a-8274-7ca3ef0e7fd7-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fhwbc\" (UID: \"dcd3d034-aa51-460a-8274-7ca3ef0e7fd7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fhwbc"
Jul 23 23:19:17.534891 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:17.534873    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dcd3d034-aa51-460a-8274-7ca3ef0e7fd7-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fhwbc\" (UID: \"dcd3d034-aa51-460a-8274-7ca3ef0e7fd7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fhwbc"
Jul 23 23:19:17.542786 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:17.542764    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsb2z\" (UniqueName: \"kubernetes.io/projected/dcd3d034-aa51-460a-8274-7ca3ef0e7fd7-kube-api-access-nsb2z\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fhwbc\" (UID: \"dcd3d034-aa51-460a-8274-7ca3ef0e7fd7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fhwbc"
Jul 23 23:19:17.715625 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:17.715543    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fhwbc"
Jul 23 23:19:17.839709 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:17.839683    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fhwbc"]
Jul 23 23:19:17.841602 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:19:17.841575    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcd3d034_aa51_460a_8274_7ca3ef0e7fd7.slice/crio-c224967e97805e23a1c3c4299f8871720d0fc6126da2e022fe4114339ec74d60 WatchSource:0}: Error finding container c224967e97805e23a1c3c4299f8871720d0fc6126da2e022fe4114339ec74d60: Status 404 returned error can't find the container with id c224967e97805e23a1c3c4299f8871720d0fc6126da2e022fe4114339ec74d60
Jul 23 23:19:18.007221 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:18.007159    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pvffr"]
Jul 23 23:19:18.010641 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:18.010626    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pvffr"
Jul 23 23:19:18.019030 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:18.019004    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pvffr"]
Jul 23 23:19:18.037748 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:18.037718    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krctl\" (UniqueName: \"kubernetes.io/projected/81d19c8e-0a24-4bdd-8d6f-4b83d1d54968-kube-api-access-krctl\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pvffr\" (UID: \"81d19c8e-0a24-4bdd-8d6f-4b83d1d54968\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pvffr"
Jul 23 23:19:18.037896 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:18.037768    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/81d19c8e-0a24-4bdd-8d6f-4b83d1d54968-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pvffr\" (UID: \"81d19c8e-0a24-4bdd-8d6f-4b83d1d54968\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pvffr"
Jul 23 23:19:18.037896 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:18.037833    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/81d19c8e-0a24-4bdd-8d6f-4b83d1d54968-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pvffr\" (UID: \"81d19c8e-0a24-4bdd-8d6f-4b83d1d54968\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pvffr"
Jul 23 23:19:18.138340 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:18.138302    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/81d19c8e-0a24-4bdd-8d6f-4b83d1d54968-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pvffr\" (UID: \"81d19c8e-0a24-4bdd-8d6f-4b83d1d54968\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pvffr"
Jul 23 23:19:18.138507 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:18.138355    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/81d19c8e-0a24-4bdd-8d6f-4b83d1d54968-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pvffr\" (UID: \"81d19c8e-0a24-4bdd-8d6f-4b83d1d54968\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pvffr"
Jul 23 23:19:18.138507 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:18.138396    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krctl\" (UniqueName: \"kubernetes.io/projected/81d19c8e-0a24-4bdd-8d6f-4b83d1d54968-kube-api-access-krctl\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pvffr\" (UID: \"81d19c8e-0a24-4bdd-8d6f-4b83d1d54968\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pvffr"
Jul 23 23:19:18.138734 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:18.138712    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/81d19c8e-0a24-4bdd-8d6f-4b83d1d54968-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pvffr\" (UID: \"81d19c8e-0a24-4bdd-8d6f-4b83d1d54968\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pvffr"
Jul 23 23:19:18.138770 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:18.138728    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/81d19c8e-0a24-4bdd-8d6f-4b83d1d54968-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pvffr\" (UID: \"81d19c8e-0a24-4bdd-8d6f-4b83d1d54968\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pvffr"
Jul 23 23:19:18.147221 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:18.147189    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krctl\" (UniqueName: \"kubernetes.io/projected/81d19c8e-0a24-4bdd-8d6f-4b83d1d54968-kube-api-access-krctl\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pvffr\" (UID: \"81d19c8e-0a24-4bdd-8d6f-4b83d1d54968\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pvffr"
Jul 23 23:19:18.327545 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:18.327523    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pvffr"
Jul 23 23:19:18.404612 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:18.404583    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s5vzb"]
Jul 23 23:19:18.408280 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:18.408260    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s5vzb"
Jul 23 23:19:18.415806 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:18.415765    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s5vzb"]
Jul 23 23:19:18.440970 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:18.440943    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96ec0397-59ad-4135-a7b3-cfb4c116f779-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s5vzb\" (UID: \"96ec0397-59ad-4135-a7b3-cfb4c116f779\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s5vzb"
Jul 23 23:19:18.441119 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:18.441036    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6h7s\" (UniqueName: \"kubernetes.io/projected/96ec0397-59ad-4135-a7b3-cfb4c116f779-kube-api-access-v6h7s\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s5vzb\" (UID: \"96ec0397-59ad-4135-a7b3-cfb4c116f779\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s5vzb"
Jul 23 23:19:18.441119 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:18.441081    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96ec0397-59ad-4135-a7b3-cfb4c116f779-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s5vzb\" (UID: \"96ec0397-59ad-4135-a7b3-cfb4c116f779\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s5vzb"
Jul 23 23:19:18.465139 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:18.465117    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pvffr"]
Jul 23 23:19:18.467311 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:19:18.467274    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81d19c8e_0a24_4bdd_8d6f_4b83d1d54968.slice/crio-6408f188292b3e6fc1be911f36c6a129b4bc3d0841bab2a9effa654774cca92b WatchSource:0}: Error finding container 6408f188292b3e6fc1be911f36c6a129b4bc3d0841bab2a9effa654774cca92b: Status 404 returned error can't find the container with id 6408f188292b3e6fc1be911f36c6a129b4bc3d0841bab2a9effa654774cca92b
Jul 23 23:19:18.472446 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:18.472424    2548 generic.go:356] "Generic (PLEG): container finished" podID="dde480a2-3c65-4992-9070-febf7b7477a8" containerID="c716fc6bffded7f7351d2a6ff1418100377f097443c6fb0c9b4fef93474a28b5" exitCode=0
Jul 23 23:19:18.472532 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:18.472505    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nzckn" event={"ID":"dde480a2-3c65-4992-9070-febf7b7477a8","Type":"ContainerDied","Data":"c716fc6bffded7f7351d2a6ff1418100377f097443c6fb0c9b4fef93474a28b5"}
Jul 23 23:19:18.474164 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:18.474143    2548 generic.go:356] "Generic (PLEG): container finished" podID="dcd3d034-aa51-460a-8274-7ca3ef0e7fd7" containerID="b03135f7fdcd2664bfa1b631a6ab56ff6bcec01f8ef3a5151ac037db3dd50aba" exitCode=0
Jul 23 23:19:18.474232 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:18.474196    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fhwbc" event={"ID":"dcd3d034-aa51-460a-8274-7ca3ef0e7fd7","Type":"ContainerDied","Data":"b03135f7fdcd2664bfa1b631a6ab56ff6bcec01f8ef3a5151ac037db3dd50aba"}
Jul 23 23:19:18.474292 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:18.474234    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fhwbc" event={"ID":"dcd3d034-aa51-460a-8274-7ca3ef0e7fd7","Type":"ContainerStarted","Data":"c224967e97805e23a1c3c4299f8871720d0fc6126da2e022fe4114339ec74d60"}
Jul 23 23:19:18.542236 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:18.542201    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6h7s\" (UniqueName: \"kubernetes.io/projected/96ec0397-59ad-4135-a7b3-cfb4c116f779-kube-api-access-v6h7s\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s5vzb\" (UID: \"96ec0397-59ad-4135-a7b3-cfb4c116f779\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s5vzb"
Jul 23 23:19:18.542364 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:18.542316    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96ec0397-59ad-4135-a7b3-cfb4c116f779-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s5vzb\" (UID: \"96ec0397-59ad-4135-a7b3-cfb4c116f779\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s5vzb"
Jul 23 23:19:18.542432 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:18.542379    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96ec0397-59ad-4135-a7b3-cfb4c116f779-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s5vzb\" (UID: \"96ec0397-59ad-4135-a7b3-cfb4c116f779\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s5vzb"
Jul 23 23:19:18.542768 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:18.542748    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96ec0397-59ad-4135-a7b3-cfb4c116f779-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s5vzb\" (UID: \"96ec0397-59ad-4135-a7b3-cfb4c116f779\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s5vzb"
Jul 23 23:19:18.542837 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:18.542814    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96ec0397-59ad-4135-a7b3-cfb4c116f779-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s5vzb\" (UID: \"96ec0397-59ad-4135-a7b3-cfb4c116f779\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s5vzb"
Jul 23 23:19:18.551140 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:18.551114    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6h7s\" (UniqueName: \"kubernetes.io/projected/96ec0397-59ad-4135-a7b3-cfb4c116f779-kube-api-access-v6h7s\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s5vzb\" (UID: \"96ec0397-59ad-4135-a7b3-cfb4c116f779\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s5vzb"
Jul 23 23:19:18.764954 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:18.764912    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s5vzb"
Jul 23 23:19:18.910149 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:18.910123    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s5vzb"]
Jul 23 23:19:18.911651 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:19:18.911624    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96ec0397_59ad_4135_a7b3_cfb4c116f779.slice/crio-10a0165219d64a7a13067cdb552248e9371cea7af5f538c784d58eb5852cd01b WatchSource:0}: Error finding container 10a0165219d64a7a13067cdb552248e9371cea7af5f538c784d58eb5852cd01b: Status 404 returned error can't find the container with id 10a0165219d64a7a13067cdb552248e9371cea7af5f538c784d58eb5852cd01b
Jul 23 23:19:19.481654 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:19.481622    2548 generic.go:356] "Generic (PLEG): container finished" podID="dcd3d034-aa51-460a-8274-7ca3ef0e7fd7" containerID="816d674110008db4b1f6b9cb02af30c7d7d5c91de43d6a1ec576bc992c4a91e5" exitCode=0
Jul 23 23:19:19.481791 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:19.481702    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fhwbc" event={"ID":"dcd3d034-aa51-460a-8274-7ca3ef0e7fd7","Type":"ContainerDied","Data":"816d674110008db4b1f6b9cb02af30c7d7d5c91de43d6a1ec576bc992c4a91e5"}
Jul 23 23:19:19.483858 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:19.483831    2548 generic.go:356] "Generic (PLEG): container finished" podID="dde480a2-3c65-4992-9070-febf7b7477a8" containerID="5ac0c58e0d80f43d4e791dfd251673a4b54dbcb796b51ea29c246e7bad87af36" exitCode=0
Jul 23 23:19:19.483950 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:19.483920    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nzckn" event={"ID":"dde480a2-3c65-4992-9070-febf7b7477a8","Type":"ContainerDied","Data":"5ac0c58e0d80f43d4e791dfd251673a4b54dbcb796b51ea29c246e7bad87af36"}
Jul 23 23:19:19.485304 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:19.485278    2548 generic.go:356] "Generic (PLEG): container finished" podID="96ec0397-59ad-4135-a7b3-cfb4c116f779" containerID="bc1e5a76f31bf9ee8774e533977a8199f0e8f7049840b9b6847178b696559edb" exitCode=0
Jul 23 23:19:19.485394 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:19.485346    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s5vzb" event={"ID":"96ec0397-59ad-4135-a7b3-cfb4c116f779","Type":"ContainerDied","Data":"bc1e5a76f31bf9ee8774e533977a8199f0e8f7049840b9b6847178b696559edb"}
Jul 23 23:19:19.485394 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:19.485388    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s5vzb" event={"ID":"96ec0397-59ad-4135-a7b3-cfb4c116f779","Type":"ContainerStarted","Data":"10a0165219d64a7a13067cdb552248e9371cea7af5f538c784d58eb5852cd01b"}
Jul 23 23:19:19.487100 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:19.487077    2548 generic.go:356] "Generic (PLEG): container finished" podID="81d19c8e-0a24-4bdd-8d6f-4b83d1d54968" containerID="9af7248287e17c30cdefc83052bf0d595fec3d1cbab58e1be5aaf5a406da405b" exitCode=0
Jul 23 23:19:19.487179 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:19.487155    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pvffr" event={"ID":"81d19c8e-0a24-4bdd-8d6f-4b83d1d54968","Type":"ContainerDied","Data":"9af7248287e17c30cdefc83052bf0d595fec3d1cbab58e1be5aaf5a406da405b"}
Jul 23 23:19:19.487221 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:19.487195    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pvffr" event={"ID":"81d19c8e-0a24-4bdd-8d6f-4b83d1d54968","Type":"ContainerStarted","Data":"6408f188292b3e6fc1be911f36c6a129b4bc3d0841bab2a9effa654774cca92b"}
Jul 23 23:19:20.494222 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:20.494187    2548 generic.go:356] "Generic (PLEG): container finished" podID="96ec0397-59ad-4135-a7b3-cfb4c116f779" containerID="d704021573d295b4d804311590a8c713969b7d7eeb2d358ff592b448f726fffb" exitCode=0
Jul 23 23:19:20.494621 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:20.494246    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s5vzb" event={"ID":"96ec0397-59ad-4135-a7b3-cfb4c116f779","Type":"ContainerDied","Data":"d704021573d295b4d804311590a8c713969b7d7eeb2d358ff592b448f726fffb"}
Jul 23 23:19:20.496604 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:20.496583    2548 generic.go:356] "Generic (PLEG): container finished" podID="dcd3d034-aa51-460a-8274-7ca3ef0e7fd7" containerID="21b4ec48f2c41c50f034cfab2f2e7e4c9610c750130dfc59ee7e48a5679e6c70" exitCode=0
Jul 23 23:19:20.496717 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:20.496668    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fhwbc" event={"ID":"dcd3d034-aa51-460a-8274-7ca3ef0e7fd7","Type":"ContainerDied","Data":"21b4ec48f2c41c50f034cfab2f2e7e4c9610c750130dfc59ee7e48a5679e6c70"}
Jul 23 23:19:20.628386 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:20.628355    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nzckn"
Jul 23 23:19:20.658961 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:20.658935    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6j9ld\" (UniqueName: \"kubernetes.io/projected/dde480a2-3c65-4992-9070-febf7b7477a8-kube-api-access-6j9ld\") pod \"dde480a2-3c65-4992-9070-febf7b7477a8\" (UID: \"dde480a2-3c65-4992-9070-febf7b7477a8\") "
Jul 23 23:19:20.659109 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:20.659053    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dde480a2-3c65-4992-9070-febf7b7477a8-bundle\") pod \"dde480a2-3c65-4992-9070-febf7b7477a8\" (UID: \"dde480a2-3c65-4992-9070-febf7b7477a8\") "
Jul 23 23:19:20.659109 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:20.659094    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dde480a2-3c65-4992-9070-febf7b7477a8-util\") pod \"dde480a2-3c65-4992-9070-febf7b7477a8\" (UID: \"dde480a2-3c65-4992-9070-febf7b7477a8\") "
Jul 23 23:19:20.659804 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:20.659768    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dde480a2-3c65-4992-9070-febf7b7477a8-bundle" (OuterVolumeSpecName: "bundle") pod "dde480a2-3c65-4992-9070-febf7b7477a8" (UID: "dde480a2-3c65-4992-9070-febf7b7477a8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:19:20.661313 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:20.661287    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dde480a2-3c65-4992-9070-febf7b7477a8-kube-api-access-6j9ld" (OuterVolumeSpecName: "kube-api-access-6j9ld") pod "dde480a2-3c65-4992-9070-febf7b7477a8" (UID: "dde480a2-3c65-4992-9070-febf7b7477a8"). InnerVolumeSpecName "kube-api-access-6j9ld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:19:20.664734 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:20.664711    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dde480a2-3c65-4992-9070-febf7b7477a8-util" (OuterVolumeSpecName: "util") pod "dde480a2-3c65-4992-9070-febf7b7477a8" (UID: "dde480a2-3c65-4992-9070-febf7b7477a8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:19:20.759762 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:20.759733    2548 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dde480a2-3c65-4992-9070-febf7b7477a8-util\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:19:20.759861 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:20.759763    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6j9ld\" (UniqueName: \"kubernetes.io/projected/dde480a2-3c65-4992-9070-febf7b7477a8-kube-api-access-6j9ld\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:19:20.759861 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:20.759774    2548 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dde480a2-3c65-4992-9070-febf7b7477a8-bundle\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:19:21.503304 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:21.503265    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nzckn" event={"ID":"dde480a2-3c65-4992-9070-febf7b7477a8","Type":"ContainerDied","Data":"a9ce4a8a895e2d400ad2da4d1402ab76e53e383ea45ba9cea1a142235589e471"}
Jul 23 23:19:21.503750 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:21.503318    2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a9ce4a8a895e2d400ad2da4d1402ab76e53e383ea45ba9cea1a142235589e471"
Jul 23 23:19:21.503750 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:21.503283    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nzckn"
Jul 23 23:19:21.505711 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:21.505681    2548 generic.go:356] "Generic (PLEG): container finished" podID="96ec0397-59ad-4135-a7b3-cfb4c116f779" containerID="967453e39f5a356e8203725bf005726dd2250ab60db2612be9435a6e86194bbc" exitCode=0
Jul 23 23:19:21.505854 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:21.505761    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s5vzb" event={"ID":"96ec0397-59ad-4135-a7b3-cfb4c116f779","Type":"ContainerDied","Data":"967453e39f5a356e8203725bf005726dd2250ab60db2612be9435a6e86194bbc"}
Jul 23 23:19:21.507457 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:21.507432    2548 generic.go:356] "Generic (PLEG): container finished" podID="81d19c8e-0a24-4bdd-8d6f-4b83d1d54968" containerID="4cef6cd9a7308e52a7a541955a259a5c0aeb769fb7f86d965149426d23709b38" exitCode=0
Jul 23 23:19:21.507561 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:21.507523    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pvffr" event={"ID":"81d19c8e-0a24-4bdd-8d6f-4b83d1d54968","Type":"ContainerDied","Data":"4cef6cd9a7308e52a7a541955a259a5c0aeb769fb7f86d965149426d23709b38"}
Jul 23 23:19:21.642102 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:21.642079    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fhwbc"
Jul 23 23:19:21.664806 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:21.664785    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dcd3d034-aa51-460a-8274-7ca3ef0e7fd7-bundle\") pod \"dcd3d034-aa51-460a-8274-7ca3ef0e7fd7\" (UID: \"dcd3d034-aa51-460a-8274-7ca3ef0e7fd7\") "
Jul 23 23:19:21.664924 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:21.664821    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nsb2z\" (UniqueName: \"kubernetes.io/projected/dcd3d034-aa51-460a-8274-7ca3ef0e7fd7-kube-api-access-nsb2z\") pod \"dcd3d034-aa51-460a-8274-7ca3ef0e7fd7\" (UID: \"dcd3d034-aa51-460a-8274-7ca3ef0e7fd7\") "
Jul 23 23:19:21.664924 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:21.664855    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dcd3d034-aa51-460a-8274-7ca3ef0e7fd7-util\") pod \"dcd3d034-aa51-460a-8274-7ca3ef0e7fd7\" (UID: \"dcd3d034-aa51-460a-8274-7ca3ef0e7fd7\") "
Jul 23 23:19:21.665946 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:21.665743    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcd3d034-aa51-460a-8274-7ca3ef0e7fd7-bundle" (OuterVolumeSpecName: "bundle") pod "dcd3d034-aa51-460a-8274-7ca3ef0e7fd7" (UID: "dcd3d034-aa51-460a-8274-7ca3ef0e7fd7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:19:21.667410 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:21.667378    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcd3d034-aa51-460a-8274-7ca3ef0e7fd7-kube-api-access-nsb2z" (OuterVolumeSpecName: "kube-api-access-nsb2z") pod "dcd3d034-aa51-460a-8274-7ca3ef0e7fd7" (UID: "dcd3d034-aa51-460a-8274-7ca3ef0e7fd7"). InnerVolumeSpecName "kube-api-access-nsb2z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:19:21.670491 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:21.670469    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcd3d034-aa51-460a-8274-7ca3ef0e7fd7-util" (OuterVolumeSpecName: "util") pod "dcd3d034-aa51-460a-8274-7ca3ef0e7fd7" (UID: "dcd3d034-aa51-460a-8274-7ca3ef0e7fd7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:19:21.765406 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:21.765333    2548 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dcd3d034-aa51-460a-8274-7ca3ef0e7fd7-bundle\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:19:21.765406 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:21.765359    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nsb2z\" (UniqueName: \"kubernetes.io/projected/dcd3d034-aa51-460a-8274-7ca3ef0e7fd7-kube-api-access-nsb2z\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:19:21.765406 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:21.765368    2548 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dcd3d034-aa51-460a-8274-7ca3ef0e7fd7-util\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:19:22.514564 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:22.514528    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fhwbc"
Jul 23 23:19:22.514564 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:22.514550    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fhwbc" event={"ID":"dcd3d034-aa51-460a-8274-7ca3ef0e7fd7","Type":"ContainerDied","Data":"c224967e97805e23a1c3c4299f8871720d0fc6126da2e022fe4114339ec74d60"}
Jul 23 23:19:22.515073 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:22.514599    2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c224967e97805e23a1c3c4299f8871720d0fc6126da2e022fe4114339ec74d60"
Jul 23 23:19:22.516885 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:22.516854    2548 generic.go:356] "Generic (PLEG): container finished" podID="81d19c8e-0a24-4bdd-8d6f-4b83d1d54968" containerID="563ecec375b0189f3b864de954a5b34841d07148691341281979f8373cb9a388" exitCode=0
Jul 23 23:19:22.517026 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:22.516920    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pvffr" event={"ID":"81d19c8e-0a24-4bdd-8d6f-4b83d1d54968","Type":"ContainerDied","Data":"563ecec375b0189f3b864de954a5b34841d07148691341281979f8373cb9a388"}
Jul 23 23:19:22.650792 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:22.650766    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s5vzb"
Jul 23 23:19:22.670750 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:22.670722    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96ec0397-59ad-4135-a7b3-cfb4c116f779-util\") pod \"96ec0397-59ad-4135-a7b3-cfb4c116f779\" (UID: \"96ec0397-59ad-4135-a7b3-cfb4c116f779\") "
Jul 23 23:19:22.670924 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:22.670785    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96ec0397-59ad-4135-a7b3-cfb4c116f779-bundle\") pod \"96ec0397-59ad-4135-a7b3-cfb4c116f779\" (UID: \"96ec0397-59ad-4135-a7b3-cfb4c116f779\") "
Jul 23 23:19:22.670924 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:22.670819    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6h7s\" (UniqueName: \"kubernetes.io/projected/96ec0397-59ad-4135-a7b3-cfb4c116f779-kube-api-access-v6h7s\") pod \"96ec0397-59ad-4135-a7b3-cfb4c116f779\" (UID: \"96ec0397-59ad-4135-a7b3-cfb4c116f779\") "
Jul 23 23:19:22.671353 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:22.671320    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96ec0397-59ad-4135-a7b3-cfb4c116f779-bundle" (OuterVolumeSpecName: "bundle") pod "96ec0397-59ad-4135-a7b3-cfb4c116f779" (UID: "96ec0397-59ad-4135-a7b3-cfb4c116f779"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:19:22.673256 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:22.673227    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96ec0397-59ad-4135-a7b3-cfb4c116f779-kube-api-access-v6h7s" (OuterVolumeSpecName: "kube-api-access-v6h7s") pod "96ec0397-59ad-4135-a7b3-cfb4c116f779" (UID: "96ec0397-59ad-4135-a7b3-cfb4c116f779"). InnerVolumeSpecName "kube-api-access-v6h7s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:19:22.679033 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:22.679006    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96ec0397-59ad-4135-a7b3-cfb4c116f779-util" (OuterVolumeSpecName: "util") pod "96ec0397-59ad-4135-a7b3-cfb4c116f779" (UID: "96ec0397-59ad-4135-a7b3-cfb4c116f779"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:19:22.771787 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:22.771715    2548 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96ec0397-59ad-4135-a7b3-cfb4c116f779-util\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:19:22.771787 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:22.771758    2548 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96ec0397-59ad-4135-a7b3-cfb4c116f779-bundle\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:19:22.771787 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:22.771768    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6h7s\" (UniqueName: \"kubernetes.io/projected/96ec0397-59ad-4135-a7b3-cfb4c116f779-kube-api-access-v6h7s\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:19:23.523311 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:23.523284    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s5vzb"
Jul 23 23:19:23.523701 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:23.523310    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s5vzb" event={"ID":"96ec0397-59ad-4135-a7b3-cfb4c116f779","Type":"ContainerDied","Data":"10a0165219d64a7a13067cdb552248e9371cea7af5f538c784d58eb5852cd01b"}
Jul 23 23:19:23.523701 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:23.523357    2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="10a0165219d64a7a13067cdb552248e9371cea7af5f538c784d58eb5852cd01b"
Jul 23 23:19:23.653950 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:23.653922    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pvffr"
Jul 23 23:19:23.676886 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:23.676855    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-krctl\" (UniqueName: \"kubernetes.io/projected/81d19c8e-0a24-4bdd-8d6f-4b83d1d54968-kube-api-access-krctl\") pod \"81d19c8e-0a24-4bdd-8d6f-4b83d1d54968\" (UID: \"81d19c8e-0a24-4bdd-8d6f-4b83d1d54968\") "
Jul 23 23:19:23.677075 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:23.676903    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/81d19c8e-0a24-4bdd-8d6f-4b83d1d54968-util\") pod \"81d19c8e-0a24-4bdd-8d6f-4b83d1d54968\" (UID: \"81d19c8e-0a24-4bdd-8d6f-4b83d1d54968\") "
Jul 23 23:19:23.677075 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:23.676955    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/81d19c8e-0a24-4bdd-8d6f-4b83d1d54968-bundle\") pod \"81d19c8e-0a24-4bdd-8d6f-4b83d1d54968\" (UID: \"81d19c8e-0a24-4bdd-8d6f-4b83d1d54968\") "
Jul 23 23:19:23.677694 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:23.677654    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81d19c8e-0a24-4bdd-8d6f-4b83d1d54968-bundle" (OuterVolumeSpecName: "bundle") pod "81d19c8e-0a24-4bdd-8d6f-4b83d1d54968" (UID: "81d19c8e-0a24-4bdd-8d6f-4b83d1d54968"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:19:23.679399 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:23.679360    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81d19c8e-0a24-4bdd-8d6f-4b83d1d54968-kube-api-access-krctl" (OuterVolumeSpecName: "kube-api-access-krctl") pod "81d19c8e-0a24-4bdd-8d6f-4b83d1d54968" (UID: "81d19c8e-0a24-4bdd-8d6f-4b83d1d54968"). InnerVolumeSpecName "kube-api-access-krctl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:19:23.703053 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:23.702961    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81d19c8e-0a24-4bdd-8d6f-4b83d1d54968-util" (OuterVolumeSpecName: "util") pod "81d19c8e-0a24-4bdd-8d6f-4b83d1d54968" (UID: "81d19c8e-0a24-4bdd-8d6f-4b83d1d54968"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:19:23.778098 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:23.778018    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krctl\" (UniqueName: \"kubernetes.io/projected/81d19c8e-0a24-4bdd-8d6f-4b83d1d54968-kube-api-access-krctl\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:19:23.778098 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:23.778054    2548 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/81d19c8e-0a24-4bdd-8d6f-4b83d1d54968-util\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:19:23.778098 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:23.778068    2548 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/81d19c8e-0a24-4bdd-8d6f-4b83d1d54968-bundle\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:19:24.530365 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:24.530336    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pvffr"
Jul 23 23:19:24.530726 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:24.530332    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pvffr" event={"ID":"81d19c8e-0a24-4bdd-8d6f-4b83d1d54968","Type":"ContainerDied","Data":"6408f188292b3e6fc1be911f36c6a129b4bc3d0841bab2a9effa654774cca92b"}
Jul 23 23:19:24.530726 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:24.530453    2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6408f188292b3e6fc1be911f36c6a129b4bc3d0841bab2a9effa654774cca92b"
Jul 23 23:19:45.807074 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.807036    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b65d587b7-z6wc7"]
Jul 23 23:19:45.807461 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.807360    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dde480a2-3c65-4992-9070-febf7b7477a8" containerName="pull"
Jul 23 23:19:45.807461 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.807374    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="dde480a2-3c65-4992-9070-febf7b7477a8" containerName="pull"
Jul 23 23:19:45.807461 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.807384    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81d19c8e-0a24-4bdd-8d6f-4b83d1d54968" containerName="pull"
Jul 23 23:19:45.807461 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.807391    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="81d19c8e-0a24-4bdd-8d6f-4b83d1d54968" containerName="pull"
Jul 23 23:19:45.807461 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.807408    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81d19c8e-0a24-4bdd-8d6f-4b83d1d54968" containerName="util"
Jul 23 23:19:45.807461 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.807413    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="81d19c8e-0a24-4bdd-8d6f-4b83d1d54968" containerName="util"
Jul 23 23:19:45.807461 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.807421    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96ec0397-59ad-4135-a7b3-cfb4c116f779" containerName="util"
Jul 23 23:19:45.807461 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.807426    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="96ec0397-59ad-4135-a7b3-cfb4c116f779" containerName="util"
Jul 23 23:19:45.807461 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.807433    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96ec0397-59ad-4135-a7b3-cfb4c116f779" containerName="pull"
Jul 23 23:19:45.807461 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.807438    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="96ec0397-59ad-4135-a7b3-cfb4c116f779" containerName="pull"
Jul 23 23:19:45.807461 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.807454    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dde480a2-3c65-4992-9070-febf7b7477a8" containerName="util"
Jul 23 23:19:45.807461 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.807462    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="dde480a2-3c65-4992-9070-febf7b7477a8" containerName="util"
Jul 23 23:19:45.807461 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.807468    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcd3d034-aa51-460a-8274-7ca3ef0e7fd7" containerName="util"
Jul 23 23:19:45.807850 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.807473    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcd3d034-aa51-460a-8274-7ca3ef0e7fd7" containerName="util"
Jul 23 23:19:45.807850 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.807478    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96ec0397-59ad-4135-a7b3-cfb4c116f779" containerName="extract"
Jul 23 23:19:45.807850 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.807484    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="96ec0397-59ad-4135-a7b3-cfb4c116f779" containerName="extract"
Jul 23 23:19:45.807850 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.807490    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81d19c8e-0a24-4bdd-8d6f-4b83d1d54968" containerName="extract"
Jul 23 23:19:45.807850 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.807495    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="81d19c8e-0a24-4bdd-8d6f-4b83d1d54968" containerName="extract"
Jul 23 23:19:45.807850 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.807501    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcd3d034-aa51-460a-8274-7ca3ef0e7fd7" containerName="extract"
Jul 23 23:19:45.807850 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.807506    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcd3d034-aa51-460a-8274-7ca3ef0e7fd7" containerName="extract"
Jul 23 23:19:45.807850 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.807512    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dde480a2-3c65-4992-9070-febf7b7477a8" containerName="extract"
Jul 23 23:19:45.807850 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.807517    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="dde480a2-3c65-4992-9070-febf7b7477a8" containerName="extract"
Jul 23 23:19:45.807850 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.807521    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcd3d034-aa51-460a-8274-7ca3ef0e7fd7" containerName="pull"
Jul 23 23:19:45.807850 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.807528    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcd3d034-aa51-460a-8274-7ca3ef0e7fd7" containerName="pull"
Jul 23 23:19:45.807850 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.807583    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="81d19c8e-0a24-4bdd-8d6f-4b83d1d54968" containerName="extract"
Jul 23 23:19:45.807850 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.807596    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="dde480a2-3c65-4992-9070-febf7b7477a8" containerName="extract"
Jul 23 23:19:45.807850 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.807601    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="96ec0397-59ad-4135-a7b3-cfb4c116f779" containerName="extract"
Jul 23 23:19:45.807850 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.807607    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="dcd3d034-aa51-460a-8274-7ca3ef0e7fd7" containerName="extract"
Jul 23 23:19:45.810465 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.810449    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b65d587b7-z6wc7"
Jul 23 23:19:45.822208 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.822178    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b65d587b7-z6wc7"]
Jul 23 23:19:45.858937 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.858902    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1e2072ae-1c14-4027-bfda-59fab38a2621-console-serving-cert\") pod \"console-6b65d587b7-z6wc7\" (UID: \"1e2072ae-1c14-4027-bfda-59fab38a2621\") " pod="openshift-console/console-6b65d587b7-z6wc7"
Jul 23 23:19:45.858937 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.858943    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1e2072ae-1c14-4027-bfda-59fab38a2621-console-oauth-config\") pod \"console-6b65d587b7-z6wc7\" (UID: \"1e2072ae-1c14-4027-bfda-59fab38a2621\") " pod="openshift-console/console-6b65d587b7-z6wc7"
Jul 23 23:19:45.859176 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.859028    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1e2072ae-1c14-4027-bfda-59fab38a2621-service-ca\") pod \"console-6b65d587b7-z6wc7\" (UID: \"1e2072ae-1c14-4027-bfda-59fab38a2621\") " pod="openshift-console/console-6b65d587b7-z6wc7"
Jul 23 23:19:45.859176 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.859060    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wzs2\" (UniqueName: \"kubernetes.io/projected/1e2072ae-1c14-4027-bfda-59fab38a2621-kube-api-access-6wzs2\") pod \"console-6b65d587b7-z6wc7\" (UID: \"1e2072ae-1c14-4027-bfda-59fab38a2621\") " pod="openshift-console/console-6b65d587b7-z6wc7"
Jul 23 23:19:45.859176 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.859078    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1e2072ae-1c14-4027-bfda-59fab38a2621-oauth-serving-cert\") pod \"console-6b65d587b7-z6wc7\" (UID: \"1e2072ae-1c14-4027-bfda-59fab38a2621\") " pod="openshift-console/console-6b65d587b7-z6wc7"
Jul 23 23:19:45.859176 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.859112    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1e2072ae-1c14-4027-bfda-59fab38a2621-console-config\") pod \"console-6b65d587b7-z6wc7\" (UID: \"1e2072ae-1c14-4027-bfda-59fab38a2621\") " pod="openshift-console/console-6b65d587b7-z6wc7"
Jul 23 23:19:45.859176 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.859140    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e2072ae-1c14-4027-bfda-59fab38a2621-trusted-ca-bundle\") pod \"console-6b65d587b7-z6wc7\" (UID: \"1e2072ae-1c14-4027-bfda-59fab38a2621\") " pod="openshift-console/console-6b65d587b7-z6wc7"
Jul 23 23:19:45.959553 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.959520    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1e2072ae-1c14-4027-bfda-59fab38a2621-service-ca\") pod \"console-6b65d587b7-z6wc7\" (UID: \"1e2072ae-1c14-4027-bfda-59fab38a2621\") " pod="openshift-console/console-6b65d587b7-z6wc7"
Jul 23 23:19:45.959553 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.959558    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wzs2\" (UniqueName: \"kubernetes.io/projected/1e2072ae-1c14-4027-bfda-59fab38a2621-kube-api-access-6wzs2\") pod \"console-6b65d587b7-z6wc7\" (UID: \"1e2072ae-1c14-4027-bfda-59fab38a2621\") " pod="openshift-console/console-6b65d587b7-z6wc7"
Jul 23 23:19:45.959797 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.959575    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1e2072ae-1c14-4027-bfda-59fab38a2621-oauth-serving-cert\") pod \"console-6b65d587b7-z6wc7\" (UID: \"1e2072ae-1c14-4027-bfda-59fab38a2621\") " pod="openshift-console/console-6b65d587b7-z6wc7"
Jul 23 23:19:45.959797 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.959615    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1e2072ae-1c14-4027-bfda-59fab38a2621-console-config\") pod \"console-6b65d587b7-z6wc7\" (UID: \"1e2072ae-1c14-4027-bfda-59fab38a2621\") " pod="openshift-console/console-6b65d587b7-z6wc7"
Jul 23 23:19:45.959797 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.959658    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e2072ae-1c14-4027-bfda-59fab38a2621-trusted-ca-bundle\") pod \"console-6b65d587b7-z6wc7\" (UID: \"1e2072ae-1c14-4027-bfda-59fab38a2621\") " pod="openshift-console/console-6b65d587b7-z6wc7"
Jul 23 23:19:45.959797 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.959698    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1e2072ae-1c14-4027-bfda-59fab38a2621-console-serving-cert\") pod \"console-6b65d587b7-z6wc7\" (UID: \"1e2072ae-1c14-4027-bfda-59fab38a2621\") " pod="openshift-console/console-6b65d587b7-z6wc7"
Jul 23 23:19:45.959797 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.959733    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1e2072ae-1c14-4027-bfda-59fab38a2621-console-oauth-config\") pod \"console-6b65d587b7-z6wc7\" (UID: \"1e2072ae-1c14-4027-bfda-59fab38a2621\") " pod="openshift-console/console-6b65d587b7-z6wc7"
Jul 23 23:19:45.960407 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.960361    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1e2072ae-1c14-4027-bfda-59fab38a2621-oauth-serving-cert\") pod \"console-6b65d587b7-z6wc7\" (UID: \"1e2072ae-1c14-4027-bfda-59fab38a2621\") " pod="openshift-console/console-6b65d587b7-z6wc7"
Jul 23 23:19:45.960547 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.960489    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1e2072ae-1c14-4027-bfda-59fab38a2621-service-ca\") pod \"console-6b65d587b7-z6wc7\" (UID: \"1e2072ae-1c14-4027-bfda-59fab38a2621\") " pod="openshift-console/console-6b65d587b7-z6wc7"
Jul 23 23:19:45.960547 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.960522    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1e2072ae-1c14-4027-bfda-59fab38a2621-console-config\") pod \"console-6b65d587b7-z6wc7\" (UID: \"1e2072ae-1c14-4027-bfda-59fab38a2621\") " pod="openshift-console/console-6b65d587b7-z6wc7"
Jul 23 23:19:45.960630 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.960602    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e2072ae-1c14-4027-bfda-59fab38a2621-trusted-ca-bundle\") pod \"console-6b65d587b7-z6wc7\" (UID: \"1e2072ae-1c14-4027-bfda-59fab38a2621\") " pod="openshift-console/console-6b65d587b7-z6wc7"
Jul 23 23:19:45.962281 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.962259    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1e2072ae-1c14-4027-bfda-59fab38a2621-console-oauth-config\") pod \"console-6b65d587b7-z6wc7\" (UID: \"1e2072ae-1c14-4027-bfda-59fab38a2621\") " pod="openshift-console/console-6b65d587b7-z6wc7"
Jul 23 23:19:45.962424 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.962406    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1e2072ae-1c14-4027-bfda-59fab38a2621-console-serving-cert\") pod \"console-6b65d587b7-z6wc7\" (UID: \"1e2072ae-1c14-4027-bfda-59fab38a2621\") " pod="openshift-console/console-6b65d587b7-z6wc7"
Jul 23 23:19:45.968125 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:45.968094    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wzs2\" (UniqueName: \"kubernetes.io/projected/1e2072ae-1c14-4027-bfda-59fab38a2621-kube-api-access-6wzs2\") pod \"console-6b65d587b7-z6wc7\" (UID: \"1e2072ae-1c14-4027-bfda-59fab38a2621\") " pod="openshift-console/console-6b65d587b7-z6wc7"
Jul 23 23:19:46.119154 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:46.119059    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b65d587b7-z6wc7"
Jul 23 23:19:46.251548 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:46.251522    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b65d587b7-z6wc7"]
Jul 23 23:19:46.253719 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:19:46.253693    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e2072ae_1c14_4027_bfda_59fab38a2621.slice/crio-a443046fd8573d3fbe1320cffef14b678dc256d1daaecd627dc4251b56d9815d WatchSource:0}: Error finding container a443046fd8573d3fbe1320cffef14b678dc256d1daaecd627dc4251b56d9815d: Status 404 returned error can't find the container with id a443046fd8573d3fbe1320cffef14b678dc256d1daaecd627dc4251b56d9815d
Jul 23 23:19:46.633312 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:46.633276    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b65d587b7-z6wc7" event={"ID":"1e2072ae-1c14-4027-bfda-59fab38a2621","Type":"ContainerStarted","Data":"bcfdf15d992de1190222202a63c52cb5c878c0835435a28580e784b27c78e0ea"}
Jul 23 23:19:46.633481 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:46.633343    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b65d587b7-z6wc7" event={"ID":"1e2072ae-1c14-4027-bfda-59fab38a2621","Type":"ContainerStarted","Data":"a443046fd8573d3fbe1320cffef14b678dc256d1daaecd627dc4251b56d9815d"}
Jul 23 23:19:46.653295 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:46.653247    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b65d587b7-z6wc7" podStartSLOduration=1.653232773 podStartE2EDuration="1.653232773s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:19:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:19:46.651725559 +0000 UTC m=+388.395454608" watchObservedRunningTime="2026-07-23 23:19:46.653232773 +0000 UTC m=+388.396961822"
Jul 23 23:19:47.307722 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:47.307685    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-xpzb9"]
Jul 23 23:19:47.313998 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:47.312140    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xpzb9"
Jul 23 23:19:47.315619 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:47.315588    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 23 23:19:47.315768 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:47.315627    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-rtq24\""
Jul 23 23:19:47.315768 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:47.315637    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 23 23:19:47.317090 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:47.317047    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 23 23:19:47.317212 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:47.317126    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 23 23:19:47.320201 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:47.320175    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-xpzb9"]
Jul 23 23:19:47.371362 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:47.371328    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9a4cd733-8c3d-4d3f-b8d3-ec25cd8ce0e2-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-xpzb9\" (UID: \"9a4cd733-8c3d-4d3f-b8d3-ec25cd8ce0e2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xpzb9"
Jul 23 23:19:47.371538 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:47.371402    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rln7\" (UniqueName: \"kubernetes.io/projected/9a4cd733-8c3d-4d3f-b8d3-ec25cd8ce0e2-kube-api-access-4rln7\") pod \"kuadrant-console-plugin-66b5774985-xpzb9\" (UID: \"9a4cd733-8c3d-4d3f-b8d3-ec25cd8ce0e2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xpzb9"
Jul 23 23:19:47.371538 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:47.371426    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a4cd733-8c3d-4d3f-b8d3-ec25cd8ce0e2-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-xpzb9\" (UID: \"9a4cd733-8c3d-4d3f-b8d3-ec25cd8ce0e2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xpzb9"
Jul 23 23:19:47.472139 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:47.472103    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rln7\" (UniqueName: \"kubernetes.io/projected/9a4cd733-8c3d-4d3f-b8d3-ec25cd8ce0e2-kube-api-access-4rln7\") pod \"kuadrant-console-plugin-66b5774985-xpzb9\" (UID: \"9a4cd733-8c3d-4d3f-b8d3-ec25cd8ce0e2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xpzb9"
Jul 23 23:19:47.472139 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:47.472144    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a4cd733-8c3d-4d3f-b8d3-ec25cd8ce0e2-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-xpzb9\" (UID: \"9a4cd733-8c3d-4d3f-b8d3-ec25cd8ce0e2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xpzb9"
Jul 23 23:19:47.472358 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:47.472195    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9a4cd733-8c3d-4d3f-b8d3-ec25cd8ce0e2-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-xpzb9\" (UID: \"9a4cd733-8c3d-4d3f-b8d3-ec25cd8ce0e2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xpzb9"
Jul 23 23:19:47.472936 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:47.472918    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9a4cd733-8c3d-4d3f-b8d3-ec25cd8ce0e2-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-xpzb9\" (UID: \"9a4cd733-8c3d-4d3f-b8d3-ec25cd8ce0e2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xpzb9"
Jul 23 23:19:47.474577 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:47.474559    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a4cd733-8c3d-4d3f-b8d3-ec25cd8ce0e2-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-xpzb9\" (UID: \"9a4cd733-8c3d-4d3f-b8d3-ec25cd8ce0e2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xpzb9"
Jul 23 23:19:47.489872 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:47.489844    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rln7\" (UniqueName: \"kubernetes.io/projected/9a4cd733-8c3d-4d3f-b8d3-ec25cd8ce0e2-kube-api-access-4rln7\") pod \"kuadrant-console-plugin-66b5774985-xpzb9\" (UID: \"9a4cd733-8c3d-4d3f-b8d3-ec25cd8ce0e2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xpzb9"
Jul 23 23:19:47.624252 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:47.624165    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xpzb9"
Jul 23 23:19:47.758950 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:47.758907    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-xpzb9"]
Jul 23 23:19:47.761403 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:19:47.761380    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a4cd733_8c3d_4d3f_b8d3_ec25cd8ce0e2.slice/crio-fa2939d37433efee81996f75e4d7ba6835215e81f87ec1d0af526cf23efdca39 WatchSource:0}: Error finding container fa2939d37433efee81996f75e4d7ba6835215e81f87ec1d0af526cf23efdca39: Status 404 returned error can't find the container with id fa2939d37433efee81996f75e4d7ba6835215e81f87ec1d0af526cf23efdca39
Jul 23 23:19:48.648717 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:48.648669    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xpzb9" event={"ID":"9a4cd733-8c3d-4d3f-b8d3-ec25cd8ce0e2","Type":"ContainerStarted","Data":"fa2939d37433efee81996f75e4d7ba6835215e81f87ec1d0af526cf23efdca39"}
Jul 23 23:19:56.119939 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:56.119898    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b65d587b7-z6wc7"
Jul 23 23:19:56.120438 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:56.119967    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b65d587b7-z6wc7"
Jul 23 23:19:56.125494 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:56.125468    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b65d587b7-z6wc7"
Jul 23 23:19:56.702214 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:56.702185    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b65d587b7-z6wc7"
Jul 23 23:19:56.778966 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:19:56.778921    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-859d9575c6-g5jmh"]
Jul 23 23:20:09.767353 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:09.767321    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xpzb9" event={"ID":"9a4cd733-8c3d-4d3f-b8d3-ec25cd8ce0e2","Type":"ContainerStarted","Data":"a53bf33163a6c96f782a3765bd7d9c02e88ad861b32b9b8f69a1df2883b2489f"}
Jul 23 23:20:09.785427 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:09.785377    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xpzb9" podStartSLOduration=0.852352894 podStartE2EDuration="22.785361495s" totalImagesPullingTime="21.933008601s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:19:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:20:09.783387081 +0000 UTC m=+411.527116130" watchObservedRunningTime="2026-07-23 23:20:09.785361495 +0000 UTC m=+411.529090544"
Jul 23 23:20:21.801611 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:21.801553    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-859d9575c6-g5jmh" podUID="15a6e91d-e650-4d7c-a059-10b2c3b3c636" containerName="console" containerID="cri-o://161801e5b89725ada46e2384b39c0dc0d4edce6cfc2cd332548b391c190b16fb" gracePeriod=15
Jul 23 23:20:22.041644 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:22.041621    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-859d9575c6-g5jmh"
Jul 23 23:20:22.068566 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:22.068496    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15a6e91d-e650-4d7c-a059-10b2c3b3c636-oauth-serving-cert\") pod \"15a6e91d-e650-4d7c-a059-10b2c3b3c636\" (UID: \"15a6e91d-e650-4d7c-a059-10b2c3b3c636\") "
Jul 23 23:20:22.068566 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:22.068533    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15a6e91d-e650-4d7c-a059-10b2c3b3c636-trusted-ca-bundle\") pod \"15a6e91d-e650-4d7c-a059-10b2c3b3c636\" (UID: \"15a6e91d-e650-4d7c-a059-10b2c3b3c636\") "
Jul 23 23:20:22.068768 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:22.068576    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15a6e91d-e650-4d7c-a059-10b2c3b3c636-console-oauth-config\") pod \"15a6e91d-e650-4d7c-a059-10b2c3b3c636\" (UID: \"15a6e91d-e650-4d7c-a059-10b2c3b3c636\") "
Jul 23 23:20:22.068768 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:22.068608    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15a6e91d-e650-4d7c-a059-10b2c3b3c636-service-ca\") pod \"15a6e91d-e650-4d7c-a059-10b2c3b3c636\" (UID: \"15a6e91d-e650-4d7c-a059-10b2c3b3c636\") "
Jul 23 23:20:22.068768 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:22.068630    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15a6e91d-e650-4d7c-a059-10b2c3b3c636-console-config\") pod \"15a6e91d-e650-4d7c-a059-10b2c3b3c636\" (UID: \"15a6e91d-e650-4d7c-a059-10b2c3b3c636\") "
Jul 23 23:20:22.068768 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:22.068735    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15a6e91d-e650-4d7c-a059-10b2c3b3c636-console-serving-cert\") pod \"15a6e91d-e650-4d7c-a059-10b2c3b3c636\" (UID: \"15a6e91d-e650-4d7c-a059-10b2c3b3c636\") "
Jul 23 23:20:22.068970 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:22.068770    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cw2f5\" (UniqueName: \"kubernetes.io/projected/15a6e91d-e650-4d7c-a059-10b2c3b3c636-kube-api-access-cw2f5\") pod \"15a6e91d-e650-4d7c-a059-10b2c3b3c636\" (UID: \"15a6e91d-e650-4d7c-a059-10b2c3b3c636\") "
Jul 23 23:20:22.069046 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:22.069020    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15a6e91d-e650-4d7c-a059-10b2c3b3c636-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "15a6e91d-e650-4d7c-a059-10b2c3b3c636" (UID: "15a6e91d-e650-4d7c-a059-10b2c3b3c636"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:20:22.069220 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:22.069196    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15a6e91d-e650-4d7c-a059-10b2c3b3c636-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "15a6e91d-e650-4d7c-a059-10b2c3b3c636" (UID: "15a6e91d-e650-4d7c-a059-10b2c3b3c636"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:20:22.069287 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:22.069213    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15a6e91d-e650-4d7c-a059-10b2c3b3c636-console-config" (OuterVolumeSpecName: "console-config") pod "15a6e91d-e650-4d7c-a059-10b2c3b3c636" (UID: "15a6e91d-e650-4d7c-a059-10b2c3b3c636"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:20:22.069287 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:22.069203    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15a6e91d-e650-4d7c-a059-10b2c3b3c636-service-ca" (OuterVolumeSpecName: "service-ca") pod "15a6e91d-e650-4d7c-a059-10b2c3b3c636" (UID: "15a6e91d-e650-4d7c-a059-10b2c3b3c636"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:20:22.071242 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:22.071215    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15a6e91d-e650-4d7c-a059-10b2c3b3c636-kube-api-access-cw2f5" (OuterVolumeSpecName: "kube-api-access-cw2f5") pod "15a6e91d-e650-4d7c-a059-10b2c3b3c636" (UID: "15a6e91d-e650-4d7c-a059-10b2c3b3c636"). InnerVolumeSpecName "kube-api-access-cw2f5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:20:22.071341 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:22.071267    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15a6e91d-e650-4d7c-a059-10b2c3b3c636-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "15a6e91d-e650-4d7c-a059-10b2c3b3c636" (UID: "15a6e91d-e650-4d7c-a059-10b2c3b3c636"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:20:22.071341 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:22.071294    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15a6e91d-e650-4d7c-a059-10b2c3b3c636-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "15a6e91d-e650-4d7c-a059-10b2c3b3c636" (UID: "15a6e91d-e650-4d7c-a059-10b2c3b3c636"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:20:22.169481 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:22.169452    2548 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15a6e91d-e650-4d7c-a059-10b2c3b3c636-oauth-serving-cert\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:20:22.169481 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:22.169479    2548 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15a6e91d-e650-4d7c-a059-10b2c3b3c636-trusted-ca-bundle\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:20:22.169680 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:22.169493    2548 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15a6e91d-e650-4d7c-a059-10b2c3b3c636-console-oauth-config\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:20:22.169680 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:22.169506    2548 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15a6e91d-e650-4d7c-a059-10b2c3b3c636-service-ca\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:20:22.169680 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:22.169519    2548 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15a6e91d-e650-4d7c-a059-10b2c3b3c636-console-config\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:20:22.169680 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:22.169529    2548 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15a6e91d-e650-4d7c-a059-10b2c3b3c636-console-serving-cert\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:20:22.169680 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:22.169542    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cw2f5\" (UniqueName: \"kubernetes.io/projected/15a6e91d-e650-4d7c-a059-10b2c3b3c636-kube-api-access-cw2f5\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:20:22.830122 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:22.830088    2548 generic.go:356] "Generic (PLEG): container finished" podID="15a6e91d-e650-4d7c-a059-10b2c3b3c636" containerID="161801e5b89725ada46e2384b39c0dc0d4edce6cfc2cd332548b391c190b16fb" exitCode=0
Jul 23 23:20:22.830501 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:22.830126    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-859d9575c6-g5jmh"
Jul 23 23:20:22.830501 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:22.830172    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-859d9575c6-g5jmh" event={"ID":"15a6e91d-e650-4d7c-a059-10b2c3b3c636","Type":"ContainerDied","Data":"161801e5b89725ada46e2384b39c0dc0d4edce6cfc2cd332548b391c190b16fb"}
Jul 23 23:20:22.830501 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:22.830220    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-859d9575c6-g5jmh" event={"ID":"15a6e91d-e650-4d7c-a059-10b2c3b3c636","Type":"ContainerDied","Data":"d6bc2f28d2e630dfc86831a0d6312511d907c9bd892fce814951998d13ac7850"}
Jul 23 23:20:22.830501 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:22.830243    2548 scope.go:117] "RemoveContainer" containerID="161801e5b89725ada46e2384b39c0dc0d4edce6cfc2cd332548b391c190b16fb"
Jul 23 23:20:22.838775 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:22.838757    2548 scope.go:117] "RemoveContainer" containerID="161801e5b89725ada46e2384b39c0dc0d4edce6cfc2cd332548b391c190b16fb"
Jul 23 23:20:22.844673 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:20:22.844644    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"161801e5b89725ada46e2384b39c0dc0d4edce6cfc2cd332548b391c190b16fb\": container with ID starting with 161801e5b89725ada46e2384b39c0dc0d4edce6cfc2cd332548b391c190b16fb not found: ID does not exist" containerID="161801e5b89725ada46e2384b39c0dc0d4edce6cfc2cd332548b391c190b16fb"
Jul 23 23:20:22.844787 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:22.844679    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"161801e5b89725ada46e2384b39c0dc0d4edce6cfc2cd332548b391c190b16fb"} err="failed to get container status \"161801e5b89725ada46e2384b39c0dc0d4edce6cfc2cd332548b391c190b16fb\": rpc error: code = NotFound desc = could not find container \"161801e5b89725ada46e2384b39c0dc0d4edce6cfc2cd332548b391c190b16fb\": container with ID starting with 161801e5b89725ada46e2384b39c0dc0d4edce6cfc2cd332548b391c190b16fb not found: ID does not exist"
Jul 23 23:20:22.850879 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:22.850852    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-859d9575c6-g5jmh"]
Jul 23 23:20:22.857769 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:22.857745    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-859d9575c6-g5jmh"]
Jul 23 23:20:24.747271 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:24.747239    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15a6e91d-e650-4d7c-a059-10b2c3b3c636" path="/var/lib/kubelet/pods/15a6e91d-e650-4d7c-a059-10b2c3b3c636/volumes"
Jul 23 23:20:28.731736 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:28.731705    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:20:28.732204 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:28.732027    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15a6e91d-e650-4d7c-a059-10b2c3b3c636" containerName="console"
Jul 23 23:20:28.732204 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:28.732038    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="15a6e91d-e650-4d7c-a059-10b2c3b3c636" containerName="console"
Jul 23 23:20:28.732204 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:28.732095    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="15a6e91d-e650-4d7c-a059-10b2c3b3c636" containerName="console"
Jul 23 23:20:28.945864 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:28.945833    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-bshbw"
Jul 23 23:20:28.951944 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:28.951914    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 23 23:20:28.955428 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:28.955405    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:20:28.955550 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:28.955457    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:20:29.017532 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:29.017447    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rp9hk\" (UniqueName: \"kubernetes.io/projected/94abf77d-d985-4e91-aba8-937179a84c97-kube-api-access-rp9hk\") pod \"limitador-limitador-54d6d988d5-bshbw\" (UID: \"94abf77d-d985-4e91-aba8-937179a84c97\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-bshbw"
Jul 23 23:20:29.017532 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:29.017490    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/94abf77d-d985-4e91-aba8-937179a84c97-config-file\") pod \"limitador-limitador-54d6d988d5-bshbw\" (UID: \"94abf77d-d985-4e91-aba8-937179a84c97\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-bshbw"
Jul 23 23:20:29.021389 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:29.021362    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-x4qfz"]
Jul 23 23:20:29.036855 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:29.036828    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-x4qfz"]
Jul 23 23:20:29.037009 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:29.036921    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-x4qfz"
Jul 23 23:20:29.039577 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:29.039552    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-t48pg\""
Jul 23 23:20:29.118689 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:29.118655    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lhbm\" (UniqueName: \"kubernetes.io/projected/14136b84-92aa-4cff-a23e-a73e0938a0c9-kube-api-access-2lhbm\") pod \"authorino-58cd955675-x4qfz\" (UID: \"14136b84-92aa-4cff-a23e-a73e0938a0c9\") " pod="kuadrant-system/authorino-58cd955675-x4qfz"
Jul 23 23:20:29.118887 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:29.118743    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rp9hk\" (UniqueName: \"kubernetes.io/projected/94abf77d-d985-4e91-aba8-937179a84c97-kube-api-access-rp9hk\") pod \"limitador-limitador-54d6d988d5-bshbw\" (UID: \"94abf77d-d985-4e91-aba8-937179a84c97\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-bshbw"
Jul 23 23:20:29.118887 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:29.118784    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/94abf77d-d985-4e91-aba8-937179a84c97-config-file\") pod \"limitador-limitador-54d6d988d5-bshbw\" (UID: \"94abf77d-d985-4e91-aba8-937179a84c97\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-bshbw"
Jul 23 23:20:29.119455 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:29.119431    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/94abf77d-d985-4e91-aba8-937179a84c97-config-file\") pod \"limitador-limitador-54d6d988d5-bshbw\" (UID: \"94abf77d-d985-4e91-aba8-937179a84c97\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-bshbw"
Jul 23 23:20:29.126656 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:29.126631    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rp9hk\" (UniqueName: \"kubernetes.io/projected/94abf77d-d985-4e91-aba8-937179a84c97-kube-api-access-rp9hk\") pod \"limitador-limitador-54d6d988d5-bshbw\" (UID: \"94abf77d-d985-4e91-aba8-937179a84c97\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-bshbw"
Jul 23 23:20:29.219393 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:29.219360    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lhbm\" (UniqueName: \"kubernetes.io/projected/14136b84-92aa-4cff-a23e-a73e0938a0c9-kube-api-access-2lhbm\") pod \"authorino-58cd955675-x4qfz\" (UID: \"14136b84-92aa-4cff-a23e-a73e0938a0c9\") " pod="kuadrant-system/authorino-58cd955675-x4qfz"
Jul 23 23:20:29.228219 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:29.228193    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lhbm\" (UniqueName: \"kubernetes.io/projected/14136b84-92aa-4cff-a23e-a73e0938a0c9-kube-api-access-2lhbm\") pod \"authorino-58cd955675-x4qfz\" (UID: \"14136b84-92aa-4cff-a23e-a73e0938a0c9\") " pod="kuadrant-system/authorino-58cd955675-x4qfz"
Jul 23 23:20:29.242351 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:29.242322    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-tn58t"]
Jul 23 23:20:29.262501 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:29.262471    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-bshbw"
Jul 23 23:20:29.270037 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:29.269968    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-tn58t"]
Jul 23 23:20:29.270135 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:29.270090    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-tn58t"
Jul 23 23:20:29.320813 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:29.320765    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8g5l4\" (UniqueName: \"kubernetes.io/projected/0863f7d9-0b86-4536-b02a-f9feb30134b8-kube-api-access-8g5l4\") pod \"authorino-75d556bd77-tn58t\" (UID: \"0863f7d9-0b86-4536-b02a-f9feb30134b8\") " pod="kuadrant-system/authorino-75d556bd77-tn58t"
Jul 23 23:20:29.346603 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:29.346571    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-x4qfz"
Jul 23 23:20:29.411886 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:29.411856    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:20:29.413942 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:20:29.413914    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94abf77d_d985_4e91_aba8_937179a84c97.slice/crio-cd105200755c47b21c54105724be918869161c59270237df714e813e3cfa2623 WatchSource:0}: Error finding container cd105200755c47b21c54105724be918869161c59270237df714e813e3cfa2623: Status 404 returned error can't find the container with id cd105200755c47b21c54105724be918869161c59270237df714e813e3cfa2623
Jul 23 23:20:29.422292 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:29.422258    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8g5l4\" (UniqueName: \"kubernetes.io/projected/0863f7d9-0b86-4536-b02a-f9feb30134b8-kube-api-access-8g5l4\") pod \"authorino-75d556bd77-tn58t\" (UID: \"0863f7d9-0b86-4536-b02a-f9feb30134b8\") " pod="kuadrant-system/authorino-75d556bd77-tn58t"
Jul 23 23:20:29.431053 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:29.431022    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8g5l4\" (UniqueName: \"kubernetes.io/projected/0863f7d9-0b86-4536-b02a-f9feb30134b8-kube-api-access-8g5l4\") pod \"authorino-75d556bd77-tn58t\" (UID: \"0863f7d9-0b86-4536-b02a-f9feb30134b8\") " pod="kuadrant-system/authorino-75d556bd77-tn58t"
Jul 23 23:20:29.498095 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:29.498071    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-x4qfz"]
Jul 23 23:20:29.499534 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:20:29.499511    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod14136b84_92aa_4cff_a23e_a73e0938a0c9.slice/crio-93c5e221c21e524f9bbccb603ca5034c02b5440114f3224f112f01652c0476de WatchSource:0}: Error finding container 93c5e221c21e524f9bbccb603ca5034c02b5440114f3224f112f01652c0476de: Status 404 returned error can't find the container with id 93c5e221c21e524f9bbccb603ca5034c02b5440114f3224f112f01652c0476de
Jul 23 23:20:29.579181 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:29.579087    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-tn58t"
Jul 23 23:20:29.730749 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:29.730726    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-tn58t"]
Jul 23 23:20:29.732438 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:20:29.732410    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0863f7d9_0b86_4536_b02a_f9feb30134b8.slice/crio-e6d6ece1c5792982a187aab63259edf941e24cdc81bf68d160733ad183c69561 WatchSource:0}: Error finding container e6d6ece1c5792982a187aab63259edf941e24cdc81bf68d160733ad183c69561: Status 404 returned error can't find the container with id e6d6ece1c5792982a187aab63259edf941e24cdc81bf68d160733ad183c69561
Jul 23 23:20:29.865125 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:29.865049    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-tn58t" event={"ID":"0863f7d9-0b86-4536-b02a-f9feb30134b8","Type":"ContainerStarted","Data":"e6d6ece1c5792982a187aab63259edf941e24cdc81bf68d160733ad183c69561"}
Jul 23 23:20:29.866309 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:29.866278    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-bshbw" event={"ID":"94abf77d-d985-4e91-aba8-937179a84c97","Type":"ContainerStarted","Data":"cd105200755c47b21c54105724be918869161c59270237df714e813e3cfa2623"}
Jul 23 23:20:29.867337 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:29.867315    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-x4qfz" event={"ID":"14136b84-92aa-4cff-a23e-a73e0938a0c9","Type":"ContainerStarted","Data":"93c5e221c21e524f9bbccb603ca5034c02b5440114f3224f112f01652c0476de"}
Jul 23 23:20:36.912580 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:36.912535    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-bshbw" event={"ID":"94abf77d-d985-4e91-aba8-937179a84c97","Type":"ContainerStarted","Data":"e28b283a4d93fbb4f1438e91ae2a000eeee1940690c0b6ee3802948770e5074d"}
Jul 23 23:20:36.913072 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:36.912704    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-bshbw"
Jul 23 23:20:36.914109 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:36.914073    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-x4qfz" event={"ID":"14136b84-92aa-4cff-a23e-a73e0938a0c9","Type":"ContainerStarted","Data":"c20d2d11a1880c1909d79e03bc8bbc3accb2c5187ffa3ca2a00080d634caa7c3"}
Jul 23 23:20:36.915512 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:36.915489    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-tn58t" event={"ID":"0863f7d9-0b86-4536-b02a-f9feb30134b8","Type":"ContainerStarted","Data":"e87c0676f19788dd0b5f3a8c32849edc5c32cb8484b428019c151629876a2015"}
Jul 23 23:20:36.934630 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:36.934575    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-bshbw" podStartSLOduration=2.285748833 podStartE2EDuration="8.934555978s" totalImagesPullingTime="6.648807145s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:20:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:20:36.931753534 +0000 UTC m=+438.675482598" watchObservedRunningTime="2026-07-23 23:20:36.934555978 +0000 UTC m=+438.678285028"
Jul 23 23:20:36.946311 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:36.946259    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-x4qfz" podStartSLOduration=1.394804036 podStartE2EDuration="7.946242129s" totalImagesPullingTime="6.551438093s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:20:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:20:36.945809635 +0000 UTC m=+438.689538709" watchObservedRunningTime="2026-07-23 23:20:36.946242129 +0000 UTC m=+438.689971236"
Jul 23 23:20:36.959831 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:36.959774    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-tn58t" podStartSLOduration=1.641100449 podStartE2EDuration="7.959754047s" totalImagesPullingTime="6.318653598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:20:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:20:36.958913646 +0000 UTC m=+438.702642696" watchObservedRunningTime="2026-07-23 23:20:36.959754047 +0000 UTC m=+438.703483098"
Jul 23 23:20:36.987839 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:36.987807    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-x4qfz"]
Jul 23 23:20:38.925881 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:38.925816    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-x4qfz" podUID="14136b84-92aa-4cff-a23e-a73e0938a0c9" containerName="authorino" containerID="cri-o://c20d2d11a1880c1909d79e03bc8bbc3accb2c5187ffa3ca2a00080d634caa7c3" gracePeriod=30
Jul 23 23:20:39.167947 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:39.167918    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-x4qfz"
Jul 23 23:20:39.214168 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:39.214089    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2lhbm\" (UniqueName: \"kubernetes.io/projected/14136b84-92aa-4cff-a23e-a73e0938a0c9-kube-api-access-2lhbm\") pod \"14136b84-92aa-4cff-a23e-a73e0938a0c9\" (UID: \"14136b84-92aa-4cff-a23e-a73e0938a0c9\") "
Jul 23 23:20:39.216111 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:39.216084    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14136b84-92aa-4cff-a23e-a73e0938a0c9-kube-api-access-2lhbm" (OuterVolumeSpecName: "kube-api-access-2lhbm") pod "14136b84-92aa-4cff-a23e-a73e0938a0c9" (UID: "14136b84-92aa-4cff-a23e-a73e0938a0c9"). InnerVolumeSpecName "kube-api-access-2lhbm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:20:39.315604 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:39.315564    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2lhbm\" (UniqueName: \"kubernetes.io/projected/14136b84-92aa-4cff-a23e-a73e0938a0c9-kube-api-access-2lhbm\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:20:39.935950 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:39.935915    2548 generic.go:356] "Generic (PLEG): container finished" podID="14136b84-92aa-4cff-a23e-a73e0938a0c9" containerID="c20d2d11a1880c1909d79e03bc8bbc3accb2c5187ffa3ca2a00080d634caa7c3" exitCode=0
Jul 23 23:20:39.936373 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:39.936003    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-x4qfz"
Jul 23 23:20:39.936373 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:39.936001    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-x4qfz" event={"ID":"14136b84-92aa-4cff-a23e-a73e0938a0c9","Type":"ContainerDied","Data":"c20d2d11a1880c1909d79e03bc8bbc3accb2c5187ffa3ca2a00080d634caa7c3"}
Jul 23 23:20:39.936373 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:39.936118    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-x4qfz" event={"ID":"14136b84-92aa-4cff-a23e-a73e0938a0c9","Type":"ContainerDied","Data":"93c5e221c21e524f9bbccb603ca5034c02b5440114f3224f112f01652c0476de"}
Jul 23 23:20:39.936373 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:39.936136    2548 scope.go:117] "RemoveContainer" containerID="c20d2d11a1880c1909d79e03bc8bbc3accb2c5187ffa3ca2a00080d634caa7c3"
Jul 23 23:20:39.945119 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:39.945090    2548 scope.go:117] "RemoveContainer" containerID="c20d2d11a1880c1909d79e03bc8bbc3accb2c5187ffa3ca2a00080d634caa7c3"
Jul 23 23:20:39.945489 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:20:39.945453    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c20d2d11a1880c1909d79e03bc8bbc3accb2c5187ffa3ca2a00080d634caa7c3\": container with ID starting with c20d2d11a1880c1909d79e03bc8bbc3accb2c5187ffa3ca2a00080d634caa7c3 not found: ID does not exist" containerID="c20d2d11a1880c1909d79e03bc8bbc3accb2c5187ffa3ca2a00080d634caa7c3"
Jul 23 23:20:39.945585 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:39.945515    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c20d2d11a1880c1909d79e03bc8bbc3accb2c5187ffa3ca2a00080d634caa7c3"} err="failed to get container status \"c20d2d11a1880c1909d79e03bc8bbc3accb2c5187ffa3ca2a00080d634caa7c3\": rpc error: code = NotFound desc = could not find container \"c20d2d11a1880c1909d79e03bc8bbc3accb2c5187ffa3ca2a00080d634caa7c3\": container with ID starting with c20d2d11a1880c1909d79e03bc8bbc3accb2c5187ffa3ca2a00080d634caa7c3 not found: ID does not exist"
Jul 23 23:20:39.957821 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:39.957795    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-x4qfz"]
Jul 23 23:20:39.962996 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:39.962960    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-x4qfz"]
Jul 23 23:20:40.747381 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:40.747284    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14136b84-92aa-4cff-a23e-a73e0938a0c9" path="/var/lib/kubelet/pods/14136b84-92aa-4cff-a23e-a73e0938a0c9/volumes"
Jul 23 23:20:47.923420 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:20:47.923376    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-bshbw"
Jul 23 23:21:01.976162 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:01.976124    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-kbgn5"]
Jul 23 23:21:01.976671 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:01.976450    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14136b84-92aa-4cff-a23e-a73e0938a0c9" containerName="authorino"
Jul 23 23:21:01.976671 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:01.976465    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="14136b84-92aa-4cff-a23e-a73e0938a0c9" containerName="authorino"
Jul 23 23:21:01.976671 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:01.976537    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="14136b84-92aa-4cff-a23e-a73e0938a0c9" containerName="authorino"
Jul 23 23:21:01.981896 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:01.981869    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-kbgn5"
Jul 23 23:21:01.983454 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:01.983426    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-kbgn5"]
Jul 23 23:21:02.113959 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:02.113917    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbldl\" (UniqueName: \"kubernetes.io/projected/37b85497-cb89-4a64-aef1-83f552ab207f-kube-api-access-xbldl\") pod \"authorino-5d4b67c446-kbgn5\" (UID: \"37b85497-cb89-4a64-aef1-83f552ab207f\") " pod="kuadrant-system/authorino-5d4b67c446-kbgn5"
Jul 23 23:21:02.212918 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:02.212881    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-kbgn5"]
Jul 23 23:21:02.213176 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:21:02.213156    2548 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-xbldl], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-kbgn5" podUID="37b85497-cb89-4a64-aef1-83f552ab207f"
Jul 23 23:21:02.215041 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:02.215008    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbldl\" (UniqueName: \"kubernetes.io/projected/37b85497-cb89-4a64-aef1-83f552ab207f-kube-api-access-xbldl\") pod \"authorino-5d4b67c446-kbgn5\" (UID: \"37b85497-cb89-4a64-aef1-83f552ab207f\") " pod="kuadrant-system/authorino-5d4b67c446-kbgn5"
Jul 23 23:21:02.223146 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:02.223118    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbldl\" (UniqueName: \"kubernetes.io/projected/37b85497-cb89-4a64-aef1-83f552ab207f-kube-api-access-xbldl\") pod \"authorino-5d4b67c446-kbgn5\" (UID: \"37b85497-cb89-4a64-aef1-83f552ab207f\") " pod="kuadrant-system/authorino-5d4b67c446-kbgn5"
Jul 23 23:21:02.238047 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:02.237961    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5f4c48574c-p6nzj"]
Jul 23 23:21:02.242237 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:02.242216    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5f4c48574c-p6nzj"
Jul 23 23:21:02.247737 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:02.247708    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5f4c48574c-p6nzj"]
Jul 23 23:21:02.315861 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:02.315824    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvxfs\" (UniqueName: \"kubernetes.io/projected/1b9d77a4-ad65-4b2c-a3db-f0e0cd210ba9-kube-api-access-fvxfs\") pod \"authorino-5f4c48574c-p6nzj\" (UID: \"1b9d77a4-ad65-4b2c-a3db-f0e0cd210ba9\") " pod="kuadrant-system/authorino-5f4c48574c-p6nzj"
Jul 23 23:21:02.417137 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:02.417093    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvxfs\" (UniqueName: \"kubernetes.io/projected/1b9d77a4-ad65-4b2c-a3db-f0e0cd210ba9-kube-api-access-fvxfs\") pod \"authorino-5f4c48574c-p6nzj\" (UID: \"1b9d77a4-ad65-4b2c-a3db-f0e0cd210ba9\") " pod="kuadrant-system/authorino-5f4c48574c-p6nzj"
Jul 23 23:21:02.425638 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:02.425601    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvxfs\" (UniqueName: \"kubernetes.io/projected/1b9d77a4-ad65-4b2c-a3db-f0e0cd210ba9-kube-api-access-fvxfs\") pod \"authorino-5f4c48574c-p6nzj\" (UID: \"1b9d77a4-ad65-4b2c-a3db-f0e0cd210ba9\") " pod="kuadrant-system/authorino-5f4c48574c-p6nzj"
Jul 23 23:21:02.552872 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:02.552786    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5f4c48574c-p6nzj"
Jul 23 23:21:02.686706 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:02.686683    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5f4c48574c-p6nzj"]
Jul 23 23:21:02.688513 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:21:02.688487    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1b9d77a4_ad65_4b2c_a3db_f0e0cd210ba9.slice/crio-8223a151f35465fc6d4d996263a4c62c23308633ef0a644ea6955b908776899e WatchSource:0}: Error finding container 8223a151f35465fc6d4d996263a4c62c23308633ef0a644ea6955b908776899e: Status 404 returned error can't find the container with id 8223a151f35465fc6d4d996263a4c62c23308633ef0a644ea6955b908776899e
Jul 23 23:21:02.815636 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:02.815548    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5f4c48574c-p6nzj"]
Jul 23 23:21:02.843198 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:02.843158    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7947b97487-bbg5w"]
Jul 23 23:21:02.849400 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:02.849374    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7947b97487-bbg5w"
Jul 23 23:21:02.851822 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:02.851791    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 23 23:21:02.853252 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:02.853230    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7947b97487-bbg5w"]
Jul 23 23:21:03.023072 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:03.023038    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c9cac6f8-2139-4258-988c-db13febd1a0e-tls-cert\") pod \"authorino-7947b97487-bbg5w\" (UID: \"c9cac6f8-2139-4258-988c-db13febd1a0e\") " pod="kuadrant-system/authorino-7947b97487-bbg5w"
Jul 23 23:21:03.023475 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:03.023166    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9v4nw\" (UniqueName: \"kubernetes.io/projected/c9cac6f8-2139-4258-988c-db13febd1a0e-kube-api-access-9v4nw\") pod \"authorino-7947b97487-bbg5w\" (UID: \"c9cac6f8-2139-4258-988c-db13febd1a0e\") " pod="kuadrant-system/authorino-7947b97487-bbg5w"
Jul 23 23:21:03.050911 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:03.050876    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5f4c48574c-p6nzj" event={"ID":"1b9d77a4-ad65-4b2c-a3db-f0e0cd210ba9","Type":"ContainerStarted","Data":"8223a151f35465fc6d4d996263a4c62c23308633ef0a644ea6955b908776899e"}
Jul 23 23:21:03.050911 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:03.050900    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-kbgn5"
Jul 23 23:21:03.070637 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:03.070574    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-kbgn5"
Jul 23 23:21:03.124191 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:03.124158    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v4nw\" (UniqueName: \"kubernetes.io/projected/c9cac6f8-2139-4258-988c-db13febd1a0e-kube-api-access-9v4nw\") pod \"authorino-7947b97487-bbg5w\" (UID: \"c9cac6f8-2139-4258-988c-db13febd1a0e\") " pod="kuadrant-system/authorino-7947b97487-bbg5w"
Jul 23 23:21:03.124350 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:03.124206    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c9cac6f8-2139-4258-988c-db13febd1a0e-tls-cert\") pod \"authorino-7947b97487-bbg5w\" (UID: \"c9cac6f8-2139-4258-988c-db13febd1a0e\") " pod="kuadrant-system/authorino-7947b97487-bbg5w"
Jul 23 23:21:03.126744 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:03.126719    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c9cac6f8-2139-4258-988c-db13febd1a0e-tls-cert\") pod \"authorino-7947b97487-bbg5w\" (UID: \"c9cac6f8-2139-4258-988c-db13febd1a0e\") " pod="kuadrant-system/authorino-7947b97487-bbg5w"
Jul 23 23:21:03.132940 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:03.132910    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9v4nw\" (UniqueName: \"kubernetes.io/projected/c9cac6f8-2139-4258-988c-db13febd1a0e-kube-api-access-9v4nw\") pod \"authorino-7947b97487-bbg5w\" (UID: \"c9cac6f8-2139-4258-988c-db13febd1a0e\") " pod="kuadrant-system/authorino-7947b97487-bbg5w"
Jul 23 23:21:03.159831 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:03.159790    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7947b97487-bbg5w"
Jul 23 23:21:03.225519 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:03.225487    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xbldl\" (UniqueName: \"kubernetes.io/projected/37b85497-cb89-4a64-aef1-83f552ab207f-kube-api-access-xbldl\") pod \"37b85497-cb89-4a64-aef1-83f552ab207f\" (UID: \"37b85497-cb89-4a64-aef1-83f552ab207f\") "
Jul 23 23:21:03.227661 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:03.227628    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37b85497-cb89-4a64-aef1-83f552ab207f-kube-api-access-xbldl" (OuterVolumeSpecName: "kube-api-access-xbldl") pod "37b85497-cb89-4a64-aef1-83f552ab207f" (UID: "37b85497-cb89-4a64-aef1-83f552ab207f"). InnerVolumeSpecName "kube-api-access-xbldl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:21:03.292495 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:03.292375    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7947b97487-bbg5w"]
Jul 23 23:21:03.295148 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:21:03.295108    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc9cac6f8_2139_4258_988c_db13febd1a0e.slice/crio-bed2df12d18d765ed455d2ef1f56aa82792e283a9958a40023027cbd53d5e973 WatchSource:0}: Error finding container bed2df12d18d765ed455d2ef1f56aa82792e283a9958a40023027cbd53d5e973: Status 404 returned error can't find the container with id bed2df12d18d765ed455d2ef1f56aa82792e283a9958a40023027cbd53d5e973
Jul 23 23:21:03.326409 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:03.326326    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xbldl\" (UniqueName: \"kubernetes.io/projected/37b85497-cb89-4a64-aef1-83f552ab207f-kube-api-access-xbldl\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:21:04.056919 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:04.056876    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5f4c48574c-p6nzj" event={"ID":"1b9d77a4-ad65-4b2c-a3db-f0e0cd210ba9","Type":"ContainerStarted","Data":"608c49e6e40e0cb8a1b24843e0ae55bc96ab29a5457c0260ef39c414a9b06ab5"}
Jul 23 23:21:04.056919 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:04.056890    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5f4c48574c-p6nzj" podUID="1b9d77a4-ad65-4b2c-a3db-f0e0cd210ba9" containerName="authorino" containerID="cri-o://608c49e6e40e0cb8a1b24843e0ae55bc96ab29a5457c0260ef39c414a9b06ab5" gracePeriod=30
Jul 23 23:21:04.058618 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:04.058594    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7947b97487-bbg5w" event={"ID":"c9cac6f8-2139-4258-988c-db13febd1a0e","Type":"ContainerStarted","Data":"0500d3545da051ce02202536f9ab389a3ec3bac11e501d18160a8e6c1fc2e04c"}
Jul 23 23:21:04.058726 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:04.058621    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-kbgn5"
Jul 23 23:21:04.058726 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:04.058628    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7947b97487-bbg5w" event={"ID":"c9cac6f8-2139-4258-988c-db13febd1a0e","Type":"ContainerStarted","Data":"bed2df12d18d765ed455d2ef1f56aa82792e283a9958a40023027cbd53d5e973"}
Jul 23 23:21:04.072872 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:04.072830    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5f4c48574c-p6nzj" podStartSLOduration=1.7339986029999999 podStartE2EDuration="2.072816263s" totalImagesPullingTime="338.81766ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:21:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:21:04.071093168 +0000 UTC m=+465.814822240" watchObservedRunningTime="2026-07-23 23:21:04.072816263 +0000 UTC m=+465.816545312"
Jul 23 23:21:04.092968 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:04.092939    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-kbgn5"]
Jul 23 23:21:04.096493 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:04.096468    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-kbgn5"]
Jul 23 23:21:04.108789 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:04.108740    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7947b97487-bbg5w" podStartSLOduration=1.783700054 podStartE2EDuration="2.108725253s" totalImagesPullingTime="325.025199ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:21:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:21:04.107229704 +0000 UTC m=+465.850958761" watchObservedRunningTime="2026-07-23 23:21:04.108725253 +0000 UTC m=+465.852454301"
Jul 23 23:21:04.131232 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:04.131201    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-tn58t"]
Jul 23 23:21:04.131389 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:04.131367    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-tn58t" podUID="0863f7d9-0b86-4536-b02a-f9feb30134b8" containerName="authorino" containerID="cri-o://e87c0676f19788dd0b5f3a8c32849edc5c32cb8484b428019c151629876a2015" gracePeriod=30
Jul 23 23:21:04.390452 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:04.390427    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5f4c48574c-p6nzj"
Jul 23 23:21:04.393082 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:04.393062    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-tn58t"
Jul 23 23:21:04.534898 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:04.534861    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fvxfs\" (UniqueName: \"kubernetes.io/projected/1b9d77a4-ad65-4b2c-a3db-f0e0cd210ba9-kube-api-access-fvxfs\") pod \"1b9d77a4-ad65-4b2c-a3db-f0e0cd210ba9\" (UID: \"1b9d77a4-ad65-4b2c-a3db-f0e0cd210ba9\") "
Jul 23 23:21:04.535113 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:04.534906    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8g5l4\" (UniqueName: \"kubernetes.io/projected/0863f7d9-0b86-4536-b02a-f9feb30134b8-kube-api-access-8g5l4\") pod \"0863f7d9-0b86-4536-b02a-f9feb30134b8\" (UID: \"0863f7d9-0b86-4536-b02a-f9feb30134b8\") "
Jul 23 23:21:04.536989 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:04.536948    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b9d77a4-ad65-4b2c-a3db-f0e0cd210ba9-kube-api-access-fvxfs" (OuterVolumeSpecName: "kube-api-access-fvxfs") pod "1b9d77a4-ad65-4b2c-a3db-f0e0cd210ba9" (UID: "1b9d77a4-ad65-4b2c-a3db-f0e0cd210ba9"). InnerVolumeSpecName "kube-api-access-fvxfs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:21:04.537101 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:04.537017    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0863f7d9-0b86-4536-b02a-f9feb30134b8-kube-api-access-8g5l4" (OuterVolumeSpecName: "kube-api-access-8g5l4") pod "0863f7d9-0b86-4536-b02a-f9feb30134b8" (UID: "0863f7d9-0b86-4536-b02a-f9feb30134b8"). InnerVolumeSpecName "kube-api-access-8g5l4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:21:04.636248 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:04.636159    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fvxfs\" (UniqueName: \"kubernetes.io/projected/1b9d77a4-ad65-4b2c-a3db-f0e0cd210ba9-kube-api-access-fvxfs\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:21:04.636248 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:04.636192    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8g5l4\" (UniqueName: \"kubernetes.io/projected/0863f7d9-0b86-4536-b02a-f9feb30134b8-kube-api-access-8g5l4\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:21:04.747907 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:04.747874    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37b85497-cb89-4a64-aef1-83f552ab207f" path="/var/lib/kubelet/pods/37b85497-cb89-4a64-aef1-83f552ab207f/volumes"
Jul 23 23:21:05.065044 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:05.065007    2548 generic.go:356] "Generic (PLEG): container finished" podID="1b9d77a4-ad65-4b2c-a3db-f0e0cd210ba9" containerID="608c49e6e40e0cb8a1b24843e0ae55bc96ab29a5457c0260ef39c414a9b06ab5" exitCode=0
Jul 23 23:21:05.065044 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:05.065045    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5f4c48574c-p6nzj"
Jul 23 23:21:05.065575 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:05.065081    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5f4c48574c-p6nzj" event={"ID":"1b9d77a4-ad65-4b2c-a3db-f0e0cd210ba9","Type":"ContainerDied","Data":"608c49e6e40e0cb8a1b24843e0ae55bc96ab29a5457c0260ef39c414a9b06ab5"}
Jul 23 23:21:05.065575 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:05.065118    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5f4c48574c-p6nzj" event={"ID":"1b9d77a4-ad65-4b2c-a3db-f0e0cd210ba9","Type":"ContainerDied","Data":"8223a151f35465fc6d4d996263a4c62c23308633ef0a644ea6955b908776899e"}
Jul 23 23:21:05.065575 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:05.065134    2548 scope.go:117] "RemoveContainer" containerID="608c49e6e40e0cb8a1b24843e0ae55bc96ab29a5457c0260ef39c414a9b06ab5"
Jul 23 23:21:05.066402 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:05.066289    2548 generic.go:356] "Generic (PLEG): container finished" podID="0863f7d9-0b86-4536-b02a-f9feb30134b8" containerID="e87c0676f19788dd0b5f3a8c32849edc5c32cb8484b428019c151629876a2015" exitCode=0
Jul 23 23:21:05.066402 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:05.066339    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-tn58t"
Jul 23 23:21:05.066402 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:05.066392    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-tn58t" event={"ID":"0863f7d9-0b86-4536-b02a-f9feb30134b8","Type":"ContainerDied","Data":"e87c0676f19788dd0b5f3a8c32849edc5c32cb8484b428019c151629876a2015"}
Jul 23 23:21:05.066554 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:05.066421    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-tn58t" event={"ID":"0863f7d9-0b86-4536-b02a-f9feb30134b8","Type":"ContainerDied","Data":"e6d6ece1c5792982a187aab63259edf941e24cdc81bf68d160733ad183c69561"}
Jul 23 23:21:05.073863 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:05.073836    2548 scope.go:117] "RemoveContainer" containerID="608c49e6e40e0cb8a1b24843e0ae55bc96ab29a5457c0260ef39c414a9b06ab5"
Jul 23 23:21:05.074131 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:21:05.074110    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"608c49e6e40e0cb8a1b24843e0ae55bc96ab29a5457c0260ef39c414a9b06ab5\": container with ID starting with 608c49e6e40e0cb8a1b24843e0ae55bc96ab29a5457c0260ef39c414a9b06ab5 not found: ID does not exist" containerID="608c49e6e40e0cb8a1b24843e0ae55bc96ab29a5457c0260ef39c414a9b06ab5"
Jul 23 23:21:05.074222 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:05.074138    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"608c49e6e40e0cb8a1b24843e0ae55bc96ab29a5457c0260ef39c414a9b06ab5"} err="failed to get container status \"608c49e6e40e0cb8a1b24843e0ae55bc96ab29a5457c0260ef39c414a9b06ab5\": rpc error: code = NotFound desc = could not find container \"608c49e6e40e0cb8a1b24843e0ae55bc96ab29a5457c0260ef39c414a9b06ab5\": container with ID starting with 608c49e6e40e0cb8a1b24843e0ae55bc96ab29a5457c0260ef39c414a9b06ab5 not found: ID does not exist"
Jul 23 23:21:05.074222 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:05.074155    2548 scope.go:117] "RemoveContainer" containerID="e87c0676f19788dd0b5f3a8c32849edc5c32cb8484b428019c151629876a2015"
Jul 23 23:21:05.082065 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:05.082048    2548 scope.go:117] "RemoveContainer" containerID="e87c0676f19788dd0b5f3a8c32849edc5c32cb8484b428019c151629876a2015"
Jul 23 23:21:05.082338 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:21:05.082317    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e87c0676f19788dd0b5f3a8c32849edc5c32cb8484b428019c151629876a2015\": container with ID starting with e87c0676f19788dd0b5f3a8c32849edc5c32cb8484b428019c151629876a2015 not found: ID does not exist" containerID="e87c0676f19788dd0b5f3a8c32849edc5c32cb8484b428019c151629876a2015"
Jul 23 23:21:05.082399 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:05.082345    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e87c0676f19788dd0b5f3a8c32849edc5c32cb8484b428019c151629876a2015"} err="failed to get container status \"e87c0676f19788dd0b5f3a8c32849edc5c32cb8484b428019c151629876a2015\": rpc error: code = NotFound desc = could not find container \"e87c0676f19788dd0b5f3a8c32849edc5c32cb8484b428019c151629876a2015\": container with ID starting with e87c0676f19788dd0b5f3a8c32849edc5c32cb8484b428019c151629876a2015 not found: ID does not exist"
Jul 23 23:21:05.082500 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:05.082483    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-tn58t"]
Jul 23 23:21:05.085901 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:05.085883    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-tn58t"]
Jul 23 23:21:05.095884 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:05.095863    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5f4c48574c-p6nzj"]
Jul 23 23:21:05.099699 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:05.099676    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5f4c48574c-p6nzj"]
Jul 23 23:21:06.752228 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:06.752190    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0863f7d9-0b86-4536-b02a-f9feb30134b8" path="/var/lib/kubelet/pods/0863f7d9-0b86-4536-b02a-f9feb30134b8/volumes"
Jul 23 23:21:06.752744 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:06.752501    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b9d77a4-ad65-4b2c-a3db-f0e0cd210ba9" path="/var/lib/kubelet/pods/1b9d77a4-ad65-4b2c-a3db-f0e0cd210ba9/volumes"
Jul 23 23:21:45.143046 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:21:45.143013    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:22:13.186699 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:22:13.186626    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:22:26.777033 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:22:26.776990    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:22:33.476039 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:22:33.476001    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:22:50.179904 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:22:50.179868    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:23:02.687876 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:02.687830    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:23:07.487028 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:07.486992    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt"]
Jul 23 23:23:07.487566 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:07.487394    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b9d77a4-ad65-4b2c-a3db-f0e0cd210ba9" containerName="authorino"
Jul 23 23:23:07.487566 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:07.487413    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b9d77a4-ad65-4b2c-a3db-f0e0cd210ba9" containerName="authorino"
Jul 23 23:23:07.487566 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:07.487438    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0863f7d9-0b86-4536-b02a-f9feb30134b8" containerName="authorino"
Jul 23 23:23:07.487566 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:07.487447    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="0863f7d9-0b86-4536-b02a-f9feb30134b8" containerName="authorino"
Jul 23 23:23:07.487566 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:07.487541    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="0863f7d9-0b86-4536-b02a-f9feb30134b8" containerName="authorino"
Jul 23 23:23:07.487566 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:07.487555    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b9d77a4-ad65-4b2c-a3db-f0e0cd210ba9" containerName="authorino"
Jul 23 23:23:07.490644 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:07.490625    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt"
Jul 23 23:23:07.494465 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:07.494441    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 23 23:23:07.494622 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:07.494609    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-g6lpf\""
Jul 23 23:23:07.494754 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:07.494742    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 23 23:23:07.494997 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:07.494966    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 23 23:23:07.500523 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:07.500502    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt"]
Jul 23 23:23:07.604643 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:07.604605    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f534a349-dcfc-48c7-9c43-9f9732cfde69-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt\" (UID: \"f534a349-dcfc-48c7-9c43-9f9732cfde69\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt"
Jul 23 23:23:07.604819 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:07.604732    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjc79\" (UniqueName: \"kubernetes.io/projected/f534a349-dcfc-48c7-9c43-9f9732cfde69-kube-api-access-jjc79\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt\" (UID: \"f534a349-dcfc-48c7-9c43-9f9732cfde69\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt"
Jul 23 23:23:07.604819 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:07.604802    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f534a349-dcfc-48c7-9c43-9f9732cfde69-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt\" (UID: \"f534a349-dcfc-48c7-9c43-9f9732cfde69\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt"
Jul 23 23:23:07.604928 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:07.604842    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f534a349-dcfc-48c7-9c43-9f9732cfde69-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt\" (UID: \"f534a349-dcfc-48c7-9c43-9f9732cfde69\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt"
Jul 23 23:23:07.604928 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:07.604900    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f534a349-dcfc-48c7-9c43-9f9732cfde69-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt\" (UID: \"f534a349-dcfc-48c7-9c43-9f9732cfde69\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt"
Jul 23 23:23:07.605075 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:07.604950    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f534a349-dcfc-48c7-9c43-9f9732cfde69-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt\" (UID: \"f534a349-dcfc-48c7-9c43-9f9732cfde69\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt"
Jul 23 23:23:07.706161 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:07.706123    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f534a349-dcfc-48c7-9c43-9f9732cfde69-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt\" (UID: \"f534a349-dcfc-48c7-9c43-9f9732cfde69\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt"
Jul 23 23:23:07.706351 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:07.706174    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjc79\" (UniqueName: \"kubernetes.io/projected/f534a349-dcfc-48c7-9c43-9f9732cfde69-kube-api-access-jjc79\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt\" (UID: \"f534a349-dcfc-48c7-9c43-9f9732cfde69\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt"
Jul 23 23:23:07.706351 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:07.706197    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f534a349-dcfc-48c7-9c43-9f9732cfde69-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt\" (UID: \"f534a349-dcfc-48c7-9c43-9f9732cfde69\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt"
Jul 23 23:23:07.706351 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:07.706221    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f534a349-dcfc-48c7-9c43-9f9732cfde69-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt\" (UID: \"f534a349-dcfc-48c7-9c43-9f9732cfde69\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt"
Jul 23 23:23:07.706351 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:07.706252    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f534a349-dcfc-48c7-9c43-9f9732cfde69-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt\" (UID: \"f534a349-dcfc-48c7-9c43-9f9732cfde69\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt"
Jul 23 23:23:07.706351 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:07.706280    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f534a349-dcfc-48c7-9c43-9f9732cfde69-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt\" (UID: \"f534a349-dcfc-48c7-9c43-9f9732cfde69\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt"
Jul 23 23:23:07.706622 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:07.706604    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f534a349-dcfc-48c7-9c43-9f9732cfde69-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt\" (UID: \"f534a349-dcfc-48c7-9c43-9f9732cfde69\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt"
Jul 23 23:23:07.706682 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:07.706662    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f534a349-dcfc-48c7-9c43-9f9732cfde69-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt\" (UID: \"f534a349-dcfc-48c7-9c43-9f9732cfde69\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt"
Jul 23 23:23:07.706770 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:07.706748    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f534a349-dcfc-48c7-9c43-9f9732cfde69-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt\" (UID: \"f534a349-dcfc-48c7-9c43-9f9732cfde69\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt"
Jul 23 23:23:07.708579 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:07.708557    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f534a349-dcfc-48c7-9c43-9f9732cfde69-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt\" (UID: \"f534a349-dcfc-48c7-9c43-9f9732cfde69\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt"
Jul 23 23:23:07.708810 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:07.708791    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f534a349-dcfc-48c7-9c43-9f9732cfde69-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt\" (UID: \"f534a349-dcfc-48c7-9c43-9f9732cfde69\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt"
Jul 23 23:23:07.713610 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:07.713585    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjc79\" (UniqueName: \"kubernetes.io/projected/f534a349-dcfc-48c7-9c43-9f9732cfde69-kube-api-access-jjc79\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt\" (UID: \"f534a349-dcfc-48c7-9c43-9f9732cfde69\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt"
Jul 23 23:23:07.800549 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:07.800456    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt"
Jul 23 23:23:07.930206 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:07.930184    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt"]
Jul 23 23:23:07.931901 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:23:07.931869    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf534a349_dcfc_48c7_9c43_9f9732cfde69.slice/crio-b2252ca129295906d712edea3719b5bc90d58a30790bd5f02ed8b81986eef7ca WatchSource:0}: Error finding container b2252ca129295906d712edea3719b5bc90d58a30790bd5f02ed8b81986eef7ca: Status 404 returned error can't find the container with id b2252ca129295906d712edea3719b5bc90d58a30790bd5f02ed8b81986eef7ca
Jul 23 23:23:08.680801 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:08.680754    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt" event={"ID":"f534a349-dcfc-48c7-9c43-9f9732cfde69","Type":"ContainerStarted","Data":"b2252ca129295906d712edea3719b5bc90d58a30790bd5f02ed8b81986eef7ca"}
Jul 23 23:23:09.688039 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:09.687999    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt" event={"ID":"f534a349-dcfc-48c7-9c43-9f9732cfde69","Type":"ContainerStarted","Data":"6a834c6b457c0c481ef79db2f54c07ac3faf07bd16777f7337df4e82b5017060"}
Jul 23 23:23:09.708657 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:09.708598    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt" podStartSLOduration=1.716462789 podStartE2EDuration="2.70858093s" totalImagesPullingTime="992.118141ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:23:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:23:09.706918487 +0000 UTC m=+591.450647536" watchObservedRunningTime="2026-07-23 23:23:09.70858093 +0000 UTC m=+591.452309978"
Jul 23 23:23:15.390379 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:15.390340    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:23:17.800818 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:17.800775    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt"
Jul 23 23:23:17.800818 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:17.800829    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt"
Jul 23 23:23:17.813372 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:17.813349    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt"
Jul 23 23:23:18.747751 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:23:18.747715    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt"
Jul 23 23:24:01.481223 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:24:01.481191    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-86db8d67c8-4997k"]
Jul 23 23:24:01.484807 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:24:01.484788    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-86db8d67c8-4997k"
Jul 23 23:24:01.494274 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:24:01.494249    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-86db8d67c8-4997k"]
Jul 23 23:24:01.559164 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:24:01.559128    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/88a33ff0-1277-4284-9fe7-351f45c14df9-tls-cert\") pod \"authorino-86db8d67c8-4997k\" (UID: \"88a33ff0-1277-4284-9fe7-351f45c14df9\") " pod="kuadrant-system/authorino-86db8d67c8-4997k"
Jul 23 23:24:01.559315 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:24:01.559173    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mkjk\" (UniqueName: \"kubernetes.io/projected/88a33ff0-1277-4284-9fe7-351f45c14df9-kube-api-access-7mkjk\") pod \"authorino-86db8d67c8-4997k\" (UID: \"88a33ff0-1277-4284-9fe7-351f45c14df9\") " pod="kuadrant-system/authorino-86db8d67c8-4997k"
Jul 23 23:24:01.660397 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:24:01.660365    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/88a33ff0-1277-4284-9fe7-351f45c14df9-tls-cert\") pod \"authorino-86db8d67c8-4997k\" (UID: \"88a33ff0-1277-4284-9fe7-351f45c14df9\") " pod="kuadrant-system/authorino-86db8d67c8-4997k"
Jul 23 23:24:01.660546 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:24:01.660407    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mkjk\" (UniqueName: \"kubernetes.io/projected/88a33ff0-1277-4284-9fe7-351f45c14df9-kube-api-access-7mkjk\") pod \"authorino-86db8d67c8-4997k\" (UID: \"88a33ff0-1277-4284-9fe7-351f45c14df9\") " pod="kuadrant-system/authorino-86db8d67c8-4997k"
Jul 23 23:24:01.662761 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:24:01.662733    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/88a33ff0-1277-4284-9fe7-351f45c14df9-tls-cert\") pod \"authorino-86db8d67c8-4997k\" (UID: \"88a33ff0-1277-4284-9fe7-351f45c14df9\") " pod="kuadrant-system/authorino-86db8d67c8-4997k"
Jul 23 23:24:01.668076 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:24:01.668054    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mkjk\" (UniqueName: \"kubernetes.io/projected/88a33ff0-1277-4284-9fe7-351f45c14df9-kube-api-access-7mkjk\") pod \"authorino-86db8d67c8-4997k\" (UID: \"88a33ff0-1277-4284-9fe7-351f45c14df9\") " pod="kuadrant-system/authorino-86db8d67c8-4997k"
Jul 23 23:24:01.795039 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:24:01.794930    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-86db8d67c8-4997k"
Jul 23 23:24:01.923559 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:24:01.923528    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-86db8d67c8-4997k"]
Jul 23 23:24:01.924878 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:24:01.924854    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod88a33ff0_1277_4284_9fe7_351f45c14df9.slice/crio-67452a6031f1b95c1843e362288a4ccd6d133e2de8ffd4e7bc6ca8acd4068313 WatchSource:0}: Error finding container 67452a6031f1b95c1843e362288a4ccd6d133e2de8ffd4e7bc6ca8acd4068313: Status 404 returned error can't find the container with id 67452a6031f1b95c1843e362288a4ccd6d133e2de8ffd4e7bc6ca8acd4068313
Jul 23 23:24:01.948444 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:24:01.948411    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-86db8d67c8-4997k" event={"ID":"88a33ff0-1277-4284-9fe7-351f45c14df9","Type":"ContainerStarted","Data":"67452a6031f1b95c1843e362288a4ccd6d133e2de8ffd4e7bc6ca8acd4068313"}
Jul 23 23:24:02.957135 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:24:02.957081    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-86db8d67c8-4997k" event={"ID":"88a33ff0-1277-4284-9fe7-351f45c14df9","Type":"ContainerStarted","Data":"52b406d7a3b560929b95f3a4f3cba5d5d7745a413382ffbc9d5993d0106c01d2"}
Jul 23 23:24:02.973796 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:24:02.973746    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-86db8d67c8-4997k" podStartSLOduration=1.567076857 podStartE2EDuration="1.97372939s" totalImagesPullingTime="406.652533ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:24:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:24:02.972258065 +0000 UTC m=+644.715987107" watchObservedRunningTime="2026-07-23 23:24:02.97372939 +0000 UTC m=+644.717458461"
Jul 23 23:24:03.002924 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:24:03.002884    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7947b97487-bbg5w"]
Jul 23 23:24:03.003548 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:24:03.003478    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7947b97487-bbg5w" podUID="c9cac6f8-2139-4258-988c-db13febd1a0e" containerName="authorino" containerID="cri-o://0500d3545da051ce02202536f9ab389a3ec3bac11e501d18160a8e6c1fc2e04c" gracePeriod=30
Jul 23 23:24:03.282991 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:24:03.282958    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7947b97487-bbg5w"
Jul 23 23:24:03.375654 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:24:03.375616    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c9cac6f8-2139-4258-988c-db13febd1a0e-tls-cert\") pod \"c9cac6f8-2139-4258-988c-db13febd1a0e\" (UID: \"c9cac6f8-2139-4258-988c-db13febd1a0e\") "
Jul 23 23:24:03.375826 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:24:03.375730    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9v4nw\" (UniqueName: \"kubernetes.io/projected/c9cac6f8-2139-4258-988c-db13febd1a0e-kube-api-access-9v4nw\") pod \"c9cac6f8-2139-4258-988c-db13febd1a0e\" (UID: \"c9cac6f8-2139-4258-988c-db13febd1a0e\") "
Jul 23 23:24:03.377691 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:24:03.377660    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9cac6f8-2139-4258-988c-db13febd1a0e-kube-api-access-9v4nw" (OuterVolumeSpecName: "kube-api-access-9v4nw") pod "c9cac6f8-2139-4258-988c-db13febd1a0e" (UID: "c9cac6f8-2139-4258-988c-db13febd1a0e"). InnerVolumeSpecName "kube-api-access-9v4nw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:24:03.385603 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:24:03.385510    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9cac6f8-2139-4258-988c-db13febd1a0e-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "c9cac6f8-2139-4258-988c-db13febd1a0e" (UID: "c9cac6f8-2139-4258-988c-db13febd1a0e"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:24:03.477168 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:24:03.477136    2548 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c9cac6f8-2139-4258-988c-db13febd1a0e-tls-cert\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:24:03.477168 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:24:03.477166    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9v4nw\" (UniqueName: \"kubernetes.io/projected/c9cac6f8-2139-4258-988c-db13febd1a0e-kube-api-access-9v4nw\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:24:03.965106 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:24:03.965016    2548 generic.go:356] "Generic (PLEG): container finished" podID="c9cac6f8-2139-4258-988c-db13febd1a0e" containerID="0500d3545da051ce02202536f9ab389a3ec3bac11e501d18160a8e6c1fc2e04c" exitCode=0
Jul 23 23:24:03.965106 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:24:03.965077    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7947b97487-bbg5w"
Jul 23 23:24:03.965616 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:24:03.965106    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7947b97487-bbg5w" event={"ID":"c9cac6f8-2139-4258-988c-db13febd1a0e","Type":"ContainerDied","Data":"0500d3545da051ce02202536f9ab389a3ec3bac11e501d18160a8e6c1fc2e04c"}
Jul 23 23:24:03.965616 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:24:03.965163    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7947b97487-bbg5w" event={"ID":"c9cac6f8-2139-4258-988c-db13febd1a0e","Type":"ContainerDied","Data":"bed2df12d18d765ed455d2ef1f56aa82792e283a9958a40023027cbd53d5e973"}
Jul 23 23:24:03.965616 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:24:03.965195    2548 scope.go:117] "RemoveContainer" containerID="0500d3545da051ce02202536f9ab389a3ec3bac11e501d18160a8e6c1fc2e04c"
Jul 23 23:24:03.976431 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:24:03.976377    2548 scope.go:117] "RemoveContainer" containerID="0500d3545da051ce02202536f9ab389a3ec3bac11e501d18160a8e6c1fc2e04c"
Jul 23 23:24:03.976796 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:24:03.976758    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0500d3545da051ce02202536f9ab389a3ec3bac11e501d18160a8e6c1fc2e04c\": container with ID starting with 0500d3545da051ce02202536f9ab389a3ec3bac11e501d18160a8e6c1fc2e04c not found: ID does not exist" containerID="0500d3545da051ce02202536f9ab389a3ec3bac11e501d18160a8e6c1fc2e04c"
Jul 23 23:24:03.976863 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:24:03.976831    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0500d3545da051ce02202536f9ab389a3ec3bac11e501d18160a8e6c1fc2e04c"} err="failed to get container status \"0500d3545da051ce02202536f9ab389a3ec3bac11e501d18160a8e6c1fc2e04c\": rpc error: code = NotFound desc = could not find container \"0500d3545da051ce02202536f9ab389a3ec3bac11e501d18160a8e6c1fc2e04c\": container with ID starting with 0500d3545da051ce02202536f9ab389a3ec3bac11e501d18160a8e6c1fc2e04c not found: ID does not exist"
Jul 23 23:24:03.990689 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:24:03.990659    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7947b97487-bbg5w"]
Jul 23 23:24:03.994326 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:24:03.994299    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7947b97487-bbg5w"]
Jul 23 23:24:04.747501 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:24:04.747471    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9cac6f8-2139-4258-988c-db13febd1a0e" path="/var/lib/kubelet/pods/c9cac6f8-2139-4258-988c-db13febd1a0e/volumes"
Jul 23 23:24:31.989510 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:24:31.989471    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:24:43.477124 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:24:43.477089    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:24:53.490002 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:24:53.489949    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:26:36.787789 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:26:36.787751    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:26:46.876647 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:26:46.876549    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:26:59.388411 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:26:59.388371    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:27:16.780273 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:27:16.780237    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:27:25.878761 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:27:25.878727    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:27:34.585408 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:27:34.585370    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:27:43.781892 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:27:43.781855    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:27:54.887708 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:27:54.887671    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:28:06.687620 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:28:06.687583    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:28:17.286507 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:28:17.286472    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:28:27.386056 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:28:27.386024    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:28:49.387514 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:28:49.387479    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:29:00.788116 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:29:00.788082    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:29:23.788250 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:29:23.788214    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:29:35.685458 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:29:35.685420    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:29:51.584265 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:29:51.584183    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:30:00.794237 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:30:00.794197    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:30:10.202817 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:30:10.202784    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:30:19.577451 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:30:19.577415    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:30:28.489218 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:30:28.489183    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:32:17.193039 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:32:17.193003    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:32:28.494630 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:32:28.494587    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:32:36.983798 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:32:36.983760    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:32:50.693025 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:32:50.692925    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:32:59.987529 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:32:59.987485    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:33:10.980730 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:33:10.980694    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:33:20.379602 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:33:20.379562    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:33:30.286162 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:33:30.286127    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:33:43.081195 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:33:43.081159    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:33:50.980953 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:33:50.980913    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:34:01.652218 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:01.652180    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:34:23.488570 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:23.488534    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:34:31.314293 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:31.314256    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2"]
Jul 23 23:34:31.316636 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:31.314588    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9cac6f8-2139-4258-988c-db13febd1a0e" containerName="authorino"
Jul 23 23:34:31.316636 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:31.314601    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9cac6f8-2139-4258-988c-db13febd1a0e" containerName="authorino"
Jul 23 23:34:31.316636 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:31.314654    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9cac6f8-2139-4258-988c-db13febd1a0e" containerName="authorino"
Jul 23 23:34:31.317503 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:31.317486    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2"
Jul 23 23:34:31.319767 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:31.319748    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-de4e89b1-kserve-self-signed-certs\""
Jul 23 23:34:31.330935 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:31.330915    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2"]
Jul 23 23:34:31.426499 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:31.426462    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f975f697-bf96-4c79-bf13-1cb38c55f34a-dshm\") pod \"e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2\" (UID: \"f975f697-bf96-4c79-bf13-1cb38c55f34a\") " pod="llm/e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2"
Jul 23 23:34:31.426669 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:31.426516    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f975f697-bf96-4c79-bf13-1cb38c55f34a-home\") pod \"e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2\" (UID: \"f975f697-bf96-4c79-bf13-1cb38c55f34a\") " pod="llm/e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2"
Jul 23 23:34:31.426669 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:31.426540    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f975f697-bf96-4c79-bf13-1cb38c55f34a-tmp-dir\") pod \"e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2\" (UID: \"f975f697-bf96-4c79-bf13-1cb38c55f34a\") " pod="llm/e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2"
Jul 23 23:34:31.426669 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:31.426601    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4wxq\" (UniqueName: \"kubernetes.io/projected/f975f697-bf96-4c79-bf13-1cb38c55f34a-kube-api-access-l4wxq\") pod \"e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2\" (UID: \"f975f697-bf96-4c79-bf13-1cb38c55f34a\") " pod="llm/e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2"
Jul 23 23:34:31.426669 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:31.426664    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f975f697-bf96-4c79-bf13-1cb38c55f34a-model-cache\") pod \"e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2\" (UID: \"f975f697-bf96-4c79-bf13-1cb38c55f34a\") " pod="llm/e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2"
Jul 23 23:34:31.426811 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:31.426688    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f975f697-bf96-4c79-bf13-1cb38c55f34a-tls-certs\") pod \"e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2\" (UID: \"f975f697-bf96-4c79-bf13-1cb38c55f34a\") " pod="llm/e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2"
Jul 23 23:34:31.527991 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:31.527936    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f975f697-bf96-4c79-bf13-1cb38c55f34a-dshm\") pod \"e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2\" (UID: \"f975f697-bf96-4c79-bf13-1cb38c55f34a\") " pod="llm/e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2"
Jul 23 23:34:31.528179 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:31.528022    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f975f697-bf96-4c79-bf13-1cb38c55f34a-home\") pod \"e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2\" (UID: \"f975f697-bf96-4c79-bf13-1cb38c55f34a\") " pod="llm/e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2"
Jul 23 23:34:31.528179 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:31.528045    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f975f697-bf96-4c79-bf13-1cb38c55f34a-tmp-dir\") pod \"e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2\" (UID: \"f975f697-bf96-4c79-bf13-1cb38c55f34a\") " pod="llm/e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2"
Jul 23 23:34:31.528179 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:31.528074    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4wxq\" (UniqueName: \"kubernetes.io/projected/f975f697-bf96-4c79-bf13-1cb38c55f34a-kube-api-access-l4wxq\") pod \"e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2\" (UID: \"f975f697-bf96-4c79-bf13-1cb38c55f34a\") " pod="llm/e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2"
Jul 23 23:34:31.528179 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:31.528107    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f975f697-bf96-4c79-bf13-1cb38c55f34a-model-cache\") pod \"e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2\" (UID: \"f975f697-bf96-4c79-bf13-1cb38c55f34a\") " pod="llm/e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2"
Jul 23 23:34:31.528179 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:31.528128    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f975f697-bf96-4c79-bf13-1cb38c55f34a-tls-certs\") pod \"e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2\" (UID: \"f975f697-bf96-4c79-bf13-1cb38c55f34a\") " pod="llm/e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2"
Jul 23 23:34:31.528453 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:31.528432    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f975f697-bf96-4c79-bf13-1cb38c55f34a-home\") pod \"e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2\" (UID: \"f975f697-bf96-4c79-bf13-1cb38c55f34a\") " pod="llm/e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2"
Jul 23 23:34:31.528527 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:31.528478    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f975f697-bf96-4c79-bf13-1cb38c55f34a-tmp-dir\") pod \"e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2\" (UID: \"f975f697-bf96-4c79-bf13-1cb38c55f34a\") " pod="llm/e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2"
Jul 23 23:34:31.528527 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:31.528518    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f975f697-bf96-4c79-bf13-1cb38c55f34a-model-cache\") pod \"e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2\" (UID: \"f975f697-bf96-4c79-bf13-1cb38c55f34a\") " pod="llm/e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2"
Jul 23 23:34:31.530339 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:31.530316    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f975f697-bf96-4c79-bf13-1cb38c55f34a-dshm\") pod \"e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2\" (UID: \"f975f697-bf96-4c79-bf13-1cb38c55f34a\") " pod="llm/e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2"
Jul 23 23:34:31.530461 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:31.530440    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f975f697-bf96-4c79-bf13-1cb38c55f34a-tls-certs\") pod \"e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2\" (UID: \"f975f697-bf96-4c79-bf13-1cb38c55f34a\") " pod="llm/e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2"
Jul 23 23:34:31.535429 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:31.535405    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4wxq\" (UniqueName: \"kubernetes.io/projected/f975f697-bf96-4c79-bf13-1cb38c55f34a-kube-api-access-l4wxq\") pod \"e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2\" (UID: \"f975f697-bf96-4c79-bf13-1cb38c55f34a\") " pod="llm/e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2"
Jul 23 23:34:31.626606 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:31.626497    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2"
Jul 23 23:34:31.757636 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:31.757516    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2"]
Jul 23 23:34:31.760221 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:34:31.760186    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf975f697_bf96_4c79_bf13_1cb38c55f34a.slice/crio-b68ced2d76b6c2565d8a8dbdedce2208a4b02a5976d0cca727d9982e529ec306 WatchSource:0}: Error finding container b68ced2d76b6c2565d8a8dbdedce2208a4b02a5976d0cca727d9982e529ec306: Status 404 returned error can't find the container with id b68ced2d76b6c2565d8a8dbdedce2208a4b02a5976d0cca727d9982e529ec306
Jul 23 23:34:32.074514 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:32.074480    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2" event={"ID":"f975f697-bf96-4c79-bf13-1cb38c55f34a","Type":"ContainerStarted","Data":"ac10b37e1a2556ca15afc75d8cea7a8b0fce75f7652aa7403833c7ca47749800"}
Jul 23 23:34:32.074514 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:32.074519    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2" event={"ID":"f975f697-bf96-4c79-bf13-1cb38c55f34a","Type":"ContainerStarted","Data":"b68ced2d76b6c2565d8a8dbdedce2208a4b02a5976d0cca727d9982e529ec306"}
Jul 23 23:34:32.090747 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:32.090687    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2" podStartSLOduration=1.090672541 podStartE2EDuration="1.090672541s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:34:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:34:32.088889512 +0000 UTC m=+1273.832618561" watchObservedRunningTime="2026-07-23 23:34:32.090672541 +0000 UTC m=+1273.834401590"
Jul 23 23:34:33.776593 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:33.776559    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:34:34.889664 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:34.889622    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt"]
Jul 23 23:34:34.893136 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:34.893113    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt"
Jul 23 23:34:34.895934 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:34.895908    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-de4e89b1-kserve-self-signed-certs\""
Jul 23 23:34:34.901931 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:34.901908    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt"]
Jul 23 23:34:35.061857 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:35.061822    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a330ffc-24f4-465d-8f02-53943908364e-home\") pod \"e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt\" (UID: \"3a330ffc-24f4-465d-8f02-53943908364e\") " pod="llm/e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt"
Jul 23 23:34:35.061857 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:35.061861    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a330ffc-24f4-465d-8f02-53943908364e-tmp-dir\") pod \"e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt\" (UID: \"3a330ffc-24f4-465d-8f02-53943908364e\") " pod="llm/e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt"
Jul 23 23:34:35.062067 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:35.061881    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbqvp\" (UniqueName: \"kubernetes.io/projected/3a330ffc-24f4-465d-8f02-53943908364e-kube-api-access-fbqvp\") pod \"e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt\" (UID: \"3a330ffc-24f4-465d-8f02-53943908364e\") " pod="llm/e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt"
Jul 23 23:34:35.062067 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:35.061946    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a330ffc-24f4-465d-8f02-53943908364e-tls-certs\") pod \"e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt\" (UID: \"3a330ffc-24f4-465d-8f02-53943908364e\") " pod="llm/e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt"
Jul 23 23:34:35.062067 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:35.061971    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a330ffc-24f4-465d-8f02-53943908364e-dshm\") pod \"e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt\" (UID: \"3a330ffc-24f4-465d-8f02-53943908364e\") " pod="llm/e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt"
Jul 23 23:34:35.062067 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:35.062008    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a330ffc-24f4-465d-8f02-53943908364e-model-cache\") pod \"e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt\" (UID: \"3a330ffc-24f4-465d-8f02-53943908364e\") " pod="llm/e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt"
Jul 23 23:34:35.162629 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:35.162595    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a330ffc-24f4-465d-8f02-53943908364e-tls-certs\") pod \"e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt\" (UID: \"3a330ffc-24f4-465d-8f02-53943908364e\") " pod="llm/e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt"
Jul 23 23:34:35.162811 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:35.162651    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a330ffc-24f4-465d-8f02-53943908364e-dshm\") pod \"e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt\" (UID: \"3a330ffc-24f4-465d-8f02-53943908364e\") " pod="llm/e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt"
Jul 23 23:34:35.162811 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:35.162681    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a330ffc-24f4-465d-8f02-53943908364e-model-cache\") pod \"e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt\" (UID: \"3a330ffc-24f4-465d-8f02-53943908364e\") " pod="llm/e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt"
Jul 23 23:34:35.162811 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:35.162767    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a330ffc-24f4-465d-8f02-53943908364e-home\") pod \"e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt\" (UID: \"3a330ffc-24f4-465d-8f02-53943908364e\") " pod="llm/e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt"
Jul 23 23:34:35.162811 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:35.162793    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a330ffc-24f4-465d-8f02-53943908364e-tmp-dir\") pod \"e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt\" (UID: \"3a330ffc-24f4-465d-8f02-53943908364e\") " pod="llm/e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt"
Jul 23 23:34:35.163057 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:35.162833    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbqvp\" (UniqueName: \"kubernetes.io/projected/3a330ffc-24f4-465d-8f02-53943908364e-kube-api-access-fbqvp\") pod \"e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt\" (UID: \"3a330ffc-24f4-465d-8f02-53943908364e\") " pod="llm/e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt"
Jul 23 23:34:35.163393 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:35.163367    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a330ffc-24f4-465d-8f02-53943908364e-model-cache\") pod \"e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt\" (UID: \"3a330ffc-24f4-465d-8f02-53943908364e\") " pod="llm/e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt"
Jul 23 23:34:35.163547 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:35.163480    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a330ffc-24f4-465d-8f02-53943908364e-tmp-dir\") pod \"e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt\" (UID: \"3a330ffc-24f4-465d-8f02-53943908364e\") " pod="llm/e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt"
Jul 23 23:34:35.163547 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:35.163490    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a330ffc-24f4-465d-8f02-53943908364e-home\") pod \"e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt\" (UID: \"3a330ffc-24f4-465d-8f02-53943908364e\") " pod="llm/e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt"
Jul 23 23:34:35.165388 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:35.165364    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a330ffc-24f4-465d-8f02-53943908364e-dshm\") pod \"e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt\" (UID: \"3a330ffc-24f4-465d-8f02-53943908364e\") " pod="llm/e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt"
Jul 23 23:34:35.165503 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:35.165474    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a330ffc-24f4-465d-8f02-53943908364e-tls-certs\") pod \"e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt\" (UID: \"3a330ffc-24f4-465d-8f02-53943908364e\") " pod="llm/e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt"
Jul 23 23:34:35.175405 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:35.175379    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbqvp\" (UniqueName: \"kubernetes.io/projected/3a330ffc-24f4-465d-8f02-53943908364e-kube-api-access-fbqvp\") pod \"e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt\" (UID: \"3a330ffc-24f4-465d-8f02-53943908364e\") " pod="llm/e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt"
Jul 23 23:34:35.207114 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:35.207079    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt"
Jul 23 23:34:35.352308 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:35.352279    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt"]
Jul 23 23:34:35.354539 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:34:35.354506    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3a330ffc_24f4_465d_8f02_53943908364e.slice/crio-d78574d2a6bb199599413d245516bfa2937bf9ea3086fe76aa0cdbbb28a40c80 WatchSource:0}: Error finding container d78574d2a6bb199599413d245516bfa2937bf9ea3086fe76aa0cdbbb28a40c80: Status 404 returned error can't find the container with id d78574d2a6bb199599413d245516bfa2937bf9ea3086fe76aa0cdbbb28a40c80
Jul 23 23:34:36.096399 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:36.096364    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt" event={"ID":"3a330ffc-24f4-465d-8f02-53943908364e","Type":"ContainerStarted","Data":"5a58d2273a99f4561f3cc9dfc089ba048b947b8b746bbeb05bdc2988bceb4ff9"}
Jul 23 23:34:36.096399 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:36.096405    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt" event={"ID":"3a330ffc-24f4-465d-8f02-53943908364e","Type":"ContainerStarted","Data":"d78574d2a6bb199599413d245516bfa2937bf9ea3086fe76aa0cdbbb28a40c80"}
Jul 23 23:34:36.113109 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:36.113061    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt" podStartSLOduration=2.113043868 podStartE2EDuration="2.113043868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:34:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:34:36.111289385 +0000 UTC m=+1277.855018434" watchObservedRunningTime="2026-07-23 23:34:36.113043868 +0000 UTC m=+1277.856772916"
Jul 23 23:34:41.627413 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:41.627313    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2"
Jul 23 23:34:41.627413 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:41.627367    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2"
Jul 23 23:34:41.640147 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:41.640109    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2"
Jul 23 23:34:42.142257 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:42.142228    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2"
Jul 23 23:34:45.207843 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:45.207814    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt"
Jul 23 23:34:45.208321 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:45.207860    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt"
Jul 23 23:34:45.220081 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:45.220053    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt"
Jul 23 23:34:46.163279 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:46.163244    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt"
Jul 23 23:34:49.479419 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:34:49.479383    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:35:01.453605 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:01.453568    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2"]
Jul 23 23:35:02.684729 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:02.684692    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt"]
Jul 23 23:35:03.186231 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:03.186183    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:35:16.473211 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:16.473154    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2" podUID="f975f697-bf96-4c79-bf13-1cb38c55f34a" containerName="main" containerID="cri-o://ac10b37e1a2556ca15afc75d8cea7a8b0fce75f7652aa7403833c7ca47749800" gracePeriod=45
Jul 23 23:35:16.717333 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:16.717309    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2"
Jul 23 23:35:16.838610 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:16.838527    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f975f697-bf96-4c79-bf13-1cb38c55f34a-dshm\") pod \"f975f697-bf96-4c79-bf13-1cb38c55f34a\" (UID: \"f975f697-bf96-4c79-bf13-1cb38c55f34a\") "
Jul 23 23:35:16.838610 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:16.838563    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l4wxq\" (UniqueName: \"kubernetes.io/projected/f975f697-bf96-4c79-bf13-1cb38c55f34a-kube-api-access-l4wxq\") pod \"f975f697-bf96-4c79-bf13-1cb38c55f34a\" (UID: \"f975f697-bf96-4c79-bf13-1cb38c55f34a\") "
Jul 23 23:35:16.838610 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:16.838606    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f975f697-bf96-4c79-bf13-1cb38c55f34a-model-cache\") pod \"f975f697-bf96-4c79-bf13-1cb38c55f34a\" (UID: \"f975f697-bf96-4c79-bf13-1cb38c55f34a\") "
Jul 23 23:35:16.838900 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:16.838651    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f975f697-bf96-4c79-bf13-1cb38c55f34a-tls-certs\") pod \"f975f697-bf96-4c79-bf13-1cb38c55f34a\" (UID: \"f975f697-bf96-4c79-bf13-1cb38c55f34a\") "
Jul 23 23:35:16.838900 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:16.838700    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f975f697-bf96-4c79-bf13-1cb38c55f34a-tmp-dir\") pod \"f975f697-bf96-4c79-bf13-1cb38c55f34a\" (UID: \"f975f697-bf96-4c79-bf13-1cb38c55f34a\") "
Jul 23 23:35:16.838900 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:16.838719    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f975f697-bf96-4c79-bf13-1cb38c55f34a-home\") pod \"f975f697-bf96-4c79-bf13-1cb38c55f34a\" (UID: \"f975f697-bf96-4c79-bf13-1cb38c55f34a\") "
Jul 23 23:35:16.838900 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:16.838873    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f975f697-bf96-4c79-bf13-1cb38c55f34a-model-cache" (OuterVolumeSpecName: "model-cache") pod "f975f697-bf96-4c79-bf13-1cb38c55f34a" (UID: "f975f697-bf96-4c79-bf13-1cb38c55f34a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:35:16.839516 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:16.839464    2548 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f975f697-bf96-4c79-bf13-1cb38c55f34a-model-cache\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:35:16.839516 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:16.838964    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f975f697-bf96-4c79-bf13-1cb38c55f34a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f975f697-bf96-4c79-bf13-1cb38c55f34a" (UID: "f975f697-bf96-4c79-bf13-1cb38c55f34a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:35:16.839779 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:16.839739    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f975f697-bf96-4c79-bf13-1cb38c55f34a-home" (OuterVolumeSpecName: "home") pod "f975f697-bf96-4c79-bf13-1cb38c55f34a" (UID: "f975f697-bf96-4c79-bf13-1cb38c55f34a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:35:16.847193 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:16.847161    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f975f697-bf96-4c79-bf13-1cb38c55f34a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f975f697-bf96-4c79-bf13-1cb38c55f34a" (UID: "f975f697-bf96-4c79-bf13-1cb38c55f34a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:35:16.847193 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:16.847175    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f975f697-bf96-4c79-bf13-1cb38c55f34a-dshm" (OuterVolumeSpecName: "dshm") pod "f975f697-bf96-4c79-bf13-1cb38c55f34a" (UID: "f975f697-bf96-4c79-bf13-1cb38c55f34a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:35:16.847193 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:16.847186    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f975f697-bf96-4c79-bf13-1cb38c55f34a-kube-api-access-l4wxq" (OuterVolumeSpecName: "kube-api-access-l4wxq") pod "f975f697-bf96-4c79-bf13-1cb38c55f34a" (UID: "f975f697-bf96-4c79-bf13-1cb38c55f34a"). InnerVolumeSpecName "kube-api-access-l4wxq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:35:16.940854 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:16.940819    2548 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f975f697-bf96-4c79-bf13-1cb38c55f34a-tmp-dir\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:35:16.940854 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:16.940846    2548 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f975f697-bf96-4c79-bf13-1cb38c55f34a-home\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:35:16.940854 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:16.940854    2548 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f975f697-bf96-4c79-bf13-1cb38c55f34a-dshm\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:35:16.941098 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:16.940863    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4wxq\" (UniqueName: \"kubernetes.io/projected/f975f697-bf96-4c79-bf13-1cb38c55f34a-kube-api-access-l4wxq\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:35:16.941098 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:16.940873    2548 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f975f697-bf96-4c79-bf13-1cb38c55f34a-tls-certs\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:35:17.182390 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:17.182358    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:35:17.313617 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:17.313577    2548 generic.go:356] "Generic (PLEG): container finished" podID="f975f697-bf96-4c79-bf13-1cb38c55f34a" containerID="ac10b37e1a2556ca15afc75d8cea7a8b0fce75f7652aa7403833c7ca47749800" exitCode=0
Jul 23 23:35:17.313788 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:17.313630    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2"
Jul 23 23:35:17.313788 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:17.313665    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2" event={"ID":"f975f697-bf96-4c79-bf13-1cb38c55f34a","Type":"ContainerDied","Data":"ac10b37e1a2556ca15afc75d8cea7a8b0fce75f7652aa7403833c7ca47749800"}
Jul 23 23:35:17.313788 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:17.313708    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2" event={"ID":"f975f697-bf96-4c79-bf13-1cb38c55f34a","Type":"ContainerDied","Data":"b68ced2d76b6c2565d8a8dbdedce2208a4b02a5976d0cca727d9982e529ec306"}
Jul 23 23:35:17.313788 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:17.313727    2548 scope.go:117] "RemoveContainer" containerID="ac10b37e1a2556ca15afc75d8cea7a8b0fce75f7652aa7403833c7ca47749800"
Jul 23 23:35:17.324728 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:17.324702    2548 scope.go:117] "RemoveContainer" containerID="ac10b37e1a2556ca15afc75d8cea7a8b0fce75f7652aa7403833c7ca47749800"
Jul 23 23:35:17.325163 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:35:17.325143    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac10b37e1a2556ca15afc75d8cea7a8b0fce75f7652aa7403833c7ca47749800\": container with ID starting with ac10b37e1a2556ca15afc75d8cea7a8b0fce75f7652aa7403833c7ca47749800 not found: ID does not exist" containerID="ac10b37e1a2556ca15afc75d8cea7a8b0fce75f7652aa7403833c7ca47749800"
Jul 23 23:35:17.325267 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:17.325170    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac10b37e1a2556ca15afc75d8cea7a8b0fce75f7652aa7403833c7ca47749800"} err="failed to get container status \"ac10b37e1a2556ca15afc75d8cea7a8b0fce75f7652aa7403833c7ca47749800\": rpc error: code = NotFound desc = could not find container \"ac10b37e1a2556ca15afc75d8cea7a8b0fce75f7652aa7403833c7ca47749800\": container with ID starting with ac10b37e1a2556ca15afc75d8cea7a8b0fce75f7652aa7403833c7ca47749800 not found: ID does not exist"
Jul 23 23:35:17.335016 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:17.334990    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2"]
Jul 23 23:35:17.338423 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:17.338402    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-de4e89b1-kserve-56fd584599-h2hj2"]
Jul 23 23:35:17.704264 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:17.704223    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt" podUID="3a330ffc-24f4-465d-8f02-53943908364e" containerName="main" containerID="cri-o://5a58d2273a99f4561f3cc9dfc089ba048b947b8b746bbeb05bdc2988bceb4ff9" gracePeriod=45
Jul 23 23:35:17.948313 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:17.948287    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt"
Jul 23 23:35:18.049089 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:18.049014    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a330ffc-24f4-465d-8f02-53943908364e-tmp-dir\") pod \"3a330ffc-24f4-465d-8f02-53943908364e\" (UID: \"3a330ffc-24f4-465d-8f02-53943908364e\") "
Jul 23 23:35:18.049089 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:18.049055    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fbqvp\" (UniqueName: \"kubernetes.io/projected/3a330ffc-24f4-465d-8f02-53943908364e-kube-api-access-fbqvp\") pod \"3a330ffc-24f4-465d-8f02-53943908364e\" (UID: \"3a330ffc-24f4-465d-8f02-53943908364e\") "
Jul 23 23:35:18.049089 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:18.049081    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a330ffc-24f4-465d-8f02-53943908364e-dshm\") pod \"3a330ffc-24f4-465d-8f02-53943908364e\" (UID: \"3a330ffc-24f4-465d-8f02-53943908364e\") "
Jul 23 23:35:18.049383 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:18.049131    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a330ffc-24f4-465d-8f02-53943908364e-model-cache\") pod \"3a330ffc-24f4-465d-8f02-53943908364e\" (UID: \"3a330ffc-24f4-465d-8f02-53943908364e\") "
Jul 23 23:35:18.049383 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:18.049149    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a330ffc-24f4-465d-8f02-53943908364e-home\") pod \"3a330ffc-24f4-465d-8f02-53943908364e\" (UID: \"3a330ffc-24f4-465d-8f02-53943908364e\") "
Jul 23 23:35:18.049383 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:18.049170    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a330ffc-24f4-465d-8f02-53943908364e-tls-certs\") pod \"3a330ffc-24f4-465d-8f02-53943908364e\" (UID: \"3a330ffc-24f4-465d-8f02-53943908364e\") "
Jul 23 23:35:18.049383 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:18.049277    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a330ffc-24f4-465d-8f02-53943908364e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3a330ffc-24f4-465d-8f02-53943908364e" (UID: "3a330ffc-24f4-465d-8f02-53943908364e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:35:18.049566 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:18.049385    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a330ffc-24f4-465d-8f02-53943908364e-model-cache" (OuterVolumeSpecName: "model-cache") pod "3a330ffc-24f4-465d-8f02-53943908364e" (UID: "3a330ffc-24f4-465d-8f02-53943908364e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:35:18.049566 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:18.049479    2548 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a330ffc-24f4-465d-8f02-53943908364e-model-cache\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:35:18.049566 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:18.049488    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a330ffc-24f4-465d-8f02-53943908364e-home" (OuterVolumeSpecName: "home") pod "3a330ffc-24f4-465d-8f02-53943908364e" (UID: "3a330ffc-24f4-465d-8f02-53943908364e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:35:18.049566 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:18.049500    2548 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a330ffc-24f4-465d-8f02-53943908364e-tmp-dir\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:35:18.051317 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:18.051286    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a330ffc-24f4-465d-8f02-53943908364e-dshm" (OuterVolumeSpecName: "dshm") pod "3a330ffc-24f4-465d-8f02-53943908364e" (UID: "3a330ffc-24f4-465d-8f02-53943908364e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:35:18.051426 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:18.051324    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a330ffc-24f4-465d-8f02-53943908364e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3a330ffc-24f4-465d-8f02-53943908364e" (UID: "3a330ffc-24f4-465d-8f02-53943908364e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:35:18.051426 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:18.051396    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a330ffc-24f4-465d-8f02-53943908364e-kube-api-access-fbqvp" (OuterVolumeSpecName: "kube-api-access-fbqvp") pod "3a330ffc-24f4-465d-8f02-53943908364e" (UID: "3a330ffc-24f4-465d-8f02-53943908364e"). InnerVolumeSpecName "kube-api-access-fbqvp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:35:18.150349 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:18.150316    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fbqvp\" (UniqueName: \"kubernetes.io/projected/3a330ffc-24f4-465d-8f02-53943908364e-kube-api-access-fbqvp\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:35:18.150349 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:18.150345    2548 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a330ffc-24f4-465d-8f02-53943908364e-dshm\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:35:18.150349 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:18.150354    2548 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a330ffc-24f4-465d-8f02-53943908364e-home\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:35:18.150575 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:18.150363    2548 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a330ffc-24f4-465d-8f02-53943908364e-tls-certs\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:35:18.320804 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:18.320711    2548 generic.go:356] "Generic (PLEG): container finished" podID="3a330ffc-24f4-465d-8f02-53943908364e" containerID="5a58d2273a99f4561f3cc9dfc089ba048b947b8b746bbeb05bdc2988bceb4ff9" exitCode=0
Jul 23 23:35:18.320804 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:18.320759    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt" event={"ID":"3a330ffc-24f4-465d-8f02-53943908364e","Type":"ContainerDied","Data":"5a58d2273a99f4561f3cc9dfc089ba048b947b8b746bbeb05bdc2988bceb4ff9"}
Jul 23 23:35:18.320804 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:18.320804    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt" event={"ID":"3a330ffc-24f4-465d-8f02-53943908364e","Type":"ContainerDied","Data":"d78574d2a6bb199599413d245516bfa2937bf9ea3086fe76aa0cdbbb28a40c80"}
Jul 23 23:35:18.321065 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:18.320822    2548 scope.go:117] "RemoveContainer" containerID="5a58d2273a99f4561f3cc9dfc089ba048b947b8b746bbeb05bdc2988bceb4ff9"
Jul 23 23:35:18.321065 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:18.320769    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt"
Jul 23 23:35:18.330015 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:18.329996    2548 scope.go:117] "RemoveContainer" containerID="5a58d2273a99f4561f3cc9dfc089ba048b947b8b746bbeb05bdc2988bceb4ff9"
Jul 23 23:35:18.330268 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:35:18.330252    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a58d2273a99f4561f3cc9dfc089ba048b947b8b746bbeb05bdc2988bceb4ff9\": container with ID starting with 5a58d2273a99f4561f3cc9dfc089ba048b947b8b746bbeb05bdc2988bceb4ff9 not found: ID does not exist" containerID="5a58d2273a99f4561f3cc9dfc089ba048b947b8b746bbeb05bdc2988bceb4ff9"
Jul 23 23:35:18.330317 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:18.330277    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a58d2273a99f4561f3cc9dfc089ba048b947b8b746bbeb05bdc2988bceb4ff9"} err="failed to get container status \"5a58d2273a99f4561f3cc9dfc089ba048b947b8b746bbeb05bdc2988bceb4ff9\": rpc error: code = NotFound desc = could not find container \"5a58d2273a99f4561f3cc9dfc089ba048b947b8b746bbeb05bdc2988bceb4ff9\": container with ID starting with 5a58d2273a99f4561f3cc9dfc089ba048b947b8b746bbeb05bdc2988bceb4ff9 not found: ID does not exist"
Jul 23 23:35:18.340817 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:18.340796    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt"]
Jul 23 23:35:18.344841 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:18.344813    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-de4e89b1-kserve-5f5c846c75-bvjwt"]
Jul 23 23:35:18.748073 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:18.748046    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a330ffc-24f4-465d-8f02-53943908364e" path="/var/lib/kubelet/pods/3a330ffc-24f4-465d-8f02-53943908364e/volumes"
Jul 23 23:35:18.748484 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:18.748470    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f975f697-bf96-4c79-bf13-1cb38c55f34a" path="/var/lib/kubelet/pods/f975f697-bf96-4c79-bf13-1cb38c55f34a/volumes"
Jul 23 23:35:25.881460 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:25.881427    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:35:37.286289 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:37.286250    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:35:46.875855 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:46.875819    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:35:56.080163 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:35:56.080130    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:36:06.581392 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:36:06.581351    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:36:15.579350 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:36:15.579310    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:36:24.972043 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:36:24.972007    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:36:38.282965 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:36:38.282925    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:36:47.279911 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:36:47.279869    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:36:56.783169 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:36:56.783132    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:37:07.581562 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:37:07.581524    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:37:19.976006 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:37:19.975957    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:37:28.983490 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:37:28.983454    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:37:39.787285 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:37:39.787253    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:37:50.187614 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:37:50.187572    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:38:01.382724 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:38:01.382687    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:38:14.381696 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:38:14.381656    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:38:23.585416 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:38:23.585373    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:38:33.577579 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:38:33.577538    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:38:43.586389 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:38:43.586311    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:39:15.480363 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:39:15.480326    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:39:26.379860 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:39:26.379821    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:39:59.282249 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:39:59.282199    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:40:10.081478 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:40:10.081441    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:40:43.378782 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:40:43.378734    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:40:54.680614 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:40:54.680578    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:41:27.875776 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:41:27.875742    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:42:53.478638 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:42:53.478596    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:43:03.680851 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:43:03.680816    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:43:13.484870 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:43:13.484774    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:43:34.188208 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:43:34.188174    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:44:11.885464 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:11.885425    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:44:15.302043 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:15.302005    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-88e3c218/e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd"]
Jul 23 23:44:15.302429 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:15.302323    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f975f697-bf96-4c79-bf13-1cb38c55f34a" containerName="main"
Jul 23 23:44:15.302429 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:15.302333    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="f975f697-bf96-4c79-bf13-1cb38c55f34a" containerName="main"
Jul 23 23:44:15.302429 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:15.302357    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a330ffc-24f4-465d-8f02-53943908364e" containerName="main"
Jul 23 23:44:15.302429 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:15.302362    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a330ffc-24f4-465d-8f02-53943908364e" containerName="main"
Jul 23 23:44:15.302429 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:15.302411    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a330ffc-24f4-465d-8f02-53943908364e" containerName="main"
Jul 23 23:44:15.302429 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:15.302422    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="f975f697-bf96-4c79-bf13-1cb38c55f34a" containerName="main"
Jul 23 23:44:15.305357 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:15.305340    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-88e3c218/e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd"
Jul 23 23:44:15.309311 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:15.309285    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-88e3c218\"/\"kube-root-ca.crt\""
Jul 23 23:44:15.309311 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:15.309303    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-88e3c218\"/\"openshift-service-ca.crt\""
Jul 23 23:44:15.309510 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:15.309285    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-88e3c218\"/\"default-dockercfg-9d2kx\""
Jul 23 23:44:15.309510 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:15.309379    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-88e3c218\"/\"e2e-lifecycle-model-88e3c218-kserve-self-signed-certs\""
Jul 23 23:44:15.315503 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:15.315482    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-88e3c218/e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd"]
Jul 23 23:44:15.479953 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:15.479921    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e10c0b2-4e0a-4225-8c8c-cb2590cb7089-model-cache\") pod \"e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd\" (UID: \"5e10c0b2-4e0a-4225-8c8c-cb2590cb7089\") " pod="ai-tenant-e2e-mt-88e3c218/e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd"
Jul 23 23:44:15.479953 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:15.479959    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e10c0b2-4e0a-4225-8c8c-cb2590cb7089-tmp-dir\") pod \"e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd\" (UID: \"5e10c0b2-4e0a-4225-8c8c-cb2590cb7089\") " pod="ai-tenant-e2e-mt-88e3c218/e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd"
Jul 23 23:44:15.480216 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:15.480099    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e10c0b2-4e0a-4225-8c8c-cb2590cb7089-dshm\") pod \"e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd\" (UID: \"5e10c0b2-4e0a-4225-8c8c-cb2590cb7089\") " pod="ai-tenant-e2e-mt-88e3c218/e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd"
Jul 23 23:44:15.480216 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:15.480146    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e10c0b2-4e0a-4225-8c8c-cb2590cb7089-tls-certs\") pod \"e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd\" (UID: \"5e10c0b2-4e0a-4225-8c8c-cb2590cb7089\") " pod="ai-tenant-e2e-mt-88e3c218/e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd"
Jul 23 23:44:15.480327 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:15.480222    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqvhz\" (UniqueName: \"kubernetes.io/projected/5e10c0b2-4e0a-4225-8c8c-cb2590cb7089-kube-api-access-bqvhz\") pod \"e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd\" (UID: \"5e10c0b2-4e0a-4225-8c8c-cb2590cb7089\") " pod="ai-tenant-e2e-mt-88e3c218/e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd"
Jul 23 23:44:15.480327 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:15.480262    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e10c0b2-4e0a-4225-8c8c-cb2590cb7089-home\") pod \"e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd\" (UID: \"5e10c0b2-4e0a-4225-8c8c-cb2590cb7089\") " pod="ai-tenant-e2e-mt-88e3c218/e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd"
Jul 23 23:44:15.581611 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:15.581528    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e10c0b2-4e0a-4225-8c8c-cb2590cb7089-dshm\") pod \"e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd\" (UID: \"5e10c0b2-4e0a-4225-8c8c-cb2590cb7089\") " pod="ai-tenant-e2e-mt-88e3c218/e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd"
Jul 23 23:44:15.581611 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:15.581572    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e10c0b2-4e0a-4225-8c8c-cb2590cb7089-tls-certs\") pod \"e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd\" (UID: \"5e10c0b2-4e0a-4225-8c8c-cb2590cb7089\") " pod="ai-tenant-e2e-mt-88e3c218/e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd"
Jul 23 23:44:15.581611 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:15.581607    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqvhz\" (UniqueName: \"kubernetes.io/projected/5e10c0b2-4e0a-4225-8c8c-cb2590cb7089-kube-api-access-bqvhz\") pod \"e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd\" (UID: \"5e10c0b2-4e0a-4225-8c8c-cb2590cb7089\") " pod="ai-tenant-e2e-mt-88e3c218/e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd"
Jul 23 23:44:15.581887 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:15.581630    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e10c0b2-4e0a-4225-8c8c-cb2590cb7089-home\") pod \"e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd\" (UID: \"5e10c0b2-4e0a-4225-8c8c-cb2590cb7089\") " pod="ai-tenant-e2e-mt-88e3c218/e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd"
Jul 23 23:44:15.581887 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:15.581655    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e10c0b2-4e0a-4225-8c8c-cb2590cb7089-model-cache\") pod \"e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd\" (UID: \"5e10c0b2-4e0a-4225-8c8c-cb2590cb7089\") " pod="ai-tenant-e2e-mt-88e3c218/e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd"
Jul 23 23:44:15.581887 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:15.581671    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e10c0b2-4e0a-4225-8c8c-cb2590cb7089-tmp-dir\") pod \"e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd\" (UID: \"5e10c0b2-4e0a-4225-8c8c-cb2590cb7089\") " pod="ai-tenant-e2e-mt-88e3c218/e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd"
Jul 23 23:44:15.582095 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:15.582063    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e10c0b2-4e0a-4225-8c8c-cb2590cb7089-model-cache\") pod \"e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd\" (UID: \"5e10c0b2-4e0a-4225-8c8c-cb2590cb7089\") " pod="ai-tenant-e2e-mt-88e3c218/e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd"
Jul 23 23:44:15.582151 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:15.582113    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e10c0b2-4e0a-4225-8c8c-cb2590cb7089-home\") pod \"e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd\" (UID: \"5e10c0b2-4e0a-4225-8c8c-cb2590cb7089\") " pod="ai-tenant-e2e-mt-88e3c218/e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd"
Jul 23 23:44:15.582206 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:15.582153    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e10c0b2-4e0a-4225-8c8c-cb2590cb7089-tmp-dir\") pod \"e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd\" (UID: \"5e10c0b2-4e0a-4225-8c8c-cb2590cb7089\") " pod="ai-tenant-e2e-mt-88e3c218/e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd"
Jul 23 23:44:15.583775 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:15.583752    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e10c0b2-4e0a-4225-8c8c-cb2590cb7089-dshm\") pod \"e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd\" (UID: \"5e10c0b2-4e0a-4225-8c8c-cb2590cb7089\") " pod="ai-tenant-e2e-mt-88e3c218/e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd"
Jul 23 23:44:15.584098 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:15.584078    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e10c0b2-4e0a-4225-8c8c-cb2590cb7089-tls-certs\") pod \"e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd\" (UID: \"5e10c0b2-4e0a-4225-8c8c-cb2590cb7089\") " pod="ai-tenant-e2e-mt-88e3c218/e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd"
Jul 23 23:44:15.590831 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:15.590810    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqvhz\" (UniqueName: \"kubernetes.io/projected/5e10c0b2-4e0a-4225-8c8c-cb2590cb7089-kube-api-access-bqvhz\") pod \"e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd\" (UID: \"5e10c0b2-4e0a-4225-8c8c-cb2590cb7089\") " pod="ai-tenant-e2e-mt-88e3c218/e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd"
Jul 23 23:44:15.614349 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:15.614326    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-88e3c218/e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd"
Jul 23 23:44:15.746177 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:15.746145    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-88e3c218/e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd"]
Jul 23 23:44:15.750252 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:44:15.750221    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5e10c0b2_4e0a_4225_8c8c_cb2590cb7089.slice/crio-68e927e3a92e6a83b052f719713f58dc78015b9b27f64f26622e190c9f163fbb WatchSource:0}: Error finding container 68e927e3a92e6a83b052f719713f58dc78015b9b27f64f26622e190c9f163fbb: Status 404 returned error can't find the container with id 68e927e3a92e6a83b052f719713f58dc78015b9b27f64f26622e190c9f163fbb
Jul 23 23:44:15.985605 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:15.985564    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-88e3c218/e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd" event={"ID":"5e10c0b2-4e0a-4225-8c8c-cb2590cb7089","Type":"ContainerStarted","Data":"eee41248a184298207d485c2442c4eb3ba1a97653140de7208828c65dfdcb091"}
Jul 23 23:44:15.985804 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:15.985613    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-88e3c218/e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd" event={"ID":"5e10c0b2-4e0a-4225-8c8c-cb2590cb7089","Type":"ContainerStarted","Data":"68e927e3a92e6a83b052f719713f58dc78015b9b27f64f26622e190c9f163fbb"}
Jul 23 23:44:16.002182 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:16.002134    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-88e3c218/e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd" podStartSLOduration=1.002118699 podStartE2EDuration="1.002118699s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:44:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:44:16.000361268 +0000 UTC m=+1857.744090346" watchObservedRunningTime="2026-07-23 23:44:16.002118699 +0000 UTC m=+1857.745847749"
Jul 23 23:44:20.377059 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:20.377022    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:44:25.614995 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:25.614944    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-88e3c218/e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd"
Jul 23 23:44:25.615405 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:25.615015    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-88e3c218/e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd"
Jul 23 23:44:25.627216 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:25.627185    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-88e3c218/e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd"
Jul 23 23:44:26.045738 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:26.045704    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-88e3c218/e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd"
Jul 23 23:44:34.082833 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:34.082800    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:44:51.675182 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:44:51.675095    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:45:01.980543 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:45:01.980497    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:45:04.957163 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:45:04.957127    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-88e3c218/e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd"]
Jul 23 23:45:19.976300 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:45:19.976237    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-88e3c218/e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd" podUID="5e10c0b2-4e0a-4225-8c8c-cb2590cb7089" containerName="main" containerID="cri-o://eee41248a184298207d485c2442c4eb3ba1a97653140de7208828c65dfdcb091" gracePeriod=45
Jul 23 23:45:20.220668 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:45:20.220643    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-88e3c218/e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd"
Jul 23 23:45:20.310314 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:45:20.310223    2548 generic.go:356] "Generic (PLEG): container finished" podID="5e10c0b2-4e0a-4225-8c8c-cb2590cb7089" containerID="eee41248a184298207d485c2442c4eb3ba1a97653140de7208828c65dfdcb091" exitCode=0
Jul 23 23:45:20.310314 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:45:20.310267    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-88e3c218/e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd"
Jul 23 23:45:20.310504 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:45:20.310313    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-88e3c218/e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd" event={"ID":"5e10c0b2-4e0a-4225-8c8c-cb2590cb7089","Type":"ContainerDied","Data":"eee41248a184298207d485c2442c4eb3ba1a97653140de7208828c65dfdcb091"}
Jul 23 23:45:20.310504 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:45:20.310354    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-88e3c218/e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd" event={"ID":"5e10c0b2-4e0a-4225-8c8c-cb2590cb7089","Type":"ContainerDied","Data":"68e927e3a92e6a83b052f719713f58dc78015b9b27f64f26622e190c9f163fbb"}
Jul 23 23:45:20.310504 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:45:20.310371    2548 scope.go:117] "RemoveContainer" containerID="eee41248a184298207d485c2442c4eb3ba1a97653140de7208828c65dfdcb091"
Jul 23 23:45:20.320424 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:45:20.320404    2548 scope.go:117] "RemoveContainer" containerID="eee41248a184298207d485c2442c4eb3ba1a97653140de7208828c65dfdcb091"
Jul 23 23:45:20.320689 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:45:20.320668    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eee41248a184298207d485c2442c4eb3ba1a97653140de7208828c65dfdcb091\": container with ID starting with eee41248a184298207d485c2442c4eb3ba1a97653140de7208828c65dfdcb091 not found: ID does not exist" containerID="eee41248a184298207d485c2442c4eb3ba1a97653140de7208828c65dfdcb091"
Jul 23 23:45:20.320741 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:45:20.320696    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eee41248a184298207d485c2442c4eb3ba1a97653140de7208828c65dfdcb091"} err="failed to get container status \"eee41248a184298207d485c2442c4eb3ba1a97653140de7208828c65dfdcb091\": rpc error: code = NotFound desc = could not find container \"eee41248a184298207d485c2442c4eb3ba1a97653140de7208828c65dfdcb091\": container with ID starting with eee41248a184298207d485c2442c4eb3ba1a97653140de7208828c65dfdcb091 not found: ID does not exist"
Jul 23 23:45:20.340141 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:45:20.340114    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e10c0b2-4e0a-4225-8c8c-cb2590cb7089-tmp-dir\") pod \"5e10c0b2-4e0a-4225-8c8c-cb2590cb7089\" (UID: \"5e10c0b2-4e0a-4225-8c8c-cb2590cb7089\") "
Jul 23 23:45:20.340279 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:45:20.340165    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e10c0b2-4e0a-4225-8c8c-cb2590cb7089-tls-certs\") pod \"5e10c0b2-4e0a-4225-8c8c-cb2590cb7089\" (UID: \"5e10c0b2-4e0a-4225-8c8c-cb2590cb7089\") "
Jul 23 23:45:20.340279 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:45:20.340196    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e10c0b2-4e0a-4225-8c8c-cb2590cb7089-dshm\") pod \"5e10c0b2-4e0a-4225-8c8c-cb2590cb7089\" (UID: \"5e10c0b2-4e0a-4225-8c8c-cb2590cb7089\") "
Jul 23 23:45:20.340279 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:45:20.340214    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e10c0b2-4e0a-4225-8c8c-cb2590cb7089-model-cache\") pod \"5e10c0b2-4e0a-4225-8c8c-cb2590cb7089\" (UID: \"5e10c0b2-4e0a-4225-8c8c-cb2590cb7089\") "
Jul 23 23:45:20.340279 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:45:20.340231    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e10c0b2-4e0a-4225-8c8c-cb2590cb7089-home\") pod \"5e10c0b2-4e0a-4225-8c8c-cb2590cb7089\" (UID: \"5e10c0b2-4e0a-4225-8c8c-cb2590cb7089\") "
Jul 23 23:45:20.340279 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:45:20.340258    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bqvhz\" (UniqueName: \"kubernetes.io/projected/5e10c0b2-4e0a-4225-8c8c-cb2590cb7089-kube-api-access-bqvhz\") pod \"5e10c0b2-4e0a-4225-8c8c-cb2590cb7089\" (UID: \"5e10c0b2-4e0a-4225-8c8c-cb2590cb7089\") "
Jul 23 23:45:20.340530 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:45:20.340424    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e10c0b2-4e0a-4225-8c8c-cb2590cb7089-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5e10c0b2-4e0a-4225-8c8c-cb2590cb7089" (UID: "5e10c0b2-4e0a-4225-8c8c-cb2590cb7089"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:45:20.340575 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:45:20.340530    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e10c0b2-4e0a-4225-8c8c-cb2590cb7089-model-cache" (OuterVolumeSpecName: "model-cache") pod "5e10c0b2-4e0a-4225-8c8c-cb2590cb7089" (UID: "5e10c0b2-4e0a-4225-8c8c-cb2590cb7089"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:45:20.340575 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:45:20.340546    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e10c0b2-4e0a-4225-8c8c-cb2590cb7089-home" (OuterVolumeSpecName: "home") pod "5e10c0b2-4e0a-4225-8c8c-cb2590cb7089" (UID: "5e10c0b2-4e0a-4225-8c8c-cb2590cb7089"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:45:20.340773 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:45:20.340756    2548 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e10c0b2-4e0a-4225-8c8c-cb2590cb7089-model-cache\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:45:20.340773 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:45:20.340773    2548 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e10c0b2-4e0a-4225-8c8c-cb2590cb7089-home\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:45:20.340917 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:45:20.340782    2548 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e10c0b2-4e0a-4225-8c8c-cb2590cb7089-tmp-dir\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:45:20.342452 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:45:20.342427    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e10c0b2-4e0a-4225-8c8c-cb2590cb7089-dshm" (OuterVolumeSpecName: "dshm") pod "5e10c0b2-4e0a-4225-8c8c-cb2590cb7089" (UID: "5e10c0b2-4e0a-4225-8c8c-cb2590cb7089"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:45:20.342548 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:45:20.342465    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e10c0b2-4e0a-4225-8c8c-cb2590cb7089-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5e10c0b2-4e0a-4225-8c8c-cb2590cb7089" (UID: "5e10c0b2-4e0a-4225-8c8c-cb2590cb7089"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:45:20.342548 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:45:20.342502    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e10c0b2-4e0a-4225-8c8c-cb2590cb7089-kube-api-access-bqvhz" (OuterVolumeSpecName: "kube-api-access-bqvhz") pod "5e10c0b2-4e0a-4225-8c8c-cb2590cb7089" (UID: "5e10c0b2-4e0a-4225-8c8c-cb2590cb7089"). InnerVolumeSpecName "kube-api-access-bqvhz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:45:20.441524 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:45:20.441490    2548 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e10c0b2-4e0a-4225-8c8c-cb2590cb7089-tls-certs\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:45:20.441524 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:45:20.441516    2548 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e10c0b2-4e0a-4225-8c8c-cb2590cb7089-dshm\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:45:20.441524 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:45:20.441526    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqvhz\" (UniqueName: \"kubernetes.io/projected/5e10c0b2-4e0a-4225-8c8c-cb2590cb7089-kube-api-access-bqvhz\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:45:20.631647 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:45:20.631617    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-88e3c218/e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd"]
Jul 23 23:45:20.634798 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:45:20.634772    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-88e3c218/e2e-lifecycle-model-88e3c218-kserve-588b88d7ff-b94hd"]
Jul 23 23:45:20.747739 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:45:20.747711    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e10c0b2-4e0a-4225-8c8c-cb2590cb7089" path="/var/lib/kubelet/pods/5e10c0b2-4e0a-4225-8c8c-cb2590cb7089/volumes"
Jul 23 23:46:13.638686 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:46:13.638597    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:46:51.682009 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:46:51.681959    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:47:00.283880 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:00.283841    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:47:04.688944 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:04.688908    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-c228ab/test-model-c228ab-kserve-6d5c5fc6df-r7fq6"]
Jul 23 23:47:04.689493 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:04.689474    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e10c0b2-4e0a-4225-8c8c-cb2590cb7089" containerName="main"
Jul 23 23:47:04.689551 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:04.689498    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e10c0b2-4e0a-4225-8c8c-cb2590cb7089" containerName="main"
Jul 23 23:47:04.689635 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:04.689623    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e10c0b2-4e0a-4225-8c8c-cb2590cb7089" containerName="main"
Jul 23 23:47:04.692931 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:04.692913    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c228ab/test-model-c228ab-kserve-6d5c5fc6df-r7fq6"
Jul 23 23:47:04.695452 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:04.695429    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-c228ab\"/\"kube-root-ca.crt\""
Jul 23 23:47:04.696644 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:04.696614    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-c228ab\"/\"test-model-c228ab-kserve-self-signed-certs\""
Jul 23 23:47:04.696644 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:04.696613    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-c228ab\"/\"openshift-service-ca.crt\""
Jul 23 23:47:04.696825 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:04.696666    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-c228ab\"/\"default-dockercfg-n4t4g\""
Jul 23 23:47:04.701307 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:04.701282    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-c228ab/test-model-c228ab-kserve-6d5c5fc6df-r7fq6"]
Jul 23 23:47:04.761559 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:04.761523    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1-tls-certs\") pod \"test-model-c228ab-kserve-6d5c5fc6df-r7fq6\" (UID: \"aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1\") " pod="ai-tenant-e2e-inf-a-c228ab/test-model-c228ab-kserve-6d5c5fc6df-r7fq6"
Jul 23 23:47:04.761559 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:04.761563    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1-dshm\") pod \"test-model-c228ab-kserve-6d5c5fc6df-r7fq6\" (UID: \"aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1\") " pod="ai-tenant-e2e-inf-a-c228ab/test-model-c228ab-kserve-6d5c5fc6df-r7fq6"
Jul 23 23:47:04.761772 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:04.761611    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1-tmp-dir\") pod \"test-model-c228ab-kserve-6d5c5fc6df-r7fq6\" (UID: \"aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1\") " pod="ai-tenant-e2e-inf-a-c228ab/test-model-c228ab-kserve-6d5c5fc6df-r7fq6"
Jul 23 23:47:04.761772 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:04.761660    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1-model-cache\") pod \"test-model-c228ab-kserve-6d5c5fc6df-r7fq6\" (UID: \"aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1\") " pod="ai-tenant-e2e-inf-a-c228ab/test-model-c228ab-kserve-6d5c5fc6df-r7fq6"
Jul 23 23:47:04.761772 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:04.761687    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbvk2\" (UniqueName: \"kubernetes.io/projected/aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1-kube-api-access-tbvk2\") pod \"test-model-c228ab-kserve-6d5c5fc6df-r7fq6\" (UID: \"aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1\") " pod="ai-tenant-e2e-inf-a-c228ab/test-model-c228ab-kserve-6d5c5fc6df-r7fq6"
Jul 23 23:47:04.761772 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:04.761725    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1-home\") pod \"test-model-c228ab-kserve-6d5c5fc6df-r7fq6\" (UID: \"aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1\") " pod="ai-tenant-e2e-inf-a-c228ab/test-model-c228ab-kserve-6d5c5fc6df-r7fq6"
Jul 23 23:47:04.862408 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:04.862374    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1-tls-certs\") pod \"test-model-c228ab-kserve-6d5c5fc6df-r7fq6\" (UID: \"aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1\") " pod="ai-tenant-e2e-inf-a-c228ab/test-model-c228ab-kserve-6d5c5fc6df-r7fq6"
Jul 23 23:47:04.862408 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:04.862409    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1-dshm\") pod \"test-model-c228ab-kserve-6d5c5fc6df-r7fq6\" (UID: \"aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1\") " pod="ai-tenant-e2e-inf-a-c228ab/test-model-c228ab-kserve-6d5c5fc6df-r7fq6"
Jul 23 23:47:04.862655 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:04.862428    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1-tmp-dir\") pod \"test-model-c228ab-kserve-6d5c5fc6df-r7fq6\" (UID: \"aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1\") " pod="ai-tenant-e2e-inf-a-c228ab/test-model-c228ab-kserve-6d5c5fc6df-r7fq6"
Jul 23 23:47:04.862655 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:04.862484    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1-model-cache\") pod \"test-model-c228ab-kserve-6d5c5fc6df-r7fq6\" (UID: \"aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1\") " pod="ai-tenant-e2e-inf-a-c228ab/test-model-c228ab-kserve-6d5c5fc6df-r7fq6"
Jul 23 23:47:04.862655 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:04.862516    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbvk2\" (UniqueName: \"kubernetes.io/projected/aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1-kube-api-access-tbvk2\") pod \"test-model-c228ab-kserve-6d5c5fc6df-r7fq6\" (UID: \"aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1\") " pod="ai-tenant-e2e-inf-a-c228ab/test-model-c228ab-kserve-6d5c5fc6df-r7fq6"
Jul 23 23:47:04.862655 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:04.862551    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1-home\") pod \"test-model-c228ab-kserve-6d5c5fc6df-r7fq6\" (UID: \"aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1\") " pod="ai-tenant-e2e-inf-a-c228ab/test-model-c228ab-kserve-6d5c5fc6df-r7fq6"
Jul 23 23:47:04.862940 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:04.862917    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1-model-cache\") pod \"test-model-c228ab-kserve-6d5c5fc6df-r7fq6\" (UID: \"aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1\") " pod="ai-tenant-e2e-inf-a-c228ab/test-model-c228ab-kserve-6d5c5fc6df-r7fq6"
Jul 23 23:47:04.863047 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:04.862942    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1-tmp-dir\") pod \"test-model-c228ab-kserve-6d5c5fc6df-r7fq6\" (UID: \"aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1\") " pod="ai-tenant-e2e-inf-a-c228ab/test-model-c228ab-kserve-6d5c5fc6df-r7fq6"
Jul 23 23:47:04.863047 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:04.862996    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1-home\") pod \"test-model-c228ab-kserve-6d5c5fc6df-r7fq6\" (UID: \"aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1\") " pod="ai-tenant-e2e-inf-a-c228ab/test-model-c228ab-kserve-6d5c5fc6df-r7fq6"
Jul 23 23:47:04.864746 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:04.864724    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1-dshm\") pod \"test-model-c228ab-kserve-6d5c5fc6df-r7fq6\" (UID: \"aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1\") " pod="ai-tenant-e2e-inf-a-c228ab/test-model-c228ab-kserve-6d5c5fc6df-r7fq6"
Jul 23 23:47:04.865012 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:04.864997    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1-tls-certs\") pod \"test-model-c228ab-kserve-6d5c5fc6df-r7fq6\" (UID: \"aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1\") " pod="ai-tenant-e2e-inf-a-c228ab/test-model-c228ab-kserve-6d5c5fc6df-r7fq6"
Jul 23 23:47:04.869542 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:04.869520    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbvk2\" (UniqueName: \"kubernetes.io/projected/aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1-kube-api-access-tbvk2\") pod \"test-model-c228ab-kserve-6d5c5fc6df-r7fq6\" (UID: \"aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1\") " pod="ai-tenant-e2e-inf-a-c228ab/test-model-c228ab-kserve-6d5c5fc6df-r7fq6"
Jul 23 23:47:05.004097 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:05.004010    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c228ab/test-model-c228ab-kserve-6d5c5fc6df-r7fq6"
Jul 23 23:47:05.137706 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:05.137678    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-c228ab/test-model-c228ab-kserve-6d5c5fc6df-r7fq6"]
Jul 23 23:47:05.139275 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:47:05.139249    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaeb53d2f_8727_41ab_90cf_1fb0fd7b35d1.slice/crio-e731a6ca28e25e000faf4feb415d971628e95835149be299d2b78a1cd926969a WatchSource:0}: Error finding container e731a6ca28e25e000faf4feb415d971628e95835149be299d2b78a1cd926969a: Status 404 returned error can't find the container with id e731a6ca28e25e000faf4feb415d971628e95835149be299d2b78a1cd926969a
Jul 23 23:47:05.854284 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:05.854242    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c228ab/test-model-c228ab-kserve-6d5c5fc6df-r7fq6" event={"ID":"aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1","Type":"ContainerStarted","Data":"c187986264b0175896ff8d4129148f0d6d2ee6c4c76980d127f27cf283cae5aa"}
Jul 23 23:47:05.854829 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:05.854298    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c228ab/test-model-c228ab-kserve-6d5c5fc6df-r7fq6" event={"ID":"aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1","Type":"ContainerStarted","Data":"e731a6ca28e25e000faf4feb415d971628e95835149be299d2b78a1cd926969a"}
Jul 23 23:47:05.870236 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:05.870181    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-c228ab/test-model-c228ab-kserve-6d5c5fc6df-r7fq6" podStartSLOduration=1.8701561249999998 podStartE2EDuration="1.870156125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:47:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:47:05.868651814 +0000 UTC m=+2027.612380864" watchObservedRunningTime="2026-07-23 23:47:05.870156125 +0000 UTC m=+2027.613885175"
Jul 23 23:47:09.679619 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:09.679576    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:47:15.004613 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:15.004573    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-c228ab/test-model-c228ab-kserve-6d5c5fc6df-r7fq6"
Jul 23 23:47:15.005076 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:15.004895    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-c228ab/test-model-c228ab-kserve-6d5c5fc6df-r7fq6"
Jul 23 23:47:15.017415 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:15.017385    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-c228ab/test-model-c228ab-kserve-6d5c5fc6df-r7fq6"
Jul 23 23:47:15.917401 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:15.917374    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-c228ab/test-model-c228ab-kserve-6d5c5fc6df-r7fq6"
Jul 23 23:47:20.579626 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:20.579595    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:47:23.498969 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:23.498929    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-d7e557/test-model-d7e557-kserve-bc8ddb879-vdl8s"]
Jul 23 23:47:23.502347 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:23.502325    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d7e557/test-model-d7e557-kserve-bc8ddb879-vdl8s"
Jul 23 23:47:23.505819 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:23.505794    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-d7e557\"/\"openshift-service-ca.crt\""
Jul 23 23:47:23.505985 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:23.505821    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-d7e557\"/\"kube-root-ca.crt\""
Jul 23 23:47:23.505985 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:23.505845    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-d7e557\"/\"test-model-d7e557-kserve-self-signed-certs\""
Jul 23 23:47:23.505985 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:23.505823    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-d7e557\"/\"default-dockercfg-mg8qb\""
Jul 23 23:47:23.512401 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:23.512377    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-d7e557/test-model-d7e557-kserve-bc8ddb879-vdl8s"]
Jul 23 23:47:23.628569 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:23.628532    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95ce5041-1220-4da8-92b3-59217fffa0f0-dshm\") pod \"test-model-d7e557-kserve-bc8ddb879-vdl8s\" (UID: \"95ce5041-1220-4da8-92b3-59217fffa0f0\") " pod="ai-tenant-e2e-inf-b-d7e557/test-model-d7e557-kserve-bc8ddb879-vdl8s"
Jul 23 23:47:23.628569 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:23.628568    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95ce5041-1220-4da8-92b3-59217fffa0f0-tmp-dir\") pod \"test-model-d7e557-kserve-bc8ddb879-vdl8s\" (UID: \"95ce5041-1220-4da8-92b3-59217fffa0f0\") " pod="ai-tenant-e2e-inf-b-d7e557/test-model-d7e557-kserve-bc8ddb879-vdl8s"
Jul 23 23:47:23.628822 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:23.628591    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95ce5041-1220-4da8-92b3-59217fffa0f0-tls-certs\") pod \"test-model-d7e557-kserve-bc8ddb879-vdl8s\" (UID: \"95ce5041-1220-4da8-92b3-59217fffa0f0\") " pod="ai-tenant-e2e-inf-b-d7e557/test-model-d7e557-kserve-bc8ddb879-vdl8s"
Jul 23 23:47:23.628822 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:23.628611    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hq2rs\" (UniqueName: \"kubernetes.io/projected/95ce5041-1220-4da8-92b3-59217fffa0f0-kube-api-access-hq2rs\") pod \"test-model-d7e557-kserve-bc8ddb879-vdl8s\" (UID: \"95ce5041-1220-4da8-92b3-59217fffa0f0\") " pod="ai-tenant-e2e-inf-b-d7e557/test-model-d7e557-kserve-bc8ddb879-vdl8s"
Jul 23 23:47:23.628822 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:23.628686    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95ce5041-1220-4da8-92b3-59217fffa0f0-home\") pod \"test-model-d7e557-kserve-bc8ddb879-vdl8s\" (UID: \"95ce5041-1220-4da8-92b3-59217fffa0f0\") " pod="ai-tenant-e2e-inf-b-d7e557/test-model-d7e557-kserve-bc8ddb879-vdl8s"
Jul 23 23:47:23.628822 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:23.628799    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95ce5041-1220-4da8-92b3-59217fffa0f0-model-cache\") pod \"test-model-d7e557-kserve-bc8ddb879-vdl8s\" (UID: \"95ce5041-1220-4da8-92b3-59217fffa0f0\") " pod="ai-tenant-e2e-inf-b-d7e557/test-model-d7e557-kserve-bc8ddb879-vdl8s"
Jul 23 23:47:23.729927 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:23.729882    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95ce5041-1220-4da8-92b3-59217fffa0f0-model-cache\") pod \"test-model-d7e557-kserve-bc8ddb879-vdl8s\" (UID: \"95ce5041-1220-4da8-92b3-59217fffa0f0\") " pod="ai-tenant-e2e-inf-b-d7e557/test-model-d7e557-kserve-bc8ddb879-vdl8s"
Jul 23 23:47:23.730118 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:23.729959    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95ce5041-1220-4da8-92b3-59217fffa0f0-dshm\") pod \"test-model-d7e557-kserve-bc8ddb879-vdl8s\" (UID: \"95ce5041-1220-4da8-92b3-59217fffa0f0\") " pod="ai-tenant-e2e-inf-b-d7e557/test-model-d7e557-kserve-bc8ddb879-vdl8s"
Jul 23 23:47:23.730118 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:23.730015    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95ce5041-1220-4da8-92b3-59217fffa0f0-tmp-dir\") pod \"test-model-d7e557-kserve-bc8ddb879-vdl8s\" (UID: \"95ce5041-1220-4da8-92b3-59217fffa0f0\") " pod="ai-tenant-e2e-inf-b-d7e557/test-model-d7e557-kserve-bc8ddb879-vdl8s"
Jul 23 23:47:23.730118 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:23.730043    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95ce5041-1220-4da8-92b3-59217fffa0f0-tls-certs\") pod \"test-model-d7e557-kserve-bc8ddb879-vdl8s\" (UID: \"95ce5041-1220-4da8-92b3-59217fffa0f0\") " pod="ai-tenant-e2e-inf-b-d7e557/test-model-d7e557-kserve-bc8ddb879-vdl8s"
Jul 23 23:47:23.730118 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:23.730075    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hq2rs\" (UniqueName: \"kubernetes.io/projected/95ce5041-1220-4da8-92b3-59217fffa0f0-kube-api-access-hq2rs\") pod \"test-model-d7e557-kserve-bc8ddb879-vdl8s\" (UID: \"95ce5041-1220-4da8-92b3-59217fffa0f0\") " pod="ai-tenant-e2e-inf-b-d7e557/test-model-d7e557-kserve-bc8ddb879-vdl8s"
Jul 23 23:47:23.730118 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:23.730100    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95ce5041-1220-4da8-92b3-59217fffa0f0-home\") pod \"test-model-d7e557-kserve-bc8ddb879-vdl8s\" (UID: \"95ce5041-1220-4da8-92b3-59217fffa0f0\") " pod="ai-tenant-e2e-inf-b-d7e557/test-model-d7e557-kserve-bc8ddb879-vdl8s"
Jul 23 23:47:23.730400 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:23.730380    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95ce5041-1220-4da8-92b3-59217fffa0f0-model-cache\") pod \"test-model-d7e557-kserve-bc8ddb879-vdl8s\" (UID: \"95ce5041-1220-4da8-92b3-59217fffa0f0\") " pod="ai-tenant-e2e-inf-b-d7e557/test-model-d7e557-kserve-bc8ddb879-vdl8s"
Jul 23 23:47:23.730474 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:23.730436    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95ce5041-1220-4da8-92b3-59217fffa0f0-tmp-dir\") pod \"test-model-d7e557-kserve-bc8ddb879-vdl8s\" (UID: \"95ce5041-1220-4da8-92b3-59217fffa0f0\") " pod="ai-tenant-e2e-inf-b-d7e557/test-model-d7e557-kserve-bc8ddb879-vdl8s"
Jul 23 23:47:23.730575 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:23.730556    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95ce5041-1220-4da8-92b3-59217fffa0f0-home\") pod \"test-model-d7e557-kserve-bc8ddb879-vdl8s\" (UID: \"95ce5041-1220-4da8-92b3-59217fffa0f0\") " pod="ai-tenant-e2e-inf-b-d7e557/test-model-d7e557-kserve-bc8ddb879-vdl8s"
Jul 23 23:47:23.732366 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:23.732345    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95ce5041-1220-4da8-92b3-59217fffa0f0-dshm\") pod \"test-model-d7e557-kserve-bc8ddb879-vdl8s\" (UID: \"95ce5041-1220-4da8-92b3-59217fffa0f0\") " pod="ai-tenant-e2e-inf-b-d7e557/test-model-d7e557-kserve-bc8ddb879-vdl8s"
Jul 23 23:47:23.732721 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:23.732700    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95ce5041-1220-4da8-92b3-59217fffa0f0-tls-certs\") pod \"test-model-d7e557-kserve-bc8ddb879-vdl8s\" (UID: \"95ce5041-1220-4da8-92b3-59217fffa0f0\") " pod="ai-tenant-e2e-inf-b-d7e557/test-model-d7e557-kserve-bc8ddb879-vdl8s"
Jul 23 23:47:23.740699 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:23.740671    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hq2rs\" (UniqueName: \"kubernetes.io/projected/95ce5041-1220-4da8-92b3-59217fffa0f0-kube-api-access-hq2rs\") pod \"test-model-d7e557-kserve-bc8ddb879-vdl8s\" (UID: \"95ce5041-1220-4da8-92b3-59217fffa0f0\") " pod="ai-tenant-e2e-inf-b-d7e557/test-model-d7e557-kserve-bc8ddb879-vdl8s"
Jul 23 23:47:23.812475 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:23.812381    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d7e557/test-model-d7e557-kserve-bc8ddb879-vdl8s"
Jul 23 23:47:23.943372 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:23.943329    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-d7e557/test-model-d7e557-kserve-bc8ddb879-vdl8s"]
Jul 23 23:47:23.946336 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:47:23.946307    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod95ce5041_1220_4da8_92b3_59217fffa0f0.slice/crio-289b8d218d80673443a478a2c6e38352c1540efb23731112e8ae4e2b2e52413c WatchSource:0}: Error finding container 289b8d218d80673443a478a2c6e38352c1540efb23731112e8ae4e2b2e52413c: Status 404 returned error can't find the container with id 289b8d218d80673443a478a2c6e38352c1540efb23731112e8ae4e2b2e52413c
Jul 23 23:47:24.950115 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:24.950078    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d7e557/test-model-d7e557-kserve-bc8ddb879-vdl8s" event={"ID":"95ce5041-1220-4da8-92b3-59217fffa0f0","Type":"ContainerStarted","Data":"767a4f8f0ad6978296e117798ad9a162ea6187ffb7ae3cff7b2b7c2503780294"}
Jul 23 23:47:24.950523 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:24.950128    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d7e557/test-model-d7e557-kserve-bc8ddb879-vdl8s" event={"ID":"95ce5041-1220-4da8-92b3-59217fffa0f0","Type":"ContainerStarted","Data":"289b8d218d80673443a478a2c6e38352c1540efb23731112e8ae4e2b2e52413c"}
Jul 23 23:47:24.965895 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:24.965766    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-d7e557/test-model-d7e557-kserve-bc8ddb879-vdl8s" podStartSLOduration=1.96574502 podStartE2EDuration="1.96574502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:47:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:47:24.965478605 +0000 UTC m=+2046.709207655" watchObservedRunningTime="2026-07-23 23:47:24.96574502 +0000 UTC m=+2046.709474070"
Jul 23 23:47:30.886388 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:30.886349    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:47:33.813307 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:33.813266    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-d7e557/test-model-d7e557-kserve-bc8ddb879-vdl8s"
Jul 23 23:47:33.813752 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:33.813334    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-d7e557/test-model-d7e557-kserve-bc8ddb879-vdl8s"
Jul 23 23:47:33.826045 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:33.826011    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-d7e557/test-model-d7e557-kserve-bc8ddb879-vdl8s"
Jul 23 23:47:34.007427 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:47:34.007399    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-d7e557/test-model-d7e557-kserve-bc8ddb879-vdl8s"
Jul 23 23:48:08.451351 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:08.451272    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-c228ab/test-model-c228ab-kserve-6d5c5fc6df-r7fq6"]
Jul 23 23:48:09.350223 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:09.350184    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-d7e557/test-model-d7e557-kserve-bc8ddb879-vdl8s"]
Jul 23 23:48:13.679165 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:13.679134    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:48:23.470264 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:23.470201    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-c228ab/test-model-c228ab-kserve-6d5c5fc6df-r7fq6" podUID="aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1" containerName="main" containerID="cri-o://c187986264b0175896ff8d4129148f0d6d2ee6c4c76980d127f27cf283cae5aa" gracePeriod=45
Jul 23 23:48:23.720522 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:23.720462    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c228ab/test-model-c228ab-kserve-6d5c5fc6df-r7fq6"
Jul 23 23:48:23.866708 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:23.866677    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1-model-cache\") pod \"aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1\" (UID: \"aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1\") "
Jul 23 23:48:23.866893 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:23.866721    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1-tls-certs\") pod \"aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1\" (UID: \"aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1\") "
Jul 23 23:48:23.866893 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:23.866748    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tbvk2\" (UniqueName: \"kubernetes.io/projected/aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1-kube-api-access-tbvk2\") pod \"aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1\" (UID: \"aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1\") "
Jul 23 23:48:23.866893 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:23.866819    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1-tmp-dir\") pod \"aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1\" (UID: \"aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1\") "
Jul 23 23:48:23.866893 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:23.866859    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1-dshm\") pod \"aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1\" (UID: \"aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1\") "
Jul 23 23:48:23.866893 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:23.866878    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1-home\") pod \"aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1\" (UID: \"aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1\") "
Jul 23 23:48:23.867185 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:23.867074    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1-model-cache" (OuterVolumeSpecName: "model-cache") pod "aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1" (UID: "aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:48:23.867185 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:23.867119    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1" (UID: "aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:48:23.867291 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:23.867227    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1-home" (OuterVolumeSpecName: "home") pod "aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1" (UID: "aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:48:23.869147 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:23.869118    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1-dshm" (OuterVolumeSpecName: "dshm") pod "aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1" (UID: "aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:48:23.869240 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:23.869207    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1" (UID: "aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:48:23.869293 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:23.869258    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1-kube-api-access-tbvk2" (OuterVolumeSpecName: "kube-api-access-tbvk2") pod "aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1" (UID: "aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1"). InnerVolumeSpecName "kube-api-access-tbvk2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:48:23.967882 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:23.967848    2548 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1-dshm\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:48:23.967882 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:23.967877    2548 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1-home\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:48:23.967882 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:23.967886    2548 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1-model-cache\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:48:23.968145 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:23.967897    2548 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1-tls-certs\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:48:23.968145 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:23.967905    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tbvk2\" (UniqueName: \"kubernetes.io/projected/aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1-kube-api-access-tbvk2\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:48:23.968145 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:23.967914    2548 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1-tmp-dir\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:48:24.247462 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:24.247425    2548 generic.go:356] "Generic (PLEG): container finished" podID="aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1" containerID="c187986264b0175896ff8d4129148f0d6d2ee6c4c76980d127f27cf283cae5aa" exitCode=0
Jul 23 23:48:24.247639 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:24.247475    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c228ab/test-model-c228ab-kserve-6d5c5fc6df-r7fq6"
Jul 23 23:48:24.247639 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:24.247514    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c228ab/test-model-c228ab-kserve-6d5c5fc6df-r7fq6" event={"ID":"aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1","Type":"ContainerDied","Data":"c187986264b0175896ff8d4129148f0d6d2ee6c4c76980d127f27cf283cae5aa"}
Jul 23 23:48:24.247639 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:24.247557    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c228ab/test-model-c228ab-kserve-6d5c5fc6df-r7fq6" event={"ID":"aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1","Type":"ContainerDied","Data":"e731a6ca28e25e000faf4feb415d971628e95835149be299d2b78a1cd926969a"}
Jul 23 23:48:24.247639 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:24.247573    2548 scope.go:117] "RemoveContainer" containerID="c187986264b0175896ff8d4129148f0d6d2ee6c4c76980d127f27cf283cae5aa"
Jul 23 23:48:24.256925 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:24.256904    2548 scope.go:117] "RemoveContainer" containerID="c187986264b0175896ff8d4129148f0d6d2ee6c4c76980d127f27cf283cae5aa"
Jul 23 23:48:24.257197 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:48:24.257178    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c187986264b0175896ff8d4129148f0d6d2ee6c4c76980d127f27cf283cae5aa\": container with ID starting with c187986264b0175896ff8d4129148f0d6d2ee6c4c76980d127f27cf283cae5aa not found: ID does not exist" containerID="c187986264b0175896ff8d4129148f0d6d2ee6c4c76980d127f27cf283cae5aa"
Jul 23 23:48:24.257250 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:24.257204    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c187986264b0175896ff8d4129148f0d6d2ee6c4c76980d127f27cf283cae5aa"} err="failed to get container status \"c187986264b0175896ff8d4129148f0d6d2ee6c4c76980d127f27cf283cae5aa\": rpc error: code = NotFound desc = could not find container \"c187986264b0175896ff8d4129148f0d6d2ee6c4c76980d127f27cf283cae5aa\": container with ID starting with c187986264b0175896ff8d4129148f0d6d2ee6c4c76980d127f27cf283cae5aa not found: ID does not exist"
Jul 23 23:48:24.268060 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:24.268037    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-c228ab/test-model-c228ab-kserve-6d5c5fc6df-r7fq6"]
Jul 23 23:48:24.271914 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:24.271891    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-c228ab/test-model-c228ab-kserve-6d5c5fc6df-r7fq6"]
Jul 23 23:48:24.370000 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:24.369948    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-d7e557/test-model-d7e557-kserve-bc8ddb879-vdl8s" podUID="95ce5041-1220-4da8-92b3-59217fffa0f0" containerName="main" containerID="cri-o://767a4f8f0ad6978296e117798ad9a162ea6187ffb7ae3cff7b2b7c2503780294" gracePeriod=45
Jul 23 23:48:24.621082 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:24.621058    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d7e557/test-model-d7e557-kserve-bc8ddb879-vdl8s"
Jul 23 23:48:24.747795 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:24.747764    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1" path="/var/lib/kubelet/pods/aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1/volumes"
Jul 23 23:48:24.774182 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:24.774151    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95ce5041-1220-4da8-92b3-59217fffa0f0-model-cache\") pod \"95ce5041-1220-4da8-92b3-59217fffa0f0\" (UID: \"95ce5041-1220-4da8-92b3-59217fffa0f0\") "
Jul 23 23:48:24.774334 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:24.774193    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95ce5041-1220-4da8-92b3-59217fffa0f0-tls-certs\") pod \"95ce5041-1220-4da8-92b3-59217fffa0f0\" (UID: \"95ce5041-1220-4da8-92b3-59217fffa0f0\") "
Jul 23 23:48:24.774334 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:24.774225    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hq2rs\" (UniqueName: \"kubernetes.io/projected/95ce5041-1220-4da8-92b3-59217fffa0f0-kube-api-access-hq2rs\") pod \"95ce5041-1220-4da8-92b3-59217fffa0f0\" (UID: \"95ce5041-1220-4da8-92b3-59217fffa0f0\") "
Jul 23 23:48:24.774334 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:24.774259    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95ce5041-1220-4da8-92b3-59217fffa0f0-dshm\") pod \"95ce5041-1220-4da8-92b3-59217fffa0f0\" (UID: \"95ce5041-1220-4da8-92b3-59217fffa0f0\") "
Jul 23 23:48:24.774334 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:24.774275    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95ce5041-1220-4da8-92b3-59217fffa0f0-home\") pod \"95ce5041-1220-4da8-92b3-59217fffa0f0\" (UID: \"95ce5041-1220-4da8-92b3-59217fffa0f0\") "
Jul 23 23:48:24.774334 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:24.774300    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95ce5041-1220-4da8-92b3-59217fffa0f0-tmp-dir\") pod \"95ce5041-1220-4da8-92b3-59217fffa0f0\" (UID: \"95ce5041-1220-4da8-92b3-59217fffa0f0\") "
Jul 23 23:48:24.774592 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:24.774473    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95ce5041-1220-4da8-92b3-59217fffa0f0-model-cache" (OuterVolumeSpecName: "model-cache") pod "95ce5041-1220-4da8-92b3-59217fffa0f0" (UID: "95ce5041-1220-4da8-92b3-59217fffa0f0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:48:24.774592 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:24.774524    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95ce5041-1220-4da8-92b3-59217fffa0f0-home" (OuterVolumeSpecName: "home") pod "95ce5041-1220-4da8-92b3-59217fffa0f0" (UID: "95ce5041-1220-4da8-92b3-59217fffa0f0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:48:24.774669 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:24.774613    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95ce5041-1220-4da8-92b3-59217fffa0f0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "95ce5041-1220-4da8-92b3-59217fffa0f0" (UID: "95ce5041-1220-4da8-92b3-59217fffa0f0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:48:24.774721 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:24.774686    2548 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95ce5041-1220-4da8-92b3-59217fffa0f0-model-cache\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:48:24.774721 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:24.774706    2548 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95ce5041-1220-4da8-92b3-59217fffa0f0-home\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:48:24.776518 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:24.776484    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95ce5041-1220-4da8-92b3-59217fffa0f0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "95ce5041-1220-4da8-92b3-59217fffa0f0" (UID: "95ce5041-1220-4da8-92b3-59217fffa0f0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:48:24.776622 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:24.776519    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95ce5041-1220-4da8-92b3-59217fffa0f0-dshm" (OuterVolumeSpecName: "dshm") pod "95ce5041-1220-4da8-92b3-59217fffa0f0" (UID: "95ce5041-1220-4da8-92b3-59217fffa0f0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:48:24.776622 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:24.776541    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95ce5041-1220-4da8-92b3-59217fffa0f0-kube-api-access-hq2rs" (OuterVolumeSpecName: "kube-api-access-hq2rs") pod "95ce5041-1220-4da8-92b3-59217fffa0f0" (UID: "95ce5041-1220-4da8-92b3-59217fffa0f0"). InnerVolumeSpecName "kube-api-access-hq2rs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:48:24.875761 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:24.875670    2548 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95ce5041-1220-4da8-92b3-59217fffa0f0-tls-certs\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:48:24.875761 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:24.875700    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hq2rs\" (UniqueName: \"kubernetes.io/projected/95ce5041-1220-4da8-92b3-59217fffa0f0-kube-api-access-hq2rs\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:48:24.875761 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:24.875714    2548 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95ce5041-1220-4da8-92b3-59217fffa0f0-dshm\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:48:24.875761 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:24.875726    2548 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95ce5041-1220-4da8-92b3-59217fffa0f0-tmp-dir\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:48:25.254695 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:25.254658    2548 generic.go:356] "Generic (PLEG): container finished" podID="95ce5041-1220-4da8-92b3-59217fffa0f0" containerID="767a4f8f0ad6978296e117798ad9a162ea6187ffb7ae3cff7b2b7c2503780294" exitCode=0
Jul 23 23:48:25.254886 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:25.254692    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d7e557/test-model-d7e557-kserve-bc8ddb879-vdl8s" event={"ID":"95ce5041-1220-4da8-92b3-59217fffa0f0","Type":"ContainerDied","Data":"767a4f8f0ad6978296e117798ad9a162ea6187ffb7ae3cff7b2b7c2503780294"}
Jul 23 23:48:25.254886 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:25.254710    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d7e557/test-model-d7e557-kserve-bc8ddb879-vdl8s"
Jul 23 23:48:25.254886 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:25.254744    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d7e557/test-model-d7e557-kserve-bc8ddb879-vdl8s" event={"ID":"95ce5041-1220-4da8-92b3-59217fffa0f0","Type":"ContainerDied","Data":"289b8d218d80673443a478a2c6e38352c1540efb23731112e8ae4e2b2e52413c"}
Jul 23 23:48:25.254886 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:25.254768    2548 scope.go:117] "RemoveContainer" containerID="767a4f8f0ad6978296e117798ad9a162ea6187ffb7ae3cff7b2b7c2503780294"
Jul 23 23:48:25.264309 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:25.264291    2548 scope.go:117] "RemoveContainer" containerID="767a4f8f0ad6978296e117798ad9a162ea6187ffb7ae3cff7b2b7c2503780294"
Jul 23 23:48:25.264564 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:48:25.264545    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"767a4f8f0ad6978296e117798ad9a162ea6187ffb7ae3cff7b2b7c2503780294\": container with ID starting with 767a4f8f0ad6978296e117798ad9a162ea6187ffb7ae3cff7b2b7c2503780294 not found: ID does not exist" containerID="767a4f8f0ad6978296e117798ad9a162ea6187ffb7ae3cff7b2b7c2503780294"
Jul 23 23:48:25.264631 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:25.264572    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"767a4f8f0ad6978296e117798ad9a162ea6187ffb7ae3cff7b2b7c2503780294"} err="failed to get container status \"767a4f8f0ad6978296e117798ad9a162ea6187ffb7ae3cff7b2b7c2503780294\": rpc error: code = NotFound desc = could not find container \"767a4f8f0ad6978296e117798ad9a162ea6187ffb7ae3cff7b2b7c2503780294\": container with ID starting with 767a4f8f0ad6978296e117798ad9a162ea6187ffb7ae3cff7b2b7c2503780294 not found: ID does not exist"
Jul 23 23:48:25.275650 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:25.275632    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-d7e557/test-model-d7e557-kserve-bc8ddb879-vdl8s"]
Jul 23 23:48:25.279363 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:25.279341    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-d7e557/test-model-d7e557-kserve-bc8ddb879-vdl8s"]
Jul 23 23:48:25.383220 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:25.383189    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:48:26.747405 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:26.747373    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95ce5041-1220-4da8-92b3-59217fffa0f0" path="/var/lib/kubelet/pods/95ce5041-1220-4da8-92b3-59217fffa0f0/volumes"
Jul 23 23:48:44.978189 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:48:44.978147    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:49:03.779620 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:49:03.779577    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:49:14.880995 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:49:14.880899    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:49:35.386582 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:49:35.386545    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:50:03.681642 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:03.681601    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:50:22.083331 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:22.083294    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:50:32.979924 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:32.979883    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:50:46.400163 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:46.400085    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-867102/auth-test-model-867102-kserve-65457c7b79-v4ngj"]
Jul 23 23:50:46.400539 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:46.400443    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1" containerName="main"
Jul 23 23:50:46.400539 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:46.400454    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1" containerName="main"
Jul 23 23:50:46.400539 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:46.400468    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95ce5041-1220-4da8-92b3-59217fffa0f0" containerName="main"
Jul 23 23:50:46.400539 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:46.400473    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="95ce5041-1220-4da8-92b3-59217fffa0f0" containerName="main"
Jul 23 23:50:46.400539 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:46.400529    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="aeb53d2f-8727-41ab-90cf-1fb0fd7b35d1" containerName="main"
Jul 23 23:50:46.400539 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:46.400539    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="95ce5041-1220-4da8-92b3-59217fffa0f0" containerName="main"
Jul 23 23:50:46.403484 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:46.403468    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-867102/auth-test-model-867102-kserve-65457c7b79-v4ngj"
Jul 23 23:50:46.406148 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:46.406119    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-867102\"/\"kube-root-ca.crt\""
Jul 23 23:50:46.407599 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:46.407568    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-867102\"/\"default-dockercfg-qlbld\""
Jul 23 23:50:46.407733 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:46.407600    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-867102\"/\"openshift-service-ca.crt\""
Jul 23 23:50:46.407733 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:46.407638    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-867102\"/\"auth-test-model-867102-kserve-self-signed-certs\""
Jul 23 23:50:46.411567 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:46.411521    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-867102/auth-test-model-867102-kserve-65457c7b79-v4ngj"]
Jul 23 23:50:46.499620 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:46.499582    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1450704-30a2-41bd-a6c3-e10688cea21a-tls-certs\") pod \"auth-test-model-867102-kserve-65457c7b79-v4ngj\" (UID: \"d1450704-30a2-41bd-a6c3-e10688cea21a\") " pod="ai-tenant-e2e-shared-a-867102/auth-test-model-867102-kserve-65457c7b79-v4ngj"
Jul 23 23:50:46.499620 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:46.499622    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbc72\" (UniqueName: \"kubernetes.io/projected/d1450704-30a2-41bd-a6c3-e10688cea21a-kube-api-access-zbc72\") pod \"auth-test-model-867102-kserve-65457c7b79-v4ngj\" (UID: \"d1450704-30a2-41bd-a6c3-e10688cea21a\") " pod="ai-tenant-e2e-shared-a-867102/auth-test-model-867102-kserve-65457c7b79-v4ngj"
Jul 23 23:50:46.499847 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:46.499663    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1450704-30a2-41bd-a6c3-e10688cea21a-home\") pod \"auth-test-model-867102-kserve-65457c7b79-v4ngj\" (UID: \"d1450704-30a2-41bd-a6c3-e10688cea21a\") " pod="ai-tenant-e2e-shared-a-867102/auth-test-model-867102-kserve-65457c7b79-v4ngj"
Jul 23 23:50:46.499847 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:46.499695    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1450704-30a2-41bd-a6c3-e10688cea21a-dshm\") pod \"auth-test-model-867102-kserve-65457c7b79-v4ngj\" (UID: \"d1450704-30a2-41bd-a6c3-e10688cea21a\") " pod="ai-tenant-e2e-shared-a-867102/auth-test-model-867102-kserve-65457c7b79-v4ngj"
Jul 23 23:50:46.499847 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:46.499735    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1450704-30a2-41bd-a6c3-e10688cea21a-model-cache\") pod \"auth-test-model-867102-kserve-65457c7b79-v4ngj\" (UID: \"d1450704-30a2-41bd-a6c3-e10688cea21a\") " pod="ai-tenant-e2e-shared-a-867102/auth-test-model-867102-kserve-65457c7b79-v4ngj"
Jul 23 23:50:46.499847 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:46.499756    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1450704-30a2-41bd-a6c3-e10688cea21a-tmp-dir\") pod \"auth-test-model-867102-kserve-65457c7b79-v4ngj\" (UID: \"d1450704-30a2-41bd-a6c3-e10688cea21a\") " pod="ai-tenant-e2e-shared-a-867102/auth-test-model-867102-kserve-65457c7b79-v4ngj"
Jul 23 23:50:46.601154 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:46.601097    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1450704-30a2-41bd-a6c3-e10688cea21a-tls-certs\") pod \"auth-test-model-867102-kserve-65457c7b79-v4ngj\" (UID: \"d1450704-30a2-41bd-a6c3-e10688cea21a\") " pod="ai-tenant-e2e-shared-a-867102/auth-test-model-867102-kserve-65457c7b79-v4ngj"
Jul 23 23:50:46.601154 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:46.601152    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbc72\" (UniqueName: \"kubernetes.io/projected/d1450704-30a2-41bd-a6c3-e10688cea21a-kube-api-access-zbc72\") pod \"auth-test-model-867102-kserve-65457c7b79-v4ngj\" (UID: \"d1450704-30a2-41bd-a6c3-e10688cea21a\") " pod="ai-tenant-e2e-shared-a-867102/auth-test-model-867102-kserve-65457c7b79-v4ngj"
Jul 23 23:50:46.601501 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:46.601187    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1450704-30a2-41bd-a6c3-e10688cea21a-home\") pod \"auth-test-model-867102-kserve-65457c7b79-v4ngj\" (UID: \"d1450704-30a2-41bd-a6c3-e10688cea21a\") " pod="ai-tenant-e2e-shared-a-867102/auth-test-model-867102-kserve-65457c7b79-v4ngj"
Jul 23 23:50:46.601501 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:46.601221    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1450704-30a2-41bd-a6c3-e10688cea21a-dshm\") pod \"auth-test-model-867102-kserve-65457c7b79-v4ngj\" (UID: \"d1450704-30a2-41bd-a6c3-e10688cea21a\") " pod="ai-tenant-e2e-shared-a-867102/auth-test-model-867102-kserve-65457c7b79-v4ngj"
Jul 23 23:50:46.601501 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:46.601256    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1450704-30a2-41bd-a6c3-e10688cea21a-model-cache\") pod \"auth-test-model-867102-kserve-65457c7b79-v4ngj\" (UID: \"d1450704-30a2-41bd-a6c3-e10688cea21a\") " pod="ai-tenant-e2e-shared-a-867102/auth-test-model-867102-kserve-65457c7b79-v4ngj"
Jul 23 23:50:46.601501 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:46.601283    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1450704-30a2-41bd-a6c3-e10688cea21a-tmp-dir\") pod \"auth-test-model-867102-kserve-65457c7b79-v4ngj\" (UID: \"d1450704-30a2-41bd-a6c3-e10688cea21a\") " pod="ai-tenant-e2e-shared-a-867102/auth-test-model-867102-kserve-65457c7b79-v4ngj"
Jul 23 23:50:46.601730 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:46.601710    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1450704-30a2-41bd-a6c3-e10688cea21a-home\") pod \"auth-test-model-867102-kserve-65457c7b79-v4ngj\" (UID: \"d1450704-30a2-41bd-a6c3-e10688cea21a\") " pod="ai-tenant-e2e-shared-a-867102/auth-test-model-867102-kserve-65457c7b79-v4ngj"
Jul 23 23:50:46.601789 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:46.601713    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1450704-30a2-41bd-a6c3-e10688cea21a-model-cache\") pod \"auth-test-model-867102-kserve-65457c7b79-v4ngj\" (UID: \"d1450704-30a2-41bd-a6c3-e10688cea21a\") " pod="ai-tenant-e2e-shared-a-867102/auth-test-model-867102-kserve-65457c7b79-v4ngj"
Jul 23 23:50:46.601789 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:46.601774    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1450704-30a2-41bd-a6c3-e10688cea21a-tmp-dir\") pod \"auth-test-model-867102-kserve-65457c7b79-v4ngj\" (UID: \"d1450704-30a2-41bd-a6c3-e10688cea21a\") " pod="ai-tenant-e2e-shared-a-867102/auth-test-model-867102-kserve-65457c7b79-v4ngj"
Jul 23 23:50:46.603559 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:46.603538    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1450704-30a2-41bd-a6c3-e10688cea21a-dshm\") pod \"auth-test-model-867102-kserve-65457c7b79-v4ngj\" (UID: \"d1450704-30a2-41bd-a6c3-e10688cea21a\") " pod="ai-tenant-e2e-shared-a-867102/auth-test-model-867102-kserve-65457c7b79-v4ngj"
Jul 23 23:50:46.603688 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:46.603673    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1450704-30a2-41bd-a6c3-e10688cea21a-tls-certs\") pod \"auth-test-model-867102-kserve-65457c7b79-v4ngj\" (UID: \"d1450704-30a2-41bd-a6c3-e10688cea21a\") " pod="ai-tenant-e2e-shared-a-867102/auth-test-model-867102-kserve-65457c7b79-v4ngj"
Jul 23 23:50:46.609191 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:46.609173    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbc72\" (UniqueName: \"kubernetes.io/projected/d1450704-30a2-41bd-a6c3-e10688cea21a-kube-api-access-zbc72\") pod \"auth-test-model-867102-kserve-65457c7b79-v4ngj\" (UID: \"d1450704-30a2-41bd-a6c3-e10688cea21a\") " pod="ai-tenant-e2e-shared-a-867102/auth-test-model-867102-kserve-65457c7b79-v4ngj"
Jul 23 23:50:46.715091 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:46.714996    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-867102/auth-test-model-867102-kserve-65457c7b79-v4ngj"
Jul 23 23:50:46.845013 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:46.844970    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-867102/auth-test-model-867102-kserve-65457c7b79-v4ngj"]
Jul 23 23:50:46.846879 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:50:46.846849    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd1450704_30a2_41bd_a6c3_e10688cea21a.slice/crio-c9d0ab87440ceab2a6df7f9316fc1b8a2756749a3a712b4f8c16a07ebe3d30aa WatchSource:0}: Error finding container c9d0ab87440ceab2a6df7f9316fc1b8a2756749a3a712b4f8c16a07ebe3d30aa: Status 404 returned error can't find the container with id c9d0ab87440ceab2a6df7f9316fc1b8a2756749a3a712b4f8c16a07ebe3d30aa
Jul 23 23:50:46.946859 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:46.946823    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-867102/auth-test-model-867102-kserve-65457c7b79-v4ngj" event={"ID":"d1450704-30a2-41bd-a6c3-e10688cea21a","Type":"ContainerStarted","Data":"a20fd0fa2a1ad94a64643952f1c0d289fea037a9433599d8a2f8e2494c0c7bfb"}
Jul 23 23:50:46.947064 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:46.946870    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-867102/auth-test-model-867102-kserve-65457c7b79-v4ngj" event={"ID":"d1450704-30a2-41bd-a6c3-e10688cea21a","Type":"ContainerStarted","Data":"c9d0ab87440ceab2a6df7f9316fc1b8a2756749a3a712b4f8c16a07ebe3d30aa"}
Jul 23 23:50:46.962400 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:46.962344    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-867102/auth-test-model-867102-kserve-65457c7b79-v4ngj" podStartSLOduration=0.962323638 podStartE2EDuration="962.323638ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:50:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:50:46.961775499 +0000 UTC m=+2248.705504582" watchObservedRunningTime="2026-07-23 23:50:46.962323638 +0000 UTC m=+2248.706052688"
Jul 23 23:50:49.675079 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:49.675042    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:50:56.716057 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:56.716025    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-867102/auth-test-model-867102-kserve-65457c7b79-v4ngj"
Jul 23 23:50:56.716604 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:56.716293    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-867102/auth-test-model-867102-kserve-65457c7b79-v4ngj"
Jul 23 23:50:56.731566 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:56.731534    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-867102/auth-test-model-867102-kserve-65457c7b79-v4ngj"
Jul 23 23:50:57.012420 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:50:57.012338    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-867102/auth-test-model-867102-kserve-65457c7b79-v4ngj"
Jul 23 23:51:13.810781 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:51:13.810744    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-fc333b/auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq"]
Jul 23 23:51:13.814052 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:51:13.814028    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fc333b/auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq"
Jul 23 23:51:13.817505 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:51:13.817480    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-fc333b\"/\"default-dockercfg-2dfzf\""
Jul 23 23:51:13.817632 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:51:13.817507    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-fc333b\"/\"kube-root-ca.crt\""
Jul 23 23:51:13.817686 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:51:13.817633    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-fc333b\"/\"auth-test-model-fc333b-kserve-self-signed-certs\""
Jul 23 23:51:13.817780 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:51:13.817765    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-fc333b\"/\"openshift-service-ca.crt\""
Jul 23 23:51:13.824554 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:51:13.824532    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-fc333b/auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq"]
Jul 23 23:51:13.844594 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:51:13.844561    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df-tls-certs\") pod \"auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq\" (UID: \"9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df\") " pod="ai-tenant-e2e-shared-b-fc333b/auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq"
Jul 23 23:51:13.844752 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:51:13.844598    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df-home\") pod \"auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq\" (UID: \"9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df\") " pod="ai-tenant-e2e-shared-b-fc333b/auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq"
Jul 23 23:51:13.844752 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:51:13.844637    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df-model-cache\") pod \"auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq\" (UID: \"9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df\") " pod="ai-tenant-e2e-shared-b-fc333b/auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq"
Jul 23 23:51:13.844752 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:51:13.844687    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df-dshm\") pod \"auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq\" (UID: \"9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df\") " pod="ai-tenant-e2e-shared-b-fc333b/auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq"
Jul 23 23:51:13.844752 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:51:13.844717    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df-tmp-dir\") pod \"auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq\" (UID: \"9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df\") " pod="ai-tenant-e2e-shared-b-fc333b/auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq"
Jul 23 23:51:13.844752 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:51:13.844736    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9w6ph\" (UniqueName: \"kubernetes.io/projected/9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df-kube-api-access-9w6ph\") pod \"auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq\" (UID: \"9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df\") " pod="ai-tenant-e2e-shared-b-fc333b/auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq"
Jul 23 23:51:13.945525 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:51:13.945485    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df-tls-certs\") pod \"auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq\" (UID: \"9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df\") " pod="ai-tenant-e2e-shared-b-fc333b/auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq"
Jul 23 23:51:13.945525 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:51:13.945526    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df-home\") pod \"auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq\" (UID: \"9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df\") " pod="ai-tenant-e2e-shared-b-fc333b/auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq"
Jul 23 23:51:13.945739 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:51:13.945619    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df-model-cache\") pod \"auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq\" (UID: \"9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df\") " pod="ai-tenant-e2e-shared-b-fc333b/auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq"
Jul 23 23:51:13.945739 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:51:13.945656    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df-dshm\") pod \"auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq\" (UID: \"9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df\") " pod="ai-tenant-e2e-shared-b-fc333b/auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq"
Jul 23 23:51:13.945739 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:51:13.945673    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df-tmp-dir\") pod \"auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq\" (UID: \"9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df\") " pod="ai-tenant-e2e-shared-b-fc333b/auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq"
Jul 23 23:51:13.945739 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:51:13.945691    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w6ph\" (UniqueName: \"kubernetes.io/projected/9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df-kube-api-access-9w6ph\") pod \"auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq\" (UID: \"9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df\") " pod="ai-tenant-e2e-shared-b-fc333b/auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq"
Jul 23 23:51:13.946095 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:51:13.946071    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df-home\") pod \"auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq\" (UID: \"9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df\") " pod="ai-tenant-e2e-shared-b-fc333b/auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq"
Jul 23 23:51:13.946184 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:51:13.946164    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df-model-cache\") pod \"auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq\" (UID: \"9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df\") " pod="ai-tenant-e2e-shared-b-fc333b/auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq"
Jul 23 23:51:13.946225 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:51:13.946192    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df-tmp-dir\") pod \"auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq\" (UID: \"9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df\") " pod="ai-tenant-e2e-shared-b-fc333b/auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq"
Jul 23 23:51:13.947838 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:51:13.947817    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df-dshm\") pod \"auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq\" (UID: \"9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df\") " pod="ai-tenant-e2e-shared-b-fc333b/auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq"
Jul 23 23:51:13.948249 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:51:13.948230    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df-tls-certs\") pod \"auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq\" (UID: \"9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df\") " pod="ai-tenant-e2e-shared-b-fc333b/auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq"
Jul 23 23:51:13.953708 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:51:13.953681    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9w6ph\" (UniqueName: \"kubernetes.io/projected/9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df-kube-api-access-9w6ph\") pod \"auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq\" (UID: \"9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df\") " pod="ai-tenant-e2e-shared-b-fc333b/auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq"
Jul 23 23:51:14.124193 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:51:14.124113    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fc333b/auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq"
Jul 23 23:51:14.255595 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:51:14.255570    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-fc333b/auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq"]
Jul 23 23:51:14.257154 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:51:14.257120    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9e7a0b27_8ba2_45eb_adc0_ff19d2e7d3df.slice/crio-42c276d71ee613012aa4d8dc63daf898d5438b9f9c1ba0153d17cd681b61ed74 WatchSource:0}: Error finding container 42c276d71ee613012aa4d8dc63daf898d5438b9f9c1ba0153d17cd681b61ed74: Status 404 returned error can't find the container with id 42c276d71ee613012aa4d8dc63daf898d5438b9f9c1ba0153d17cd681b61ed74
Jul 23 23:51:15.092251 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:51:15.092218    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fc333b/auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq" event={"ID":"9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df","Type":"ContainerStarted","Data":"82642ab9aae49c41c437e3aef58b7e0de5835950e427bcaa48e48d8f240ffb54"}
Jul 23 23:51:15.092655 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:51:15.092260    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fc333b/auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq" event={"ID":"9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df","Type":"ContainerStarted","Data":"42c276d71ee613012aa4d8dc63daf898d5438b9f9c1ba0153d17cd681b61ed74"}
Jul 23 23:51:15.110245 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:51:15.110197    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-fc333b/auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq" podStartSLOduration=2.11018376 podStartE2EDuration="2.11018376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:51:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:51:15.107903645 +0000 UTC m=+2276.851632694" watchObservedRunningTime="2026-07-23 23:51:15.11018376 +0000 UTC m=+2276.853912809"
Jul 23 23:51:17.478289 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:51:17.478252    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:51:24.124778 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:51:24.124742    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-fc333b/auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq"
Jul 23 23:51:24.125197 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:51:24.124794    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-fc333b/auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq"
Jul 23 23:51:24.137747 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:51:24.137719    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-fc333b/auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq"
Jul 23 23:51:24.152408 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:51:24.152388    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-fc333b/auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq"
Jul 23 23:51:31.283490 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:51:31.283448    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:51:49.883552 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:51:49.883518    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:52:06.086986 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:52:06.086939    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:52:25.877605 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:52:25.877569    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:52:44.385139 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:52:44.385102    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:53:02.279289 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:02.279248    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:53:17.207873 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:17.207839    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-867102/auth-test-model-867102-kserve-65457c7b79-v4ngj"]
Jul 23 23:53:18.187636 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:18.187602    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-fc333b/auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq"]
Jul 23 23:53:18.878773 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:18.878738    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:53:27.311566 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:27.311524    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-867102/sub-test-model-867102-kserve-5cbddf55f5-fwtpv"]
Jul 23 23:53:27.315425 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:27.315401    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-867102/sub-test-model-867102-kserve-5cbddf55f5-fwtpv"
Jul 23 23:53:27.317923 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:27.317900    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-867102\"/\"sub-test-model-867102-kserve-self-signed-certs\""
Jul 23 23:53:27.323810 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:27.323786    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-867102/sub-test-model-867102-kserve-5cbddf55f5-fwtpv"]
Jul 23 23:53:27.388369 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:27.388333    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/719c9548-0afc-48d2-b92c-614a6faf2fdf-tmp-dir\") pod \"sub-test-model-867102-kserve-5cbddf55f5-fwtpv\" (UID: \"719c9548-0afc-48d2-b92c-614a6faf2fdf\") " pod="ai-tenant-e2e-shared-a-867102/sub-test-model-867102-kserve-5cbddf55f5-fwtpv"
Jul 23 23:53:27.388571 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:27.388413    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/719c9548-0afc-48d2-b92c-614a6faf2fdf-tls-certs\") pod \"sub-test-model-867102-kserve-5cbddf55f5-fwtpv\" (UID: \"719c9548-0afc-48d2-b92c-614a6faf2fdf\") " pod="ai-tenant-e2e-shared-a-867102/sub-test-model-867102-kserve-5cbddf55f5-fwtpv"
Jul 23 23:53:27.388571 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:27.388466    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/719c9548-0afc-48d2-b92c-614a6faf2fdf-model-cache\") pod \"sub-test-model-867102-kserve-5cbddf55f5-fwtpv\" (UID: \"719c9548-0afc-48d2-b92c-614a6faf2fdf\") " pod="ai-tenant-e2e-shared-a-867102/sub-test-model-867102-kserve-5cbddf55f5-fwtpv"
Jul 23 23:53:27.388571 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:27.388528    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/719c9548-0afc-48d2-b92c-614a6faf2fdf-dshm\") pod \"sub-test-model-867102-kserve-5cbddf55f5-fwtpv\" (UID: \"719c9548-0afc-48d2-b92c-614a6faf2fdf\") " pod="ai-tenant-e2e-shared-a-867102/sub-test-model-867102-kserve-5cbddf55f5-fwtpv"
Jul 23 23:53:27.388571 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:27.388566    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/719c9548-0afc-48d2-b92c-614a6faf2fdf-home\") pod \"sub-test-model-867102-kserve-5cbddf55f5-fwtpv\" (UID: \"719c9548-0afc-48d2-b92c-614a6faf2fdf\") " pod="ai-tenant-e2e-shared-a-867102/sub-test-model-867102-kserve-5cbddf55f5-fwtpv"
Jul 23 23:53:27.388758 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:27.388589    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqvs8\" (UniqueName: \"kubernetes.io/projected/719c9548-0afc-48d2-b92c-614a6faf2fdf-kube-api-access-zqvs8\") pod \"sub-test-model-867102-kserve-5cbddf55f5-fwtpv\" (UID: \"719c9548-0afc-48d2-b92c-614a6faf2fdf\") " pod="ai-tenant-e2e-shared-a-867102/sub-test-model-867102-kserve-5cbddf55f5-fwtpv"
Jul 23 23:53:27.489197 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:27.489155    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/719c9548-0afc-48d2-b92c-614a6faf2fdf-tls-certs\") pod \"sub-test-model-867102-kserve-5cbddf55f5-fwtpv\" (UID: \"719c9548-0afc-48d2-b92c-614a6faf2fdf\") " pod="ai-tenant-e2e-shared-a-867102/sub-test-model-867102-kserve-5cbddf55f5-fwtpv"
Jul 23 23:53:27.489197 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:27.489202    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/719c9548-0afc-48d2-b92c-614a6faf2fdf-model-cache\") pod \"sub-test-model-867102-kserve-5cbddf55f5-fwtpv\" (UID: \"719c9548-0afc-48d2-b92c-614a6faf2fdf\") " pod="ai-tenant-e2e-shared-a-867102/sub-test-model-867102-kserve-5cbddf55f5-fwtpv"
Jul 23 23:53:27.489468 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:27.489241    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/719c9548-0afc-48d2-b92c-614a6faf2fdf-dshm\") pod \"sub-test-model-867102-kserve-5cbddf55f5-fwtpv\" (UID: \"719c9548-0afc-48d2-b92c-614a6faf2fdf\") " pod="ai-tenant-e2e-shared-a-867102/sub-test-model-867102-kserve-5cbddf55f5-fwtpv"
Jul 23 23:53:27.489468 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:27.489275    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/719c9548-0afc-48d2-b92c-614a6faf2fdf-home\") pod \"sub-test-model-867102-kserve-5cbddf55f5-fwtpv\" (UID: \"719c9548-0afc-48d2-b92c-614a6faf2fdf\") " pod="ai-tenant-e2e-shared-a-867102/sub-test-model-867102-kserve-5cbddf55f5-fwtpv"
Jul 23 23:53:27.489468 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:27.489298    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqvs8\" (UniqueName: \"kubernetes.io/projected/719c9548-0afc-48d2-b92c-614a6faf2fdf-kube-api-access-zqvs8\") pod \"sub-test-model-867102-kserve-5cbddf55f5-fwtpv\" (UID: \"719c9548-0afc-48d2-b92c-614a6faf2fdf\") " pod="ai-tenant-e2e-shared-a-867102/sub-test-model-867102-kserve-5cbddf55f5-fwtpv"
Jul 23 23:53:27.489468 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:27.489335    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/719c9548-0afc-48d2-b92c-614a6faf2fdf-tmp-dir\") pod \"sub-test-model-867102-kserve-5cbddf55f5-fwtpv\" (UID: \"719c9548-0afc-48d2-b92c-614a6faf2fdf\") " pod="ai-tenant-e2e-shared-a-867102/sub-test-model-867102-kserve-5cbddf55f5-fwtpv"
Jul 23 23:53:27.489694 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:27.489665    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/719c9548-0afc-48d2-b92c-614a6faf2fdf-model-cache\") pod \"sub-test-model-867102-kserve-5cbddf55f5-fwtpv\" (UID: \"719c9548-0afc-48d2-b92c-614a6faf2fdf\") " pod="ai-tenant-e2e-shared-a-867102/sub-test-model-867102-kserve-5cbddf55f5-fwtpv"
Jul 23 23:53:27.489763 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:27.489674    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/719c9548-0afc-48d2-b92c-614a6faf2fdf-home\") pod \"sub-test-model-867102-kserve-5cbddf55f5-fwtpv\" (UID: \"719c9548-0afc-48d2-b92c-614a6faf2fdf\") " pod="ai-tenant-e2e-shared-a-867102/sub-test-model-867102-kserve-5cbddf55f5-fwtpv"
Jul 23 23:53:27.489818 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:27.489764    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/719c9548-0afc-48d2-b92c-614a6faf2fdf-tmp-dir\") pod \"sub-test-model-867102-kserve-5cbddf55f5-fwtpv\" (UID: \"719c9548-0afc-48d2-b92c-614a6faf2fdf\") " pod="ai-tenant-e2e-shared-a-867102/sub-test-model-867102-kserve-5cbddf55f5-fwtpv"
Jul 23 23:53:27.491700 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:27.491669    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/719c9548-0afc-48d2-b92c-614a6faf2fdf-dshm\") pod \"sub-test-model-867102-kserve-5cbddf55f5-fwtpv\" (UID: \"719c9548-0afc-48d2-b92c-614a6faf2fdf\") " pod="ai-tenant-e2e-shared-a-867102/sub-test-model-867102-kserve-5cbddf55f5-fwtpv"
Jul 23 23:53:27.491929 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:27.491910    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/719c9548-0afc-48d2-b92c-614a6faf2fdf-tls-certs\") pod \"sub-test-model-867102-kserve-5cbddf55f5-fwtpv\" (UID: \"719c9548-0afc-48d2-b92c-614a6faf2fdf\") " pod="ai-tenant-e2e-shared-a-867102/sub-test-model-867102-kserve-5cbddf55f5-fwtpv"
Jul 23 23:53:27.497568 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:27.497549    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqvs8\" (UniqueName: \"kubernetes.io/projected/719c9548-0afc-48d2-b92c-614a6faf2fdf-kube-api-access-zqvs8\") pod \"sub-test-model-867102-kserve-5cbddf55f5-fwtpv\" (UID: \"719c9548-0afc-48d2-b92c-614a6faf2fdf\") " pod="ai-tenant-e2e-shared-a-867102/sub-test-model-867102-kserve-5cbddf55f5-fwtpv"
Jul 23 23:53:27.625830 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:27.625730    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-867102/sub-test-model-867102-kserve-5cbddf55f5-fwtpv"
Jul 23 23:53:27.759734 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:27.759706    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-867102/sub-test-model-867102-kserve-5cbddf55f5-fwtpv"]
Jul 23 23:53:27.761619 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:53:27.761588    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod719c9548_0afc_48d2_b92c_614a6faf2fdf.slice/crio-96c9fac8e6facf3d8a3cfd135ce348609b7118687032015e516658e6bb9356c7 WatchSource:0}: Error finding container 96c9fac8e6facf3d8a3cfd135ce348609b7118687032015e516658e6bb9356c7: Status 404 returned error can't find the container with id 96c9fac8e6facf3d8a3cfd135ce348609b7118687032015e516658e6bb9356c7
Jul 23 23:53:27.774246 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:27.774220    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-867102/sub-test-model-867102-kserve-5cbddf55f5-fwtpv" event={"ID":"719c9548-0afc-48d2-b92c-614a6faf2fdf","Type":"ContainerStarted","Data":"96c9fac8e6facf3d8a3cfd135ce348609b7118687032015e516658e6bb9356c7"}
Jul 23 23:53:28.780816 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:28.780779    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-867102/sub-test-model-867102-kserve-5cbddf55f5-fwtpv" event={"ID":"719c9548-0afc-48d2-b92c-614a6faf2fdf","Type":"ContainerStarted","Data":"8acea31c8374f32111341fa33d8d67a71273c4429ec99a2d38f1095ce0922003"}
Jul 23 23:53:28.798839 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:28.798788    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-867102/sub-test-model-867102-kserve-5cbddf55f5-fwtpv" podStartSLOduration=1.79877332 podStartE2EDuration="1.79877332s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:53:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:53:28.797144235 +0000 UTC m=+2410.540873283" watchObservedRunningTime="2026-07-23 23:53:28.79877332 +0000 UTC m=+2410.542502347"
Jul 23 23:53:31.076851 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:31.076811    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:53:32.226471 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:32.226434    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-867102/auth-test-model-867102-kserve-65457c7b79-v4ngj" podUID="d1450704-30a2-41bd-a6c3-e10688cea21a" containerName="main" containerID="cri-o://a20fd0fa2a1ad94a64643952f1c0d289fea037a9433599d8a2f8e2494c0c7bfb" gracePeriod=45
Jul 23 23:53:32.483019 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:32.482937    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-867102/auth-test-model-867102-kserve-65457c7b79-v4ngj"
Jul 23 23:53:32.537323 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:32.537287    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbc72\" (UniqueName: \"kubernetes.io/projected/d1450704-30a2-41bd-a6c3-e10688cea21a-kube-api-access-zbc72\") pod \"d1450704-30a2-41bd-a6c3-e10688cea21a\" (UID: \"d1450704-30a2-41bd-a6c3-e10688cea21a\") "
Jul 23 23:53:32.537521 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:32.537333    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1450704-30a2-41bd-a6c3-e10688cea21a-model-cache\") pod \"d1450704-30a2-41bd-a6c3-e10688cea21a\" (UID: \"d1450704-30a2-41bd-a6c3-e10688cea21a\") "
Jul 23 23:53:32.537521 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:32.537394    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1450704-30a2-41bd-a6c3-e10688cea21a-tmp-dir\") pod \"d1450704-30a2-41bd-a6c3-e10688cea21a\" (UID: \"d1450704-30a2-41bd-a6c3-e10688cea21a\") "
Jul 23 23:53:32.537521 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:32.537417    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1450704-30a2-41bd-a6c3-e10688cea21a-home\") pod \"d1450704-30a2-41bd-a6c3-e10688cea21a\" (UID: \"d1450704-30a2-41bd-a6c3-e10688cea21a\") "
Jul 23 23:53:32.537521 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:32.537453    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1450704-30a2-41bd-a6c3-e10688cea21a-tls-certs\") pod \"d1450704-30a2-41bd-a6c3-e10688cea21a\" (UID: \"d1450704-30a2-41bd-a6c3-e10688cea21a\") "
Jul 23 23:53:32.537521 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:32.537485    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1450704-30a2-41bd-a6c3-e10688cea21a-dshm\") pod \"d1450704-30a2-41bd-a6c3-e10688cea21a\" (UID: \"d1450704-30a2-41bd-a6c3-e10688cea21a\") "
Jul 23 23:53:32.537790 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:32.537666    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1450704-30a2-41bd-a6c3-e10688cea21a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d1450704-30a2-41bd-a6c3-e10688cea21a" (UID: "d1450704-30a2-41bd-a6c3-e10688cea21a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:53:32.537790 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:32.537683    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1450704-30a2-41bd-a6c3-e10688cea21a-model-cache" (OuterVolumeSpecName: "model-cache") pod "d1450704-30a2-41bd-a6c3-e10688cea21a" (UID: "d1450704-30a2-41bd-a6c3-e10688cea21a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:53:32.537790 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:32.537702    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1450704-30a2-41bd-a6c3-e10688cea21a-home" (OuterVolumeSpecName: "home") pod "d1450704-30a2-41bd-a6c3-e10688cea21a" (UID: "d1450704-30a2-41bd-a6c3-e10688cea21a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:53:32.539653 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:32.539626    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1450704-30a2-41bd-a6c3-e10688cea21a-dshm" (OuterVolumeSpecName: "dshm") pod "d1450704-30a2-41bd-a6c3-e10688cea21a" (UID: "d1450704-30a2-41bd-a6c3-e10688cea21a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:53:32.539776 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:32.539661    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1450704-30a2-41bd-a6c3-e10688cea21a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d1450704-30a2-41bd-a6c3-e10688cea21a" (UID: "d1450704-30a2-41bd-a6c3-e10688cea21a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:53:32.539776 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:32.539709    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1450704-30a2-41bd-a6c3-e10688cea21a-kube-api-access-zbc72" (OuterVolumeSpecName: "kube-api-access-zbc72") pod "d1450704-30a2-41bd-a6c3-e10688cea21a" (UID: "d1450704-30a2-41bd-a6c3-e10688cea21a"). InnerVolumeSpecName "kube-api-access-zbc72". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:53:32.638425 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:32.638393    2548 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1450704-30a2-41bd-a6c3-e10688cea21a-home\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:53:32.638425 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:32.638420    2548 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1450704-30a2-41bd-a6c3-e10688cea21a-tls-certs\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:53:32.638425 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:32.638429    2548 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1450704-30a2-41bd-a6c3-e10688cea21a-dshm\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:53:32.638666 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:32.638437    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbc72\" (UniqueName: \"kubernetes.io/projected/d1450704-30a2-41bd-a6c3-e10688cea21a-kube-api-access-zbc72\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:53:32.638666 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:32.638447    2548 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1450704-30a2-41bd-a6c3-e10688cea21a-model-cache\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:53:32.638666 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:32.638455    2548 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1450704-30a2-41bd-a6c3-e10688cea21a-tmp-dir\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:53:32.802286 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:32.802254    2548 generic.go:356] "Generic (PLEG): container finished" podID="d1450704-30a2-41bd-a6c3-e10688cea21a" containerID="a20fd0fa2a1ad94a64643952f1c0d289fea037a9433599d8a2f8e2494c0c7bfb" exitCode=0
Jul 23 23:53:32.802457 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:32.802297    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-867102/auth-test-model-867102-kserve-65457c7b79-v4ngj"
Jul 23 23:53:32.802457 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:32.802343    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-867102/auth-test-model-867102-kserve-65457c7b79-v4ngj" event={"ID":"d1450704-30a2-41bd-a6c3-e10688cea21a","Type":"ContainerDied","Data":"a20fd0fa2a1ad94a64643952f1c0d289fea037a9433599d8a2f8e2494c0c7bfb"}
Jul 23 23:53:32.802457 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:32.802393    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-867102/auth-test-model-867102-kserve-65457c7b79-v4ngj" event={"ID":"d1450704-30a2-41bd-a6c3-e10688cea21a","Type":"ContainerDied","Data":"c9d0ab87440ceab2a6df7f9316fc1b8a2756749a3a712b4f8c16a07ebe3d30aa"}
Jul 23 23:53:32.802457 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:32.802415    2548 scope.go:117] "RemoveContainer" containerID="a20fd0fa2a1ad94a64643952f1c0d289fea037a9433599d8a2f8e2494c0c7bfb"
Jul 23 23:53:32.811099 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:32.811083    2548 scope.go:117] "RemoveContainer" containerID="a20fd0fa2a1ad94a64643952f1c0d289fea037a9433599d8a2f8e2494c0c7bfb"
Jul 23 23:53:32.811350 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:53:32.811331    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a20fd0fa2a1ad94a64643952f1c0d289fea037a9433599d8a2f8e2494c0c7bfb\": container with ID starting with a20fd0fa2a1ad94a64643952f1c0d289fea037a9433599d8a2f8e2494c0c7bfb not found: ID does not exist" containerID="a20fd0fa2a1ad94a64643952f1c0d289fea037a9433599d8a2f8e2494c0c7bfb"
Jul 23 23:53:32.811398 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:32.811359    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a20fd0fa2a1ad94a64643952f1c0d289fea037a9433599d8a2f8e2494c0c7bfb"} err="failed to get container status \"a20fd0fa2a1ad94a64643952f1c0d289fea037a9433599d8a2f8e2494c0c7bfb\": rpc error: code = NotFound desc = could not find container \"a20fd0fa2a1ad94a64643952f1c0d289fea037a9433599d8a2f8e2494c0c7bfb\": container with ID starting with a20fd0fa2a1ad94a64643952f1c0d289fea037a9433599d8a2f8e2494c0c7bfb not found: ID does not exist"
Jul 23 23:53:32.817802 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:32.817780    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-867102/auth-test-model-867102-kserve-65457c7b79-v4ngj"]
Jul 23 23:53:32.819930 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:32.819909    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-867102/auth-test-model-867102-kserve-65457c7b79-v4ngj"]
Jul 23 23:53:33.206831 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:33.206776    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-fc333b/auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq" podUID="9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df" containerName="main" containerID="cri-o://82642ab9aae49c41c437e3aef58b7e0de5835950e427bcaa48e48d8f240ffb54" gracePeriod=45
Jul 23 23:53:33.451334 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:33.451304    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fc333b/auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq"
Jul 23 23:53:33.547289 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:33.547203    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df-model-cache\") pod \"9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df\" (UID: \"9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df\") "
Jul 23 23:53:33.547289 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:33.547260    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df-tls-certs\") pod \"9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df\" (UID: \"9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df\") "
Jul 23 23:53:33.547289 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:33.547287    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df-home\") pod \"9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df\" (UID: \"9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df\") "
Jul 23 23:53:33.547582 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:33.547331    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df-tmp-dir\") pod \"9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df\" (UID: \"9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df\") "
Jul 23 23:53:33.547582 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:33.547350    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9w6ph\" (UniqueName: \"kubernetes.io/projected/9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df-kube-api-access-9w6ph\") pod \"9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df\" (UID: \"9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df\") "
Jul 23 23:53:33.547582 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:33.547384    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df-dshm\") pod \"9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df\" (UID: \"9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df\") "
Jul 23 23:53:33.547582 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:33.547488    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df-model-cache" (OuterVolumeSpecName: "model-cache") pod "9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df" (UID: "9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:53:33.547582 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:33.547543    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df" (UID: "9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:53:33.547834 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:33.547581    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df-home" (OuterVolumeSpecName: "home") pod "9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df" (UID: "9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:53:33.547834 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:33.547647    2548 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df-home\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:53:33.547834 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:33.547660    2548 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df-tmp-dir\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:53:33.547834 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:33.547670    2548 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df-model-cache\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:53:33.549652 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:33.549630    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df" (UID: "9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:53:33.549726 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:33.549646    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df-dshm" (OuterVolumeSpecName: "dshm") pod "9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df" (UID: "9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:53:33.549726 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:33.549660    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df-kube-api-access-9w6ph" (OuterVolumeSpecName: "kube-api-access-9w6ph") pod "9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df" (UID: "9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df"). InnerVolumeSpecName "kube-api-access-9w6ph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:53:33.648279 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:33.648246    2548 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df-tls-certs\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:53:33.648279 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:33.648274    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9w6ph\" (UniqueName: \"kubernetes.io/projected/9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df-kube-api-access-9w6ph\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:53:33.648279 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:33.648284    2548 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df-dshm\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:53:33.809069 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:33.808962    2548 generic.go:356] "Generic (PLEG): container finished" podID="9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df" containerID="82642ab9aae49c41c437e3aef58b7e0de5835950e427bcaa48e48d8f240ffb54" exitCode=0
Jul 23 23:53:33.809069 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:33.809022    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fc333b/auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq"
Jul 23 23:53:33.809069 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:33.809051    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fc333b/auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq" event={"ID":"9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df","Type":"ContainerDied","Data":"82642ab9aae49c41c437e3aef58b7e0de5835950e427bcaa48e48d8f240ffb54"}
Jul 23 23:53:33.809295 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:33.809095    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fc333b/auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq" event={"ID":"9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df","Type":"ContainerDied","Data":"42c276d71ee613012aa4d8dc63daf898d5438b9f9c1ba0153d17cd681b61ed74"}
Jul 23 23:53:33.809295 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:33.809112    2548 scope.go:117] "RemoveContainer" containerID="82642ab9aae49c41c437e3aef58b7e0de5835950e427bcaa48e48d8f240ffb54"
Jul 23 23:53:33.818727 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:33.818706    2548 scope.go:117] "RemoveContainer" containerID="82642ab9aae49c41c437e3aef58b7e0de5835950e427bcaa48e48d8f240ffb54"
Jul 23 23:53:33.819018 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:53:33.819000    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82642ab9aae49c41c437e3aef58b7e0de5835950e427bcaa48e48d8f240ffb54\": container with ID starting with 82642ab9aae49c41c437e3aef58b7e0de5835950e427bcaa48e48d8f240ffb54 not found: ID does not exist" containerID="82642ab9aae49c41c437e3aef58b7e0de5835950e427bcaa48e48d8f240ffb54"
Jul 23 23:53:33.819093 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:33.819032    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82642ab9aae49c41c437e3aef58b7e0de5835950e427bcaa48e48d8f240ffb54"} err="failed to get container status \"82642ab9aae49c41c437e3aef58b7e0de5835950e427bcaa48e48d8f240ffb54\": rpc error: code = NotFound desc = could not find container \"82642ab9aae49c41c437e3aef58b7e0de5835950e427bcaa48e48d8f240ffb54\": container with ID starting with 82642ab9aae49c41c437e3aef58b7e0de5835950e427bcaa48e48d8f240ffb54 not found: ID does not exist"
Jul 23 23:53:33.830254 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:33.830230    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-fc333b/auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq"]
Jul 23 23:53:33.833585 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:33.833565    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-fc333b/auth-test-model-fc333b-kserve-56cfbc6d99-fdjwq"]
Jul 23 23:53:34.748279 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:34.748252    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df" path="/var/lib/kubelet/pods/9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df/volumes"
Jul 23 23:53:34.748664 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:34.748656    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1450704-30a2-41bd-a6c3-e10688cea21a" path="/var/lib/kubelet/pods/d1450704-30a2-41bd-a6c3-e10688cea21a/volumes"
Jul 23 23:53:37.625883 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:37.625836    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-867102/sub-test-model-867102-kserve-5cbddf55f5-fwtpv"
Jul 23 23:53:37.626384 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:37.625905    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-867102/sub-test-model-867102-kserve-5cbddf55f5-fwtpv"
Jul 23 23:53:37.638616 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:37.638588    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-867102/sub-test-model-867102-kserve-5cbddf55f5-fwtpv"
Jul 23 23:53:37.850464 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:37.850435    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-867102/sub-test-model-867102-kserve-5cbddf55f5-fwtpv"
Jul 23 23:53:43.300391 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:43.300311    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-fc333b/sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b"]
Jul 23 23:53:43.300738 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:43.300668    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df" containerName="main"
Jul 23 23:53:43.300738 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:43.300680    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df" containerName="main"
Jul 23 23:53:43.300738 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:43.300693    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1450704-30a2-41bd-a6c3-e10688cea21a" containerName="main"
Jul 23 23:53:43.300738 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:43.300701    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1450704-30a2-41bd-a6c3-e10688cea21a" containerName="main"
Jul 23 23:53:43.300866 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:43.300763    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1450704-30a2-41bd-a6c3-e10688cea21a" containerName="main"
Jul 23 23:53:43.300866 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:43.300773    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e7a0b27-8ba2-45eb-adc0-ff19d2e7d3df" containerName="main"
Jul 23 23:53:43.305059 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:43.305038    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fc333b/sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b"
Jul 23 23:53:43.307474 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:43.307457    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-fc333b\"/\"kube-root-ca.crt\""
Jul 23 23:53:43.308461 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:43.308438    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-fc333b\"/\"openshift-service-ca.crt\""
Jul 23 23:53:43.308591 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:43.308436    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-fc333b\"/\"sub-test-model-fc333b-kserve-self-signed-certs\""
Jul 23 23:53:43.308591 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:43.308525    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-fc333b\"/\"default-dockercfg-2dfzf\""
Jul 23 23:53:43.311955 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:43.311932    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-fc333b/sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b"]
Jul 23 23:53:43.433738 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:43.433706    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91390fdb-04f1-4764-9cb0-89d2f3349516-tls-certs\") pod \"sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b\" (UID: \"91390fdb-04f1-4764-9cb0-89d2f3349516\") " pod="ai-tenant-e2e-shared-b-fc333b/sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b"
Jul 23 23:53:43.433738 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:43.433741    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdgnf\" (UniqueName: \"kubernetes.io/projected/91390fdb-04f1-4764-9cb0-89d2f3349516-kube-api-access-jdgnf\") pod \"sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b\" (UID: \"91390fdb-04f1-4764-9cb0-89d2f3349516\") " pod="ai-tenant-e2e-shared-b-fc333b/sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b"
Jul 23 23:53:43.433922 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:43.433832    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91390fdb-04f1-4764-9cb0-89d2f3349516-model-cache\") pod \"sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b\" (UID: \"91390fdb-04f1-4764-9cb0-89d2f3349516\") " pod="ai-tenant-e2e-shared-b-fc333b/sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b"
Jul 23 23:53:43.433922 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:43.433861    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91390fdb-04f1-4764-9cb0-89d2f3349516-dshm\") pod \"sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b\" (UID: \"91390fdb-04f1-4764-9cb0-89d2f3349516\") " pod="ai-tenant-e2e-shared-b-fc333b/sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b"
Jul 23 23:53:43.433922 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:43.433896    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91390fdb-04f1-4764-9cb0-89d2f3349516-home\") pod \"sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b\" (UID: \"91390fdb-04f1-4764-9cb0-89d2f3349516\") " pod="ai-tenant-e2e-shared-b-fc333b/sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b"
Jul 23 23:53:43.433922 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:43.433919    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91390fdb-04f1-4764-9cb0-89d2f3349516-tmp-dir\") pod \"sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b\" (UID: \"91390fdb-04f1-4764-9cb0-89d2f3349516\") " pod="ai-tenant-e2e-shared-b-fc333b/sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b"
Jul 23 23:53:43.535036 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:43.535001    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91390fdb-04f1-4764-9cb0-89d2f3349516-tls-certs\") pod \"sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b\" (UID: \"91390fdb-04f1-4764-9cb0-89d2f3349516\") " pod="ai-tenant-e2e-shared-b-fc333b/sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b"
Jul 23 23:53:43.535036 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:43.535038    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdgnf\" (UniqueName: \"kubernetes.io/projected/91390fdb-04f1-4764-9cb0-89d2f3349516-kube-api-access-jdgnf\") pod \"sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b\" (UID: \"91390fdb-04f1-4764-9cb0-89d2f3349516\") " pod="ai-tenant-e2e-shared-b-fc333b/sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b"
Jul 23 23:53:43.535293 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:43.535079    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91390fdb-04f1-4764-9cb0-89d2f3349516-model-cache\") pod \"sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b\" (UID: \"91390fdb-04f1-4764-9cb0-89d2f3349516\") " pod="ai-tenant-e2e-shared-b-fc333b/sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b"
Jul 23 23:53:43.535293 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:43.535107    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91390fdb-04f1-4764-9cb0-89d2f3349516-dshm\") pod \"sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b\" (UID: \"91390fdb-04f1-4764-9cb0-89d2f3349516\") " pod="ai-tenant-e2e-shared-b-fc333b/sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b"
Jul 23 23:53:43.535293 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:43.535143    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91390fdb-04f1-4764-9cb0-89d2f3349516-home\") pod \"sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b\" (UID: \"91390fdb-04f1-4764-9cb0-89d2f3349516\") " pod="ai-tenant-e2e-shared-b-fc333b/sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b"
Jul 23 23:53:43.535293 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:43.535258    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91390fdb-04f1-4764-9cb0-89d2f3349516-tmp-dir\") pod \"sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b\" (UID: \"91390fdb-04f1-4764-9cb0-89d2f3349516\") " pod="ai-tenant-e2e-shared-b-fc333b/sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b"
Jul 23 23:53:43.535620 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:43.535599    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91390fdb-04f1-4764-9cb0-89d2f3349516-model-cache\") pod \"sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b\" (UID: \"91390fdb-04f1-4764-9cb0-89d2f3349516\") " pod="ai-tenant-e2e-shared-b-fc333b/sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b"
Jul 23 23:53:43.535684 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:43.535670    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91390fdb-04f1-4764-9cb0-89d2f3349516-home\") pod \"sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b\" (UID: \"91390fdb-04f1-4764-9cb0-89d2f3349516\") " pod="ai-tenant-e2e-shared-b-fc333b/sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b"
Jul 23 23:53:43.535725 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:43.535680    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91390fdb-04f1-4764-9cb0-89d2f3349516-tmp-dir\") pod \"sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b\" (UID: \"91390fdb-04f1-4764-9cb0-89d2f3349516\") " pod="ai-tenant-e2e-shared-b-fc333b/sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b"
Jul 23 23:53:43.537384 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:43.537356    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91390fdb-04f1-4764-9cb0-89d2f3349516-dshm\") pod \"sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b\" (UID: \"91390fdb-04f1-4764-9cb0-89d2f3349516\") " pod="ai-tenant-e2e-shared-b-fc333b/sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b"
Jul 23 23:53:43.537593 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:43.537576    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91390fdb-04f1-4764-9cb0-89d2f3349516-tls-certs\") pod \"sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b\" (UID: \"91390fdb-04f1-4764-9cb0-89d2f3349516\") " pod="ai-tenant-e2e-shared-b-fc333b/sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b"
Jul 23 23:53:43.542299 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:43.542277    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdgnf\" (UniqueName: \"kubernetes.io/projected/91390fdb-04f1-4764-9cb0-89d2f3349516-kube-api-access-jdgnf\") pod \"sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b\" (UID: \"91390fdb-04f1-4764-9cb0-89d2f3349516\") " pod="ai-tenant-e2e-shared-b-fc333b/sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b"
Jul 23 23:53:43.615557 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:43.615464    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fc333b/sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b"
Jul 23 23:53:43.745446 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:43.745421    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-fc333b/sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b"]
Jul 23 23:53:43.747523 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:53:43.747476    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod91390fdb_04f1_4764_9cb0_89d2f3349516.slice/crio-213e2bd8e5debd6962a292c7eeaa7b1a65818c790b78d7a2ed83c98fd22e46fe WatchSource:0}: Error finding container 213e2bd8e5debd6962a292c7eeaa7b1a65818c790b78d7a2ed83c98fd22e46fe: Status 404 returned error can't find the container with id 213e2bd8e5debd6962a292c7eeaa7b1a65818c790b78d7a2ed83c98fd22e46fe
Jul 23 23:53:43.872542 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:43.872448    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fc333b/sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b" event={"ID":"91390fdb-04f1-4764-9cb0-89d2f3349516","Type":"ContainerStarted","Data":"eef4cec84bd8171a54010a834e4d8dbe10106ed1bf4e66efd9b819d6c823b58a"}
Jul 23 23:53:43.872542 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:43.872507    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fc333b/sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b" event={"ID":"91390fdb-04f1-4764-9cb0-89d2f3349516","Type":"ContainerStarted","Data":"213e2bd8e5debd6962a292c7eeaa7b1a65818c790b78d7a2ed83c98fd22e46fe"}
Jul 23 23:53:43.889297 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:43.889252    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-fc333b/sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b" podStartSLOduration=0.889236249 podStartE2EDuration="889.236249ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:53:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:53:43.888277609 +0000 UTC m=+2425.632006657" watchObservedRunningTime="2026-07-23 23:53:43.889236249 +0000 UTC m=+2425.632965297"
Jul 23 23:53:46.385277 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:46.385241    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:53:53.616061 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:53.616018    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-fc333b/sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b"
Jul 23 23:53:53.616578 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:53.616085    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-fc333b/sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b"
Jul 23 23:53:53.628224 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:53.628201    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-fc333b/sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b"
Jul 23 23:53:53.941174 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:53:53.941144    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-fc333b/sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b"
Jul 23 23:54:07.787856 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:07.787818    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:54:21.977152 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:21.977112    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:54:28.045849 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:28.045815    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-867102/sub-test-model-867102-kserve-5cbddf55f5-fwtpv"]
Jul 23 23:54:28.912666 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:28.912628    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-fc333b/sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b"]
Jul 23 23:54:40.086093 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:40.086050    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:54:43.067091 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:43.067057    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-867102/sub-test-model-867102-kserve-5cbddf55f5-fwtpv" podUID="719c9548-0afc-48d2-b92c-614a6faf2fdf" containerName="main" containerID="cri-o://8acea31c8374f32111341fa33d8d67a71273c4429ec99a2d38f1095ce0922003" gracePeriod=45
Jul 23 23:54:43.318823 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:43.318767    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-867102/sub-test-model-867102-kserve-5cbddf55f5-fwtpv"
Jul 23 23:54:43.462767 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:43.462707    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/719c9548-0afc-48d2-b92c-614a6faf2fdf-tmp-dir\") pod \"719c9548-0afc-48d2-b92c-614a6faf2fdf\" (UID: \"719c9548-0afc-48d2-b92c-614a6faf2fdf\") "
Jul 23 23:54:43.463079 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:43.463057    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zqvs8\" (UniqueName: \"kubernetes.io/projected/719c9548-0afc-48d2-b92c-614a6faf2fdf-kube-api-access-zqvs8\") pod \"719c9548-0afc-48d2-b92c-614a6faf2fdf\" (UID: \"719c9548-0afc-48d2-b92c-614a6faf2fdf\") "
Jul 23 23:54:43.463135 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:43.463120    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/719c9548-0afc-48d2-b92c-614a6faf2fdf-model-cache\") pod \"719c9548-0afc-48d2-b92c-614a6faf2fdf\" (UID: \"719c9548-0afc-48d2-b92c-614a6faf2fdf\") "
Jul 23 23:54:43.463193 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:43.463150    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/719c9548-0afc-48d2-b92c-614a6faf2fdf-dshm\") pod \"719c9548-0afc-48d2-b92c-614a6faf2fdf\" (UID: \"719c9548-0afc-48d2-b92c-614a6faf2fdf\") "
Jul 23 23:54:43.463193 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:43.463178    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/719c9548-0afc-48d2-b92c-614a6faf2fdf-tls-certs\") pod \"719c9548-0afc-48d2-b92c-614a6faf2fdf\" (UID: \"719c9548-0afc-48d2-b92c-614a6faf2fdf\") "
Jul 23 23:54:43.463298 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:43.463210    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/719c9548-0afc-48d2-b92c-614a6faf2fdf-home\") pod \"719c9548-0afc-48d2-b92c-614a6faf2fdf\" (UID: \"719c9548-0afc-48d2-b92c-614a6faf2fdf\") "
Jul 23 23:54:43.463446 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:43.463420    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/719c9548-0afc-48d2-b92c-614a6faf2fdf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "719c9548-0afc-48d2-b92c-614a6faf2fdf" (UID: "719c9548-0afc-48d2-b92c-614a6faf2fdf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:54:43.463555 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:43.463532    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/719c9548-0afc-48d2-b92c-614a6faf2fdf-model-cache" (OuterVolumeSpecName: "model-cache") pod "719c9548-0afc-48d2-b92c-614a6faf2fdf" (UID: "719c9548-0afc-48d2-b92c-614a6faf2fdf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:54:43.463627 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:43.463544    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/719c9548-0afc-48d2-b92c-614a6faf2fdf-home" (OuterVolumeSpecName: "home") pod "719c9548-0afc-48d2-b92c-614a6faf2fdf" (UID: "719c9548-0afc-48d2-b92c-614a6faf2fdf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:54:43.463627 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:43.463610    2548 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/719c9548-0afc-48d2-b92c-614a6faf2fdf-model-cache\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:54:43.463627 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:43.463627    2548 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/719c9548-0afc-48d2-b92c-614a6faf2fdf-tmp-dir\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:54:43.465483 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:43.465450    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/719c9548-0afc-48d2-b92c-614a6faf2fdf-kube-api-access-zqvs8" (OuterVolumeSpecName: "kube-api-access-zqvs8") pod "719c9548-0afc-48d2-b92c-614a6faf2fdf" (UID: "719c9548-0afc-48d2-b92c-614a6faf2fdf"). InnerVolumeSpecName "kube-api-access-zqvs8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:54:43.465601 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:43.465496    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/719c9548-0afc-48d2-b92c-614a6faf2fdf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "719c9548-0afc-48d2-b92c-614a6faf2fdf" (UID: "719c9548-0afc-48d2-b92c-614a6faf2fdf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:54:43.465601 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:43.465501    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/719c9548-0afc-48d2-b92c-614a6faf2fdf-dshm" (OuterVolumeSpecName: "dshm") pod "719c9548-0afc-48d2-b92c-614a6faf2fdf" (UID: "719c9548-0afc-48d2-b92c-614a6faf2fdf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:54:43.564918 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:43.564884    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqvs8\" (UniqueName: \"kubernetes.io/projected/719c9548-0afc-48d2-b92c-614a6faf2fdf-kube-api-access-zqvs8\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:54:43.564918 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:43.564915    2548 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/719c9548-0afc-48d2-b92c-614a6faf2fdf-dshm\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:54:43.564918 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:43.564926    2548 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/719c9548-0afc-48d2-b92c-614a6faf2fdf-tls-certs\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:54:43.565195 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:43.564934    2548 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/719c9548-0afc-48d2-b92c-614a6faf2fdf-home\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:54:43.931513 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:43.931469    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-fc333b/sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b" podUID="91390fdb-04f1-4764-9cb0-89d2f3349516" containerName="main" containerID="cri-o://eef4cec84bd8171a54010a834e4d8dbe10106ed1bf4e66efd9b819d6c823b58a" gracePeriod=45
Jul 23 23:54:43.935495 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:43.935448    2548 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-b-fc333b/sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b" podUID="91390fdb-04f1-4764-9cb0-89d2f3349516" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": EOF"
Jul 23 23:54:44.181059 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:44.181032    2548 generic.go:356] "Generic (PLEG): container finished" podID="91390fdb-04f1-4764-9cb0-89d2f3349516" containerID="eef4cec84bd8171a54010a834e4d8dbe10106ed1bf4e66efd9b819d6c823b58a" exitCode=0
Jul 23 23:54:44.181441 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:44.181122    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fc333b/sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b" event={"ID":"91390fdb-04f1-4764-9cb0-89d2f3349516","Type":"ContainerDied","Data":"eef4cec84bd8171a54010a834e4d8dbe10106ed1bf4e66efd9b819d6c823b58a"}
Jul 23 23:54:44.181441 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:44.181163    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fc333b/sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b" event={"ID":"91390fdb-04f1-4764-9cb0-89d2f3349516","Type":"ContainerDied","Data":"213e2bd8e5debd6962a292c7eeaa7b1a65818c790b78d7a2ed83c98fd22e46fe"}
Jul 23 23:54:44.181441 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:44.181179    2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="213e2bd8e5debd6962a292c7eeaa7b1a65818c790b78d7a2ed83c98fd22e46fe"
Jul 23 23:54:44.182550 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:44.182493    2548 generic.go:356] "Generic (PLEG): container finished" podID="719c9548-0afc-48d2-b92c-614a6faf2fdf" containerID="8acea31c8374f32111341fa33d8d67a71273c4429ec99a2d38f1095ce0922003" exitCode=0
Jul 23 23:54:44.182648 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:44.182550    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-867102/sub-test-model-867102-kserve-5cbddf55f5-fwtpv"
Jul 23 23:54:44.182648 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:44.182572    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-867102/sub-test-model-867102-kserve-5cbddf55f5-fwtpv" event={"ID":"719c9548-0afc-48d2-b92c-614a6faf2fdf","Type":"ContainerDied","Data":"8acea31c8374f32111341fa33d8d67a71273c4429ec99a2d38f1095ce0922003"}
Jul 23 23:54:44.182648 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:44.182611    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-867102/sub-test-model-867102-kserve-5cbddf55f5-fwtpv" event={"ID":"719c9548-0afc-48d2-b92c-614a6faf2fdf","Type":"ContainerDied","Data":"96c9fac8e6facf3d8a3cfd135ce348609b7118687032015e516658e6bb9356c7"}
Jul 23 23:54:44.182648 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:44.182629    2548 scope.go:117] "RemoveContainer" containerID="8acea31c8374f32111341fa33d8d67a71273c4429ec99a2d38f1095ce0922003"
Jul 23 23:54:44.184854 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:44.184840    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fc333b/sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b"
Jul 23 23:54:44.191240 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:44.191222    2548 scope.go:117] "RemoveContainer" containerID="8acea31c8374f32111341fa33d8d67a71273c4429ec99a2d38f1095ce0922003"
Jul 23 23:54:44.191503 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:54:44.191475    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8acea31c8374f32111341fa33d8d67a71273c4429ec99a2d38f1095ce0922003\": container with ID starting with 8acea31c8374f32111341fa33d8d67a71273c4429ec99a2d38f1095ce0922003 not found: ID does not exist" containerID="8acea31c8374f32111341fa33d8d67a71273c4429ec99a2d38f1095ce0922003"
Jul 23 23:54:44.191560 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:44.191511    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8acea31c8374f32111341fa33d8d67a71273c4429ec99a2d38f1095ce0922003"} err="failed to get container status \"8acea31c8374f32111341fa33d8d67a71273c4429ec99a2d38f1095ce0922003\": rpc error: code = NotFound desc = could not find container \"8acea31c8374f32111341fa33d8d67a71273c4429ec99a2d38f1095ce0922003\": container with ID starting with 8acea31c8374f32111341fa33d8d67a71273c4429ec99a2d38f1095ce0922003 not found: ID does not exist"
Jul 23 23:54:44.214339 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:44.214317    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-867102/sub-test-model-867102-kserve-5cbddf55f5-fwtpv"]
Jul 23 23:54:44.219456 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:44.219436    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-867102/sub-test-model-867102-kserve-5cbddf55f5-fwtpv"]
Jul 23 23:54:44.271735 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:44.271705    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91390fdb-04f1-4764-9cb0-89d2f3349516-dshm\") pod \"91390fdb-04f1-4764-9cb0-89d2f3349516\" (UID: \"91390fdb-04f1-4764-9cb0-89d2f3349516\") "
Jul 23 23:54:44.271874 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:44.271746    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jdgnf\" (UniqueName: \"kubernetes.io/projected/91390fdb-04f1-4764-9cb0-89d2f3349516-kube-api-access-jdgnf\") pod \"91390fdb-04f1-4764-9cb0-89d2f3349516\" (UID: \"91390fdb-04f1-4764-9cb0-89d2f3349516\") "
Jul 23 23:54:44.271874 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:44.271793    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91390fdb-04f1-4764-9cb0-89d2f3349516-tmp-dir\") pod \"91390fdb-04f1-4764-9cb0-89d2f3349516\" (UID: \"91390fdb-04f1-4764-9cb0-89d2f3349516\") "
Jul 23 23:54:44.271874 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:44.271834    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91390fdb-04f1-4764-9cb0-89d2f3349516-tls-certs\") pod \"91390fdb-04f1-4764-9cb0-89d2f3349516\" (UID: \"91390fdb-04f1-4764-9cb0-89d2f3349516\") "
Jul 23 23:54:44.271874 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:44.271858    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91390fdb-04f1-4764-9cb0-89d2f3349516-model-cache\") pod \"91390fdb-04f1-4764-9cb0-89d2f3349516\" (UID: \"91390fdb-04f1-4764-9cb0-89d2f3349516\") "
Jul 23 23:54:44.272117 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:44.271901    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91390fdb-04f1-4764-9cb0-89d2f3349516-home\") pod \"91390fdb-04f1-4764-9cb0-89d2f3349516\" (UID: \"91390fdb-04f1-4764-9cb0-89d2f3349516\") "
Jul 23 23:54:44.272169 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:44.272114    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91390fdb-04f1-4764-9cb0-89d2f3349516-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "91390fdb-04f1-4764-9cb0-89d2f3349516" (UID: "91390fdb-04f1-4764-9cb0-89d2f3349516"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:54:44.272210 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:44.272163    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91390fdb-04f1-4764-9cb0-89d2f3349516-model-cache" (OuterVolumeSpecName: "model-cache") pod "91390fdb-04f1-4764-9cb0-89d2f3349516" (UID: "91390fdb-04f1-4764-9cb0-89d2f3349516"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:54:44.272257 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:44.272216    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91390fdb-04f1-4764-9cb0-89d2f3349516-home" (OuterVolumeSpecName: "home") pod "91390fdb-04f1-4764-9cb0-89d2f3349516" (UID: "91390fdb-04f1-4764-9cb0-89d2f3349516"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:54:44.272299 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:44.272271    2548 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91390fdb-04f1-4764-9cb0-89d2f3349516-tmp-dir\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:54:44.272299 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:44.272290    2548 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91390fdb-04f1-4764-9cb0-89d2f3349516-model-cache\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:54:44.273964 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:44.273938    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91390fdb-04f1-4764-9cb0-89d2f3349516-kube-api-access-jdgnf" (OuterVolumeSpecName: "kube-api-access-jdgnf") pod "91390fdb-04f1-4764-9cb0-89d2f3349516" (UID: "91390fdb-04f1-4764-9cb0-89d2f3349516"). InnerVolumeSpecName "kube-api-access-jdgnf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:54:44.273964 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:44.273956    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91390fdb-04f1-4764-9cb0-89d2f3349516-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "91390fdb-04f1-4764-9cb0-89d2f3349516" (UID: "91390fdb-04f1-4764-9cb0-89d2f3349516"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:54:44.273964 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:44.273963    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91390fdb-04f1-4764-9cb0-89d2f3349516-dshm" (OuterVolumeSpecName: "dshm") pod "91390fdb-04f1-4764-9cb0-89d2f3349516" (UID: "91390fdb-04f1-4764-9cb0-89d2f3349516"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:54:44.373282 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:44.373244    2548 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91390fdb-04f1-4764-9cb0-89d2f3349516-tls-certs\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:54:44.373282 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:44.373279    2548 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91390fdb-04f1-4764-9cb0-89d2f3349516-home\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:54:44.373282 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:44.373291    2548 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91390fdb-04f1-4764-9cb0-89d2f3349516-dshm\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:54:44.373559 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:44.373303    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jdgnf\" (UniqueName: \"kubernetes.io/projected/91390fdb-04f1-4764-9cb0-89d2f3349516-kube-api-access-jdgnf\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:54:44.747782 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:44.747747    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="719c9548-0afc-48d2-b92c-614a6faf2fdf" path="/var/lib/kubelet/pods/719c9548-0afc-48d2-b92c-614a6faf2fdf/volumes"
Jul 23 23:54:45.188160 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:45.188133    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fc333b/sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b"
Jul 23 23:54:45.203674 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:45.203642    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-fc333b/sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b"]
Jul 23 23:54:45.208679 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:45.208656    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-fc333b/sub-test-model-fc333b-kserve-7db46fc5f9-m9j7b"]
Jul 23 23:54:46.747888 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:46.747856    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91390fdb-04f1-4764-9cb0-89d2f3349516" path="/var/lib/kubelet/pods/91390fdb-04f1-4764-9cb0-89d2f3349516/volumes"
Jul 23 23:54:57.002446 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:57.002410    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-867102/rate-test-model-867102-kserve-69cc45dbcf-stttt"]
Jul 23 23:54:57.002820 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:57.002727    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="719c9548-0afc-48d2-b92c-614a6faf2fdf" containerName="main"
Jul 23 23:54:57.002820 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:57.002737    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="719c9548-0afc-48d2-b92c-614a6faf2fdf" containerName="main"
Jul 23 23:54:57.002820 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:57.002750    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91390fdb-04f1-4764-9cb0-89d2f3349516" containerName="main"
Jul 23 23:54:57.002820 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:57.002755    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="91390fdb-04f1-4764-9cb0-89d2f3349516" containerName="main"
Jul 23 23:54:57.002820 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:57.002814    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="719c9548-0afc-48d2-b92c-614a6faf2fdf" containerName="main"
Jul 23 23:54:57.003013 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:57.002825    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="91390fdb-04f1-4764-9cb0-89d2f3349516" containerName="main"
Jul 23 23:54:57.005675 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:57.005659    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-867102/rate-test-model-867102-kserve-69cc45dbcf-stttt"
Jul 23 23:54:57.008924 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:57.008893    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-867102\"/\"kube-root-ca.crt\""
Jul 23 23:54:57.009092 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:57.008896    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-867102\"/\"default-dockercfg-qlbld\""
Jul 23 23:54:57.009092 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:57.008896    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-867102\"/\"rate-test-model-867102-kserve-self-signed-certs\""
Jul 23 23:54:57.009092 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:57.008938    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-867102\"/\"openshift-service-ca.crt\""
Jul 23 23:54:57.015814 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:57.015788    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-867102/rate-test-model-867102-kserve-69cc45dbcf-stttt"]
Jul 23 23:54:57.192288 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:57.192253    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fphnk\" (UniqueName: \"kubernetes.io/projected/a831b5aa-293c-4646-aa69-aa9f229afbeb-kube-api-access-fphnk\") pod \"rate-test-model-867102-kserve-69cc45dbcf-stttt\" (UID: \"a831b5aa-293c-4646-aa69-aa9f229afbeb\") " pod="ai-tenant-e2e-shared-a-867102/rate-test-model-867102-kserve-69cc45dbcf-stttt"
Jul 23 23:54:57.192288 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:57.192291    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a831b5aa-293c-4646-aa69-aa9f229afbeb-tls-certs\") pod \"rate-test-model-867102-kserve-69cc45dbcf-stttt\" (UID: \"a831b5aa-293c-4646-aa69-aa9f229afbeb\") " pod="ai-tenant-e2e-shared-a-867102/rate-test-model-867102-kserve-69cc45dbcf-stttt"
Jul 23 23:54:57.192509 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:57.192326    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a831b5aa-293c-4646-aa69-aa9f229afbeb-tmp-dir\") pod \"rate-test-model-867102-kserve-69cc45dbcf-stttt\" (UID: \"a831b5aa-293c-4646-aa69-aa9f229afbeb\") " pod="ai-tenant-e2e-shared-a-867102/rate-test-model-867102-kserve-69cc45dbcf-stttt"
Jul 23 23:54:57.192509 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:57.192353    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a831b5aa-293c-4646-aa69-aa9f229afbeb-home\") pod \"rate-test-model-867102-kserve-69cc45dbcf-stttt\" (UID: \"a831b5aa-293c-4646-aa69-aa9f229afbeb\") " pod="ai-tenant-e2e-shared-a-867102/rate-test-model-867102-kserve-69cc45dbcf-stttt"
Jul 23 23:54:57.192509 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:57.192398    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a831b5aa-293c-4646-aa69-aa9f229afbeb-dshm\") pod \"rate-test-model-867102-kserve-69cc45dbcf-stttt\" (UID: \"a831b5aa-293c-4646-aa69-aa9f229afbeb\") " pod="ai-tenant-e2e-shared-a-867102/rate-test-model-867102-kserve-69cc45dbcf-stttt"
Jul 23 23:54:57.192509 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:57.192450    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a831b5aa-293c-4646-aa69-aa9f229afbeb-model-cache\") pod \"rate-test-model-867102-kserve-69cc45dbcf-stttt\" (UID: \"a831b5aa-293c-4646-aa69-aa9f229afbeb\") " pod="ai-tenant-e2e-shared-a-867102/rate-test-model-867102-kserve-69cc45dbcf-stttt"
Jul 23 23:54:57.293514 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:57.293415    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a831b5aa-293c-4646-aa69-aa9f229afbeb-model-cache\") pod \"rate-test-model-867102-kserve-69cc45dbcf-stttt\" (UID: \"a831b5aa-293c-4646-aa69-aa9f229afbeb\") " pod="ai-tenant-e2e-shared-a-867102/rate-test-model-867102-kserve-69cc45dbcf-stttt"
Jul 23 23:54:57.293514 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:57.293477    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fphnk\" (UniqueName: \"kubernetes.io/projected/a831b5aa-293c-4646-aa69-aa9f229afbeb-kube-api-access-fphnk\") pod \"rate-test-model-867102-kserve-69cc45dbcf-stttt\" (UID: \"a831b5aa-293c-4646-aa69-aa9f229afbeb\") " pod="ai-tenant-e2e-shared-a-867102/rate-test-model-867102-kserve-69cc45dbcf-stttt"
Jul 23 23:54:57.293514 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:57.293506    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a831b5aa-293c-4646-aa69-aa9f229afbeb-tls-certs\") pod \"rate-test-model-867102-kserve-69cc45dbcf-stttt\" (UID: \"a831b5aa-293c-4646-aa69-aa9f229afbeb\") " pod="ai-tenant-e2e-shared-a-867102/rate-test-model-867102-kserve-69cc45dbcf-stttt"
Jul 23 23:54:57.293755 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:57.293528    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a831b5aa-293c-4646-aa69-aa9f229afbeb-tmp-dir\") pod \"rate-test-model-867102-kserve-69cc45dbcf-stttt\" (UID: \"a831b5aa-293c-4646-aa69-aa9f229afbeb\") " pod="ai-tenant-e2e-shared-a-867102/rate-test-model-867102-kserve-69cc45dbcf-stttt"
Jul 23 23:54:57.293755 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:57.293552    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a831b5aa-293c-4646-aa69-aa9f229afbeb-home\") pod \"rate-test-model-867102-kserve-69cc45dbcf-stttt\" (UID: \"a831b5aa-293c-4646-aa69-aa9f229afbeb\") " pod="ai-tenant-e2e-shared-a-867102/rate-test-model-867102-kserve-69cc45dbcf-stttt"
Jul 23 23:54:57.293755 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:57.293610    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a831b5aa-293c-4646-aa69-aa9f229afbeb-dshm\") pod \"rate-test-model-867102-kserve-69cc45dbcf-stttt\" (UID: \"a831b5aa-293c-4646-aa69-aa9f229afbeb\") " pod="ai-tenant-e2e-shared-a-867102/rate-test-model-867102-kserve-69cc45dbcf-stttt"
Jul 23 23:54:57.293908 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:57.293884    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a831b5aa-293c-4646-aa69-aa9f229afbeb-model-cache\") pod \"rate-test-model-867102-kserve-69cc45dbcf-stttt\" (UID: \"a831b5aa-293c-4646-aa69-aa9f229afbeb\") " pod="ai-tenant-e2e-shared-a-867102/rate-test-model-867102-kserve-69cc45dbcf-stttt"
Jul 23 23:54:57.293956 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:57.293924    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a831b5aa-293c-4646-aa69-aa9f229afbeb-tmp-dir\") pod \"rate-test-model-867102-kserve-69cc45dbcf-stttt\" (UID: \"a831b5aa-293c-4646-aa69-aa9f229afbeb\") " pod="ai-tenant-e2e-shared-a-867102/rate-test-model-867102-kserve-69cc45dbcf-stttt"
Jul 23 23:54:57.294019 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:57.293965    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a831b5aa-293c-4646-aa69-aa9f229afbeb-home\") pod \"rate-test-model-867102-kserve-69cc45dbcf-stttt\" (UID: \"a831b5aa-293c-4646-aa69-aa9f229afbeb\") " pod="ai-tenant-e2e-shared-a-867102/rate-test-model-867102-kserve-69cc45dbcf-stttt"
Jul 23 23:54:57.295943 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:57.295924    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a831b5aa-293c-4646-aa69-aa9f229afbeb-dshm\") pod \"rate-test-model-867102-kserve-69cc45dbcf-stttt\" (UID: \"a831b5aa-293c-4646-aa69-aa9f229afbeb\") " pod="ai-tenant-e2e-shared-a-867102/rate-test-model-867102-kserve-69cc45dbcf-stttt"
Jul 23 23:54:57.296136 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:57.296118    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a831b5aa-293c-4646-aa69-aa9f229afbeb-tls-certs\") pod \"rate-test-model-867102-kserve-69cc45dbcf-stttt\" (UID: \"a831b5aa-293c-4646-aa69-aa9f229afbeb\") " pod="ai-tenant-e2e-shared-a-867102/rate-test-model-867102-kserve-69cc45dbcf-stttt"
Jul 23 23:54:57.301599 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:57.301578    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fphnk\" (UniqueName: \"kubernetes.io/projected/a831b5aa-293c-4646-aa69-aa9f229afbeb-kube-api-access-fphnk\") pod \"rate-test-model-867102-kserve-69cc45dbcf-stttt\" (UID: \"a831b5aa-293c-4646-aa69-aa9f229afbeb\") " pod="ai-tenant-e2e-shared-a-867102/rate-test-model-867102-kserve-69cc45dbcf-stttt"
Jul 23 23:54:57.317061 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:57.317038    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-867102/rate-test-model-867102-kserve-69cc45dbcf-stttt"
Jul 23 23:54:57.453497 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:57.453460    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-867102/rate-test-model-867102-kserve-69cc45dbcf-stttt"]
Jul 23 23:54:57.456629 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:54:57.456595    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda831b5aa_293c_4646_aa69_aa9f229afbeb.slice/crio-082fa8d01f462cabaf82d3b7117c1cfe6105f055da6517f561724365ad8cf983 WatchSource:0}: Error finding container 082fa8d01f462cabaf82d3b7117c1cfe6105f055da6517f561724365ad8cf983: Status 404 returned error can't find the container with id 082fa8d01f462cabaf82d3b7117c1cfe6105f055da6517f561724365ad8cf983
Jul 23 23:54:58.260673 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:58.260639    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-867102/rate-test-model-867102-kserve-69cc45dbcf-stttt" event={"ID":"a831b5aa-293c-4646-aa69-aa9f229afbeb","Type":"ContainerStarted","Data":"c0d99fa0f49ea894fcdf2b84141ecab69f39534420d5eb695fe4726615107023"}
Jul 23 23:54:58.260673 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:58.260680    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-867102/rate-test-model-867102-kserve-69cc45dbcf-stttt" event={"ID":"a831b5aa-293c-4646-aa69-aa9f229afbeb","Type":"ContainerStarted","Data":"082fa8d01f462cabaf82d3b7117c1cfe6105f055da6517f561724365ad8cf983"}
Jul 23 23:54:58.276390 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:54:58.276333    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-867102/rate-test-model-867102-kserve-69cc45dbcf-stttt" podStartSLOduration=2.276315529 podStartE2EDuration="2.276315529s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:54:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:54:58.275918282 +0000 UTC m=+2500.019647344" watchObservedRunningTime="2026-07-23 23:54:58.276315529 +0000 UTC m=+2500.020044578"
Jul 23 23:55:00.375014 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:00.374958    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:55:07.317336 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:07.317301    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-867102/rate-test-model-867102-kserve-69cc45dbcf-stttt"
Jul 23 23:55:07.319719 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:07.317364    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-867102/rate-test-model-867102-kserve-69cc45dbcf-stttt"
Jul 23 23:55:07.330388 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:07.330365    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-867102/rate-test-model-867102-kserve-69cc45dbcf-stttt"
Jul 23 23:55:08.321882 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:08.321852    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-867102/rate-test-model-867102-kserve-69cc45dbcf-stttt"
Jul 23 23:55:22.603296 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:22.603256    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-fc333b/rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr"]
Jul 23 23:55:22.607846 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:22.607825    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fc333b/rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr"
Jul 23 23:55:22.611613 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:22.611581    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-fc333b\"/\"default-dockercfg-2dfzf\""
Jul 23 23:55:22.611613 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:22.611580    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-fc333b\"/\"kube-root-ca.crt\""
Jul 23 23:55:22.611831 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:22.611627    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-fc333b\"/\"openshift-service-ca.crt\""
Jul 23 23:55:22.611831 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:22.611585    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-fc333b\"/\"rate-test-model-fc333b-kserve-self-signed-certs\""
Jul 23 23:55:22.616606 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:22.616584    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-fc333b/rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr"]
Jul 23 23:55:22.680489 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:22.680459    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3eda1d8-e552-48c4-aa0a-359c02392c71-tls-certs\") pod \"rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr\" (UID: \"f3eda1d8-e552-48c4-aa0a-359c02392c71\") " pod="ai-tenant-e2e-shared-b-fc333b/rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr"
Jul 23 23:55:22.680650 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:22.680502    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3eda1d8-e552-48c4-aa0a-359c02392c71-model-cache\") pod \"rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr\" (UID: \"f3eda1d8-e552-48c4-aa0a-359c02392c71\") " pod="ai-tenant-e2e-shared-b-fc333b/rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr"
Jul 23 23:55:22.680650 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:22.680534    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3eda1d8-e552-48c4-aa0a-359c02392c71-tmp-dir\") pod \"rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr\" (UID: \"f3eda1d8-e552-48c4-aa0a-359c02392c71\") " pod="ai-tenant-e2e-shared-b-fc333b/rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr"
Jul 23 23:55:22.680650 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:22.680563    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3eda1d8-e552-48c4-aa0a-359c02392c71-dshm\") pod \"rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr\" (UID: \"f3eda1d8-e552-48c4-aa0a-359c02392c71\") " pod="ai-tenant-e2e-shared-b-fc333b/rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr"
Jul 23 23:55:22.680650 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:22.680594    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxwbg\" (UniqueName: \"kubernetes.io/projected/f3eda1d8-e552-48c4-aa0a-359c02392c71-kube-api-access-bxwbg\") pod \"rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr\" (UID: \"f3eda1d8-e552-48c4-aa0a-359c02392c71\") " pod="ai-tenant-e2e-shared-b-fc333b/rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr"
Jul 23 23:55:22.680787 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:22.680698    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3eda1d8-e552-48c4-aa0a-359c02392c71-home\") pod \"rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr\" (UID: \"f3eda1d8-e552-48c4-aa0a-359c02392c71\") " pod="ai-tenant-e2e-shared-b-fc333b/rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr"
Jul 23 23:55:22.781822 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:22.781790    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3eda1d8-e552-48c4-aa0a-359c02392c71-model-cache\") pod \"rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr\" (UID: \"f3eda1d8-e552-48c4-aa0a-359c02392c71\") " pod="ai-tenant-e2e-shared-b-fc333b/rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr"
Jul 23 23:55:22.782041 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:22.781832    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3eda1d8-e552-48c4-aa0a-359c02392c71-tmp-dir\") pod \"rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr\" (UID: \"f3eda1d8-e552-48c4-aa0a-359c02392c71\") " pod="ai-tenant-e2e-shared-b-fc333b/rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr"
Jul 23 23:55:22.782041 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:22.781861    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3eda1d8-e552-48c4-aa0a-359c02392c71-dshm\") pod \"rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr\" (UID: \"f3eda1d8-e552-48c4-aa0a-359c02392c71\") " pod="ai-tenant-e2e-shared-b-fc333b/rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr"
Jul 23 23:55:22.782041 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:22.781902    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxwbg\" (UniqueName: \"kubernetes.io/projected/f3eda1d8-e552-48c4-aa0a-359c02392c71-kube-api-access-bxwbg\") pod \"rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr\" (UID: \"f3eda1d8-e552-48c4-aa0a-359c02392c71\") " pod="ai-tenant-e2e-shared-b-fc333b/rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr"
Jul 23 23:55:22.782041 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:22.781936    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3eda1d8-e552-48c4-aa0a-359c02392c71-home\") pod \"rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr\" (UID: \"f3eda1d8-e552-48c4-aa0a-359c02392c71\") " pod="ai-tenant-e2e-shared-b-fc333b/rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr"
Jul 23 23:55:22.782041 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:22.782036    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3eda1d8-e552-48c4-aa0a-359c02392c71-tls-certs\") pod \"rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr\" (UID: \"f3eda1d8-e552-48c4-aa0a-359c02392c71\") " pod="ai-tenant-e2e-shared-b-fc333b/rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr"
Jul 23 23:55:22.782326 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:22.782236    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3eda1d8-e552-48c4-aa0a-359c02392c71-tmp-dir\") pod \"rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr\" (UID: \"f3eda1d8-e552-48c4-aa0a-359c02392c71\") " pod="ai-tenant-e2e-shared-b-fc333b/rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr"
Jul 23 23:55:22.782326 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:22.782302    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3eda1d8-e552-48c4-aa0a-359c02392c71-model-cache\") pod \"rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr\" (UID: \"f3eda1d8-e552-48c4-aa0a-359c02392c71\") " pod="ai-tenant-e2e-shared-b-fc333b/rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr"
Jul 23 23:55:22.782441 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:22.782336    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3eda1d8-e552-48c4-aa0a-359c02392c71-home\") pod \"rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr\" (UID: \"f3eda1d8-e552-48c4-aa0a-359c02392c71\") " pod="ai-tenant-e2e-shared-b-fc333b/rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr"
Jul 23 23:55:22.784303 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:22.784272    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3eda1d8-e552-48c4-aa0a-359c02392c71-dshm\") pod \"rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr\" (UID: \"f3eda1d8-e552-48c4-aa0a-359c02392c71\") " pod="ai-tenant-e2e-shared-b-fc333b/rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr"
Jul 23 23:55:22.784412 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:22.784369    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3eda1d8-e552-48c4-aa0a-359c02392c71-tls-certs\") pod \"rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr\" (UID: \"f3eda1d8-e552-48c4-aa0a-359c02392c71\") " pod="ai-tenant-e2e-shared-b-fc333b/rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr"
Jul 23 23:55:22.789654 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:22.789635    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxwbg\" (UniqueName: \"kubernetes.io/projected/f3eda1d8-e552-48c4-aa0a-359c02392c71-kube-api-access-bxwbg\") pod \"rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr\" (UID: \"f3eda1d8-e552-48c4-aa0a-359c02392c71\") " pod="ai-tenant-e2e-shared-b-fc333b/rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr"
Jul 23 23:55:22.917813 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:22.917781    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fc333b/rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr"
Jul 23 23:55:23.050866 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:23.050718    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-fc333b/rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr"]
Jul 23 23:55:23.053541 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:55:23.053515    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3eda1d8_e552_48c4_aa0a_359c02392c71.slice/crio-bb2745101261e0a444b6a62354dca6314d025dc5703ab1b4a262cb1a57e91354 WatchSource:0}: Error finding container bb2745101261e0a444b6a62354dca6314d025dc5703ab1b4a262cb1a57e91354: Status 404 returned error can't find the container with id bb2745101261e0a444b6a62354dca6314d025dc5703ab1b4a262cb1a57e91354
Jul 23 23:55:23.385631 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:23.385538    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fc333b/rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr" event={"ID":"f3eda1d8-e552-48c4-aa0a-359c02392c71","Type":"ContainerStarted","Data":"27ed354170710a841217fe48084050ecbd07780bc1da2de4b82325d95e15f3d9"}
Jul 23 23:55:23.385631 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:23.385581    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fc333b/rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr" event={"ID":"f3eda1d8-e552-48c4-aa0a-359c02392c71","Type":"ContainerStarted","Data":"bb2745101261e0a444b6a62354dca6314d025dc5703ab1b4a262cb1a57e91354"}
Jul 23 23:55:23.401337 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:23.401289    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-fc333b/rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr" podStartSLOduration=1.401272181 podStartE2EDuration="1.401272181s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:55:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:55:23.400048241 +0000 UTC m=+2525.143777290" watchObservedRunningTime="2026-07-23 23:55:23.401272181 +0000 UTC m=+2525.145001230"
Jul 23 23:55:26.576596 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:26.576554    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:55:32.918749 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:32.918715    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-fc333b/rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr"
Jul 23 23:55:32.919336 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:32.918790    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-fc333b/rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr"
Jul 23 23:55:32.931469 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:32.931440    2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-fc333b/rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr"
Jul 23 23:55:33.446340 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:33.446313    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-fc333b/rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr"
Jul 23 23:55:40.782114 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:55:40.782078    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:56:03.079437 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:03.079403    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:56:06.796659 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:06.796625    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-867102/rate-test-model-867102-kserve-69cc45dbcf-stttt"]
Jul 23 23:56:07.690644 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:07.690588    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-fc333b/rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr"]
Jul 23 23:56:15.491536 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:15.491503    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7647-139623459661408"]
Jul 23 23:56:15.494872 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:15.494855    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7647-139623459661408"
Jul 23 23:56:15.497166 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:15.497148    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-cdk2h\""
Jul 23 23:56:15.500707 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:15.500686    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7647-139623459661408"]
Jul 23 23:56:15.547625 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:15.547590    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzspg\" (UniqueName: \"kubernetes.io/projected/30f57c3e-ec91-4441-b396-43594bd5c4b0-kube-api-access-zzspg\") pod \"test-curl-7647-139623459661408\" (UID: \"30f57c3e-ec91-4441-b396-43594bd5c4b0\") " pod="opendatahub/test-curl-7647-139623459661408"
Jul 23 23:56:15.648129 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:15.648092    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzspg\" (UniqueName: \"kubernetes.io/projected/30f57c3e-ec91-4441-b396-43594bd5c4b0-kube-api-access-zzspg\") pod \"test-curl-7647-139623459661408\" (UID: \"30f57c3e-ec91-4441-b396-43594bd5c4b0\") " pod="opendatahub/test-curl-7647-139623459661408"
Jul 23 23:56:15.655429 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:15.655398    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzspg\" (UniqueName: \"kubernetes.io/projected/30f57c3e-ec91-4441-b396-43594bd5c4b0-kube-api-access-zzspg\") pod \"test-curl-7647-139623459661408\" (UID: \"30f57c3e-ec91-4441-b396-43594bd5c4b0\") " pod="opendatahub/test-curl-7647-139623459661408"
Jul 23 23:56:15.805413 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:15.805323    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7647-139623459661408"
Jul 23 23:56:15.933206 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:15.933182    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7647-139623459661408"]
Jul 23 23:56:15.935697 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:56:15.935669    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod30f57c3e_ec91_4441_b396_43594bd5c4b0.slice/crio-7e2a1927b81671982c97e7d49e8d97e2b8a8a91e9577a0aff3926899d01b56d8 WatchSource:0}: Error finding container 7e2a1927b81671982c97e7d49e8d97e2b8a8a91e9577a0aff3926899d01b56d8: Status 404 returned error can't find the container with id 7e2a1927b81671982c97e7d49e8d97e2b8a8a91e9577a0aff3926899d01b56d8
Jul 23 23:56:15.937315 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:15.937294    2548 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 23:56:16.654229 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:16.654199    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7647-139623459661408" event={"ID":"30f57c3e-ec91-4441-b396-43594bd5c4b0","Type":"ContainerStarted","Data":"7e2a1927b81671982c97e7d49e8d97e2b8a8a91e9577a0aff3926899d01b56d8"}
Jul 23 23:56:17.660558 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:17.660524    2548 generic.go:356] "Generic (PLEG): container finished" podID="30f57c3e-ec91-4441-b396-43594bd5c4b0" containerID="4d06e462b1448202aa50b5738012518558a2050db41dd2044eb8c05bf930225b" exitCode=0
Jul 23 23:56:17.661019 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:17.660567    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7647-139623459661408" event={"ID":"30f57c3e-ec91-4441-b396-43594bd5c4b0","Type":"ContainerDied","Data":"4d06e462b1448202aa50b5738012518558a2050db41dd2044eb8c05bf930225b"}
Jul 23 23:56:18.795184 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:18.795154    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7647-139623459661408"
Jul 23 23:56:18.827584 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:18.827555    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7647-139623459661408_30f57c3e-ec91-4441-b396-43594bd5c4b0/test-curl-7647-139623459661408/0.log"
Jul 23 23:56:18.851713 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:18.851686    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7647-139623459661408"]
Jul 23 23:56:18.855149 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:18.855127    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7647-139623459661408"]
Jul 23 23:56:18.872626 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:18.872596    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zzspg\" (UniqueName: \"kubernetes.io/projected/30f57c3e-ec91-4441-b396-43594bd5c4b0-kube-api-access-zzspg\") pod \"30f57c3e-ec91-4441-b396-43594bd5c4b0\" (UID: \"30f57c3e-ec91-4441-b396-43594bd5c4b0\") "
Jul 23 23:56:18.874793 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:18.874772    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30f57c3e-ec91-4441-b396-43594bd5c4b0-kube-api-access-zzspg" (OuterVolumeSpecName: "kube-api-access-zzspg") pod "30f57c3e-ec91-4441-b396-43594bd5c4b0" (UID: "30f57c3e-ec91-4441-b396-43594bd5c4b0"). InnerVolumeSpecName "kube-api-access-zzspg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:56:18.883003 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:18.882968    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:56:18.974042 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:18.973927    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zzspg\" (UniqueName: \"kubernetes.io/projected/30f57c3e-ec91-4441-b396-43594bd5c4b0-kube-api-access-zzspg\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:56:18.984183 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:18.984154    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7647-139623459661408"]
Jul 23 23:56:18.984479 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:18.984466    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30f57c3e-ec91-4441-b396-43594bd5c4b0" containerName="test-curl-7647-139623459661408"
Jul 23 23:56:18.984523 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:18.984481    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="30f57c3e-ec91-4441-b396-43594bd5c4b0" containerName="test-curl-7647-139623459661408"
Jul 23 23:56:18.984559 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:18.984537    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="30f57c3e-ec91-4441-b396-43594bd5c4b0" containerName="test-curl-7647-139623459661408"
Jul 23 23:56:18.987457 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:18.987442    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7647-139623459661408"
Jul 23 23:56:18.993510 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:18.993488    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7647-139623459661408"]
Jul 23 23:56:19.074624 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:19.074574    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6vhq\" (UniqueName: \"kubernetes.io/projected/c86afec9-2166-4228-80fe-8357eec1a8e8-kube-api-access-d6vhq\") pod \"test-curl-7647-139623459661408\" (UID: \"c86afec9-2166-4228-80fe-8357eec1a8e8\") " pod="opendatahub/test-curl-7647-139623459661408"
Jul 23 23:56:19.175281 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:19.175243    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6vhq\" (UniqueName: \"kubernetes.io/projected/c86afec9-2166-4228-80fe-8357eec1a8e8-kube-api-access-d6vhq\") pod \"test-curl-7647-139623459661408\" (UID: \"c86afec9-2166-4228-80fe-8357eec1a8e8\") " pod="opendatahub/test-curl-7647-139623459661408"
Jul 23 23:56:19.182992 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:19.182956    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6vhq\" (UniqueName: \"kubernetes.io/projected/c86afec9-2166-4228-80fe-8357eec1a8e8-kube-api-access-d6vhq\") pod \"test-curl-7647-139623459661408\" (UID: \"c86afec9-2166-4228-80fe-8357eec1a8e8\") " pod="opendatahub/test-curl-7647-139623459661408"
Jul 23 23:56:19.297618 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:19.297524    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7647-139623459661408"
Jul 23 23:56:19.428937 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:19.428914    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7647-139623459661408"]
Jul 23 23:56:19.430800 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:56:19.430775    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc86afec9_2166_4228_80fe_8357eec1a8e8.slice/crio-3b48241433364dcb734b8c6cbf25514478f7693cbaef28326060d8e78adeaf53 WatchSource:0}: Error finding container 3b48241433364dcb734b8c6cbf25514478f7693cbaef28326060d8e78adeaf53: Status 404 returned error can't find the container with id 3b48241433364dcb734b8c6cbf25514478f7693cbaef28326060d8e78adeaf53
Jul 23 23:56:19.672386 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:19.672348    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7647-139623459661408" event={"ID":"c86afec9-2166-4228-80fe-8357eec1a8e8","Type":"ContainerStarted","Data":"3b48241433364dcb734b8c6cbf25514478f7693cbaef28326060d8e78adeaf53"}
Jul 23 23:56:19.673546 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:19.673528    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7647-139623459661408"
Jul 23 23:56:19.673665 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:19.673573    2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7e2a1927b81671982c97e7d49e8d97e2b8a8a91e9577a0aff3926899d01b56d8"
Jul 23 23:56:19.684588 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:19.684558    2548 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7647-139623459661408" oldPodUID="30f57c3e-ec91-4441-b396-43594bd5c4b0" podUID="c86afec9-2166-4228-80fe-8357eec1a8e8"
Jul 23 23:56:20.679594 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:20.679556    2548 generic.go:356] "Generic (PLEG): container finished" podID="c86afec9-2166-4228-80fe-8357eec1a8e8" containerID="0ee3b4b8b93fd290b65f678c405733e005d5e45e79205db94972c9ef7992c595" exitCode=0
Jul 23 23:56:20.680006 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:20.679640    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7647-139623459661408" event={"ID":"c86afec9-2166-4228-80fe-8357eec1a8e8","Type":"ContainerDied","Data":"0ee3b4b8b93fd290b65f678c405733e005d5e45e79205db94972c9ef7992c595"}
Jul 23 23:56:20.747591 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:20.747559    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30f57c3e-ec91-4441-b396-43594bd5c4b0" path="/var/lib/kubelet/pods/30f57c3e-ec91-4441-b396-43594bd5c4b0/volumes"
Jul 23 23:56:21.812965 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:21.812935    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7647-139623459661408"
Jul 23 23:56:21.815351 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:21.815320    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-867102/rate-test-model-867102-kserve-69cc45dbcf-stttt" podUID="a831b5aa-293c-4646-aa69-aa9f229afbeb" containerName="main" containerID="cri-o://c0d99fa0f49ea894fcdf2b84141ecab69f39534420d5eb695fe4726615107023" gracePeriod=45
Jul 23 23:56:21.831938 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:21.831886    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7647-139623459661408_c86afec9-2166-4228-80fe-8357eec1a8e8/test-curl-7647-139623459661408/0.log"
Jul 23 23:56:21.854313 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:21.854285    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7647-139623459661408"]
Jul 23 23:56:21.858238 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:21.858213    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7647-139623459661408"]
Jul 23 23:56:21.899664 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:21.899637    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d6vhq\" (UniqueName: \"kubernetes.io/projected/c86afec9-2166-4228-80fe-8357eec1a8e8-kube-api-access-d6vhq\") pod \"c86afec9-2166-4228-80fe-8357eec1a8e8\" (UID: \"c86afec9-2166-4228-80fe-8357eec1a8e8\") "
Jul 23 23:56:21.901736 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:21.901709    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c86afec9-2166-4228-80fe-8357eec1a8e8-kube-api-access-d6vhq" (OuterVolumeSpecName: "kube-api-access-d6vhq") pod "c86afec9-2166-4228-80fe-8357eec1a8e8" (UID: "c86afec9-2166-4228-80fe-8357eec1a8e8"). InnerVolumeSpecName "kube-api-access-d6vhq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:56:22.000626 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:22.000590    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6vhq\" (UniqueName: \"kubernetes.io/projected/c86afec9-2166-4228-80fe-8357eec1a8e8-kube-api-access-d6vhq\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:56:22.060913 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:22.060886    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-867102/rate-test-model-867102-kserve-69cc45dbcf-stttt"
Jul 23 23:56:22.101794 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:22.101763    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a831b5aa-293c-4646-aa69-aa9f229afbeb-home\") pod \"a831b5aa-293c-4646-aa69-aa9f229afbeb\" (UID: \"a831b5aa-293c-4646-aa69-aa9f229afbeb\") "
Jul 23 23:56:22.101994 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:22.101806    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fphnk\" (UniqueName: \"kubernetes.io/projected/a831b5aa-293c-4646-aa69-aa9f229afbeb-kube-api-access-fphnk\") pod \"a831b5aa-293c-4646-aa69-aa9f229afbeb\" (UID: \"a831b5aa-293c-4646-aa69-aa9f229afbeb\") "
Jul 23 23:56:22.101994 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:22.101852    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a831b5aa-293c-4646-aa69-aa9f229afbeb-tls-certs\") pod \"a831b5aa-293c-4646-aa69-aa9f229afbeb\" (UID: \"a831b5aa-293c-4646-aa69-aa9f229afbeb\") "
Jul 23 23:56:22.101994 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:22.101875    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a831b5aa-293c-4646-aa69-aa9f229afbeb-tmp-dir\") pod \"a831b5aa-293c-4646-aa69-aa9f229afbeb\" (UID: \"a831b5aa-293c-4646-aa69-aa9f229afbeb\") "
Jul 23 23:56:22.101994 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:22.101944    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a831b5aa-293c-4646-aa69-aa9f229afbeb-model-cache\") pod \"a831b5aa-293c-4646-aa69-aa9f229afbeb\" (UID: \"a831b5aa-293c-4646-aa69-aa9f229afbeb\") "
Jul 23 23:56:22.102224 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:22.102034    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a831b5aa-293c-4646-aa69-aa9f229afbeb-home" (OuterVolumeSpecName: "home") pod "a831b5aa-293c-4646-aa69-aa9f229afbeb" (UID: "a831b5aa-293c-4646-aa69-aa9f229afbeb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:56:22.102224 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:22.102054    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a831b5aa-293c-4646-aa69-aa9f229afbeb-dshm\") pod \"a831b5aa-293c-4646-aa69-aa9f229afbeb\" (UID: \"a831b5aa-293c-4646-aa69-aa9f229afbeb\") "
Jul 23 23:56:22.102224 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:22.102159    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a831b5aa-293c-4646-aa69-aa9f229afbeb-model-cache" (OuterVolumeSpecName: "model-cache") pod "a831b5aa-293c-4646-aa69-aa9f229afbeb" (UID: "a831b5aa-293c-4646-aa69-aa9f229afbeb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:56:22.102224 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:22.102196    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a831b5aa-293c-4646-aa69-aa9f229afbeb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a831b5aa-293c-4646-aa69-aa9f229afbeb" (UID: "a831b5aa-293c-4646-aa69-aa9f229afbeb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:56:22.102373 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:22.102354    2548 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a831b5aa-293c-4646-aa69-aa9f229afbeb-home\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:56:22.102414 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:22.102379    2548 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a831b5aa-293c-4646-aa69-aa9f229afbeb-tmp-dir\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:56:22.102414 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:22.102393    2548 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a831b5aa-293c-4646-aa69-aa9f229afbeb-model-cache\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:56:22.104061 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:22.104040    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a831b5aa-293c-4646-aa69-aa9f229afbeb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a831b5aa-293c-4646-aa69-aa9f229afbeb" (UID: "a831b5aa-293c-4646-aa69-aa9f229afbeb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:56:22.104061 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:22.104045    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a831b5aa-293c-4646-aa69-aa9f229afbeb-dshm" (OuterVolumeSpecName: "dshm") pod "a831b5aa-293c-4646-aa69-aa9f229afbeb" (UID: "a831b5aa-293c-4646-aa69-aa9f229afbeb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:56:22.104169 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:22.104065    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a831b5aa-293c-4646-aa69-aa9f229afbeb-kube-api-access-fphnk" (OuterVolumeSpecName: "kube-api-access-fphnk") pod "a831b5aa-293c-4646-aa69-aa9f229afbeb" (UID: "a831b5aa-293c-4646-aa69-aa9f229afbeb"). InnerVolumeSpecName "kube-api-access-fphnk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:56:22.203606 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:22.203569    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fphnk\" (UniqueName: \"kubernetes.io/projected/a831b5aa-293c-4646-aa69-aa9f229afbeb-kube-api-access-fphnk\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:56:22.203606 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:22.203596    2548 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a831b5aa-293c-4646-aa69-aa9f229afbeb-tls-certs\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:56:22.203606 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:22.203605    2548 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a831b5aa-293c-4646-aa69-aa9f229afbeb-dshm\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:56:22.691166 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:22.691135    2548 generic.go:356] "Generic (PLEG): container finished" podID="a831b5aa-293c-4646-aa69-aa9f229afbeb" containerID="c0d99fa0f49ea894fcdf2b84141ecab69f39534420d5eb695fe4726615107023" exitCode=0
Jul 23 23:56:22.691360 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:22.691185    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-867102/rate-test-model-867102-kserve-69cc45dbcf-stttt"
Jul 23 23:56:22.691360 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:22.691218    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-867102/rate-test-model-867102-kserve-69cc45dbcf-stttt" event={"ID":"a831b5aa-293c-4646-aa69-aa9f229afbeb","Type":"ContainerDied","Data":"c0d99fa0f49ea894fcdf2b84141ecab69f39534420d5eb695fe4726615107023"}
Jul 23 23:56:22.691360 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:22.691258    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-867102/rate-test-model-867102-kserve-69cc45dbcf-stttt" event={"ID":"a831b5aa-293c-4646-aa69-aa9f229afbeb","Type":"ContainerDied","Data":"082fa8d01f462cabaf82d3b7117c1cfe6105f055da6517f561724365ad8cf983"}
Jul 23 23:56:22.691360 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:22.691280    2548 scope.go:117] "RemoveContainer" containerID="c0d99fa0f49ea894fcdf2b84141ecab69f39534420d5eb695fe4726615107023"
Jul 23 23:56:22.692682 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:22.692525    2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3b48241433364dcb734b8c6cbf25514478f7693cbaef28326060d8e78adeaf53"
Jul 23 23:56:22.692682 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:22.692563    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7647-139623459661408"
Jul 23 23:56:22.700019 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:22.699999    2548 scope.go:117] "RemoveContainer" containerID="c0d99fa0f49ea894fcdf2b84141ecab69f39534420d5eb695fe4726615107023"
Jul 23 23:56:22.700324 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:56:22.700303    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0d99fa0f49ea894fcdf2b84141ecab69f39534420d5eb695fe4726615107023\": container with ID starting with c0d99fa0f49ea894fcdf2b84141ecab69f39534420d5eb695fe4726615107023 not found: ID does not exist" containerID="c0d99fa0f49ea894fcdf2b84141ecab69f39534420d5eb695fe4726615107023"
Jul 23 23:56:22.700427 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:22.700335    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0d99fa0f49ea894fcdf2b84141ecab69f39534420d5eb695fe4726615107023"} err="failed to get container status \"c0d99fa0f49ea894fcdf2b84141ecab69f39534420d5eb695fe4726615107023\": rpc error: code = NotFound desc = could not find container \"c0d99fa0f49ea894fcdf2b84141ecab69f39534420d5eb695fe4726615107023\": container with ID starting with c0d99fa0f49ea894fcdf2b84141ecab69f39534420d5eb695fe4726615107023 not found: ID does not exist"
Jul 23 23:56:22.709579 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:22.709523    2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-fc333b/rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr" podUID="f3eda1d8-e552-48c4-aa0a-359c02392c71" containerName="main" containerID="cri-o://27ed354170710a841217fe48084050ecbd07780bc1da2de4b82325d95e15f3d9" gracePeriod=45
Jul 23 23:56:22.719052 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:22.719028    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-867102/rate-test-model-867102-kserve-69cc45dbcf-stttt"]
Jul 23 23:56:22.734266 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:56:22.734235    2548 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7647-139623459661408\" is forbidden: User \"system:node:ip-10-0-129-178.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-178.ec2.internal' and this object" podUID="c86afec9-2166-4228-80fe-8357eec1a8e8" pod="opendatahub/test-curl-7647-139623459661408"
Jul 23 23:56:22.736026 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:56:22.736001    2548 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7647-139623459661408\" is forbidden: User \"system:node:ip-10-0-129-178.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-178.ec2.internal' and this object" podUID="c86afec9-2166-4228-80fe-8357eec1a8e8" pod="opendatahub/test-curl-7647-139623459661408"
Jul 23 23:56:22.737159 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:22.737136    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-867102/rate-test-model-867102-kserve-69cc45dbcf-stttt"]
Jul 23 23:56:22.747832 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:22.747805    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a831b5aa-293c-4646-aa69-aa9f229afbeb" path="/var/lib/kubelet/pods/a831b5aa-293c-4646-aa69-aa9f229afbeb/volumes"
Jul 23 23:56:22.748212 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:22.748194    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c86afec9-2166-4228-80fe-8357eec1a8e8" path="/var/lib/kubelet/pods/c86afec9-2166-4228-80fe-8357eec1a8e8/volumes"
Jul 23 23:56:22.983760 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:22.983735    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fc333b/rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr"
Jul 23 23:56:23.010945 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:23.010914    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3eda1d8-e552-48c4-aa0a-359c02392c71-tls-certs\") pod \"f3eda1d8-e552-48c4-aa0a-359c02392c71\" (UID: \"f3eda1d8-e552-48c4-aa0a-359c02392c71\") "
Jul 23 23:56:23.011151 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:23.010986    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3eda1d8-e552-48c4-aa0a-359c02392c71-tmp-dir\") pod \"f3eda1d8-e552-48c4-aa0a-359c02392c71\" (UID: \"f3eda1d8-e552-48c4-aa0a-359c02392c71\") "
Jul 23 23:56:23.011151 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:23.011019    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3eda1d8-e552-48c4-aa0a-359c02392c71-model-cache\") pod \"f3eda1d8-e552-48c4-aa0a-359c02392c71\" (UID: \"f3eda1d8-e552-48c4-aa0a-359c02392c71\") "
Jul 23 23:56:23.011151 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:23.011043    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3eda1d8-e552-48c4-aa0a-359c02392c71-dshm\") pod \"f3eda1d8-e552-48c4-aa0a-359c02392c71\" (UID: \"f3eda1d8-e552-48c4-aa0a-359c02392c71\") "
Jul 23 23:56:23.011151 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:23.011066    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bxwbg\" (UniqueName: \"kubernetes.io/projected/f3eda1d8-e552-48c4-aa0a-359c02392c71-kube-api-access-bxwbg\") pod \"f3eda1d8-e552-48c4-aa0a-359c02392c71\" (UID: \"f3eda1d8-e552-48c4-aa0a-359c02392c71\") "
Jul 23 23:56:23.011151 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:23.011118    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3eda1d8-e552-48c4-aa0a-359c02392c71-home\") pod \"f3eda1d8-e552-48c4-aa0a-359c02392c71\" (UID: \"f3eda1d8-e552-48c4-aa0a-359c02392c71\") "
Jul 23 23:56:23.011439 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:23.011326    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3eda1d8-e552-48c4-aa0a-359c02392c71-model-cache" (OuterVolumeSpecName: "model-cache") pod "f3eda1d8-e552-48c4-aa0a-359c02392c71" (UID: "f3eda1d8-e552-48c4-aa0a-359c02392c71"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:56:23.011585 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:23.011563    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3eda1d8-e552-48c4-aa0a-359c02392c71-home" (OuterVolumeSpecName: "home") pod "f3eda1d8-e552-48c4-aa0a-359c02392c71" (UID: "f3eda1d8-e552-48c4-aa0a-359c02392c71"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:56:23.011585 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:23.011571    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3eda1d8-e552-48c4-aa0a-359c02392c71-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f3eda1d8-e552-48c4-aa0a-359c02392c71" (UID: "f3eda1d8-e552-48c4-aa0a-359c02392c71"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:56:23.013627 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:23.013596    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3eda1d8-e552-48c4-aa0a-359c02392c71-kube-api-access-bxwbg" (OuterVolumeSpecName: "kube-api-access-bxwbg") pod "f3eda1d8-e552-48c4-aa0a-359c02392c71" (UID: "f3eda1d8-e552-48c4-aa0a-359c02392c71"). InnerVolumeSpecName "kube-api-access-bxwbg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:56:23.013732 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:23.013631    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3eda1d8-e552-48c4-aa0a-359c02392c71-dshm" (OuterVolumeSpecName: "dshm") pod "f3eda1d8-e552-48c4-aa0a-359c02392c71" (UID: "f3eda1d8-e552-48c4-aa0a-359c02392c71"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:56:23.013732 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:23.013660    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3eda1d8-e552-48c4-aa0a-359c02392c71-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f3eda1d8-e552-48c4-aa0a-359c02392c71" (UID: "f3eda1d8-e552-48c4-aa0a-359c02392c71"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:56:23.112332 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:23.112238    2548 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3eda1d8-e552-48c4-aa0a-359c02392c71-tls-certs\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:56:23.112332 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:23.112269    2548 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3eda1d8-e552-48c4-aa0a-359c02392c71-tmp-dir\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:56:23.112332 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:23.112278    2548 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3eda1d8-e552-48c4-aa0a-359c02392c71-model-cache\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:56:23.112332 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:23.112288    2548 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3eda1d8-e552-48c4-aa0a-359c02392c71-dshm\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:56:23.112332 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:23.112297    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxwbg\" (UniqueName: \"kubernetes.io/projected/f3eda1d8-e552-48c4-aa0a-359c02392c71-kube-api-access-bxwbg\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:56:23.112332 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:23.112308    2548 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3eda1d8-e552-48c4-aa0a-359c02392c71-home\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:56:23.699251 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:23.699219    2548 generic.go:356] "Generic (PLEG): container finished" podID="f3eda1d8-e552-48c4-aa0a-359c02392c71" containerID="27ed354170710a841217fe48084050ecbd07780bc1da2de4b82325d95e15f3d9" exitCode=0
Jul 23 23:56:23.699251 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:23.699257    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fc333b/rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr" event={"ID":"f3eda1d8-e552-48c4-aa0a-359c02392c71","Type":"ContainerDied","Data":"27ed354170710a841217fe48084050ecbd07780bc1da2de4b82325d95e15f3d9"}
Jul 23 23:56:23.699513 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:23.699280    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fc333b/rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr"
Jul 23 23:56:23.699513 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:23.699293    2548 scope.go:117] "RemoveContainer" containerID="27ed354170710a841217fe48084050ecbd07780bc1da2de4b82325d95e15f3d9"
Jul 23 23:56:23.699513 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:23.699283    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fc333b/rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr" event={"ID":"f3eda1d8-e552-48c4-aa0a-359c02392c71","Type":"ContainerDied","Data":"bb2745101261e0a444b6a62354dca6314d025dc5703ab1b4a262cb1a57e91354"}
Jul 23 23:56:23.708790 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:23.708773    2548 scope.go:117] "RemoveContainer" containerID="27ed354170710a841217fe48084050ecbd07780bc1da2de4b82325d95e15f3d9"
Jul 23 23:56:23.709054 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:56:23.709035    2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27ed354170710a841217fe48084050ecbd07780bc1da2de4b82325d95e15f3d9\": container with ID starting with 27ed354170710a841217fe48084050ecbd07780bc1da2de4b82325d95e15f3d9 not found: ID does not exist" containerID="27ed354170710a841217fe48084050ecbd07780bc1da2de4b82325d95e15f3d9"
Jul 23 23:56:23.709130 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:23.709060    2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27ed354170710a841217fe48084050ecbd07780bc1da2de4b82325d95e15f3d9"} err="failed to get container status \"27ed354170710a841217fe48084050ecbd07780bc1da2de4b82325d95e15f3d9\": rpc error: code = NotFound desc = could not find container \"27ed354170710a841217fe48084050ecbd07780bc1da2de4b82325d95e15f3d9\": container with ID starting with 27ed354170710a841217fe48084050ecbd07780bc1da2de4b82325d95e15f3d9 not found: ID does not exist"
Jul 23 23:56:23.721130 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:23.721103    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-fc333b/rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr"]
Jul 23 23:56:23.726323 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:23.726301    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-fc333b/rate-test-model-fc333b-kserve-665cbb5f8c-nqzsr"]
Jul 23 23:56:24.502482 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:24.502446    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7647-139623459657168"]
Jul 23 23:56:24.502866 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:24.502780    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a831b5aa-293c-4646-aa69-aa9f229afbeb" containerName="main"
Jul 23 23:56:24.502866 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:24.502791    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="a831b5aa-293c-4646-aa69-aa9f229afbeb" containerName="main"
Jul 23 23:56:24.502866 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:24.502805    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c86afec9-2166-4228-80fe-8357eec1a8e8" containerName="test-curl-7647-139623459661408"
Jul 23 23:56:24.502866 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:24.502812    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="c86afec9-2166-4228-80fe-8357eec1a8e8" containerName="test-curl-7647-139623459661408"
Jul 23 23:56:24.502866 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:24.502826    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3eda1d8-e552-48c4-aa0a-359c02392c71" containerName="main"
Jul 23 23:56:24.502866 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:24.502832    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3eda1d8-e552-48c4-aa0a-359c02392c71" containerName="main"
Jul 23 23:56:24.503107 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:24.502882    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="a831b5aa-293c-4646-aa69-aa9f229afbeb" containerName="main"
Jul 23 23:56:24.503107 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:24.502890    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3eda1d8-e552-48c4-aa0a-359c02392c71" containerName="main"
Jul 23 23:56:24.503107 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:24.502897    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="c86afec9-2166-4228-80fe-8357eec1a8e8" containerName="test-curl-7647-139623459661408"
Jul 23 23:56:24.507142 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:24.507127    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7647-139623459657168"
Jul 23 23:56:24.509891 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:24.509869    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-cdk2h\""
Jul 23 23:56:24.511764 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:24.511740    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7647-139623459657168"]
Jul 23 23:56:24.525372 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:24.525345    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcbzn\" (UniqueName: \"kubernetes.io/projected/7036daa8-2a72-4d88-89f1-b872ed4e7804-kube-api-access-lcbzn\") pod \"test-curl-7647-139623459657168\" (UID: \"7036daa8-2a72-4d88-89f1-b872ed4e7804\") " pod="opendatahub/test-curl-7647-139623459657168"
Jul 23 23:56:24.626770 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:24.626735    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcbzn\" (UniqueName: \"kubernetes.io/projected/7036daa8-2a72-4d88-89f1-b872ed4e7804-kube-api-access-lcbzn\") pod \"test-curl-7647-139623459657168\" (UID: \"7036daa8-2a72-4d88-89f1-b872ed4e7804\") " pod="opendatahub/test-curl-7647-139623459657168"
Jul 23 23:56:24.635547 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:24.635508    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcbzn\" (UniqueName: \"kubernetes.io/projected/7036daa8-2a72-4d88-89f1-b872ed4e7804-kube-api-access-lcbzn\") pod \"test-curl-7647-139623459657168\" (UID: \"7036daa8-2a72-4d88-89f1-b872ed4e7804\") " pod="opendatahub/test-curl-7647-139623459657168"
Jul 23 23:56:24.747450 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:24.747419    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3eda1d8-e552-48c4-aa0a-359c02392c71" path="/var/lib/kubelet/pods/f3eda1d8-e552-48c4-aa0a-359c02392c71/volumes"
Jul 23 23:56:24.816749 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:24.816663    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7647-139623459657168"
Jul 23 23:56:24.943441 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:24.943410    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7647-139623459657168"]
Jul 23 23:56:24.946048 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:56:24.946012    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7036daa8_2a72_4d88_89f1_b872ed4e7804.slice/crio-d5b6bc03db00fb2fdb5f93cc5c6af09c75390b90afe60e37f09a68cc72037e25 WatchSource:0}: Error finding container d5b6bc03db00fb2fdb5f93cc5c6af09c75390b90afe60e37f09a68cc72037e25: Status 404 returned error can't find the container with id d5b6bc03db00fb2fdb5f93cc5c6af09c75390b90afe60e37f09a68cc72037e25
Jul 23 23:56:25.711830 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:25.711795    2548 generic.go:356] "Generic (PLEG): container finished" podID="7036daa8-2a72-4d88-89f1-b872ed4e7804" containerID="a6a7214383f7974d31d66a7d6027a1b813727ec32daf20a9e95f80b31b421216" exitCode=0
Jul 23 23:56:25.712240 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:25.711871    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7647-139623459657168" event={"ID":"7036daa8-2a72-4d88-89f1-b872ed4e7804","Type":"ContainerDied","Data":"a6a7214383f7974d31d66a7d6027a1b813727ec32daf20a9e95f80b31b421216"}
Jul 23 23:56:25.712240 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:25.711904    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7647-139623459657168" event={"ID":"7036daa8-2a72-4d88-89f1-b872ed4e7804","Type":"ContainerStarted","Data":"d5b6bc03db00fb2fdb5f93cc5c6af09c75390b90afe60e37f09a68cc72037e25"}
Jul 23 23:56:26.846938 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:26.846913    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7647-139623459657168"
Jul 23 23:56:26.865604 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:26.865578    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7647-139623459657168_7036daa8-2a72-4d88-89f1-b872ed4e7804/test-curl-7647-139623459657168/0.log"
Jul 23 23:56:26.887727 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:26.887694    2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7647-139623459657168"]
Jul 23 23:56:26.889846 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:26.889821    2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7647-139623459657168"]
Jul 23 23:56:26.946706 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:26.946678    2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lcbzn\" (UniqueName: \"kubernetes.io/projected/7036daa8-2a72-4d88-89f1-b872ed4e7804-kube-api-access-lcbzn\") pod \"7036daa8-2a72-4d88-89f1-b872ed4e7804\" (UID: \"7036daa8-2a72-4d88-89f1-b872ed4e7804\") "
Jul 23 23:56:26.948727 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:26.948691    2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7036daa8-2a72-4d88-89f1-b872ed4e7804-kube-api-access-lcbzn" (OuterVolumeSpecName: "kube-api-access-lcbzn") pod "7036daa8-2a72-4d88-89f1-b872ed4e7804" (UID: "7036daa8-2a72-4d88-89f1-b872ed4e7804"). InnerVolumeSpecName "kube-api-access-lcbzn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:56:27.047511 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:27.047420    2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lcbzn\" (UniqueName: \"kubernetes.io/projected/7036daa8-2a72-4d88-89f1-b872ed4e7804-kube-api-access-lcbzn\") on node \"ip-10-0-129-178.ec2.internal\" DevicePath \"\""
Jul 23 23:56:27.723516 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:27.723490    2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7647-139623459657168"
Jul 23 23:56:27.723729 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:27.723529    2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d5b6bc03db00fb2fdb5f93cc5c6af09c75390b90afe60e37f09a68cc72037e25"
Jul 23 23:56:27.736699 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:56:27.736665    2548 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7647-139623459657168\" is forbidden: User \"system:node:ip-10-0-129-178.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-178.ec2.internal' and this object" podUID="7036daa8-2a72-4d88-89f1-b872ed4e7804" pod="opendatahub/test-curl-7647-139623459657168"
Jul 23 23:56:28.746306 ip-10-0-129-178 kubenswrapper[2548]: E0723 23:56:28.746255    2548 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7647-139623459657168\" is forbidden: User \"system:node:ip-10-0-129-178.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-178.ec2.internal' and this object" podUID="7036daa8-2a72-4d88-89f1-b872ed4e7804" pod="opendatahub/test-curl-7647-139623459657168"
Jul 23 23:56:28.747446 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:28.747425    2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7036daa8-2a72-4d88-89f1-b872ed4e7804" path="/var/lib/kubelet/pods/7036daa8-2a72-4d88-89f1-b872ed4e7804/volumes"
Jul 23 23:56:40.885984 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:56:40.885893    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:57:02.388925 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:02.388887    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bshbw"]
Jul 23 23:57:11.256729 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:11.256693    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-86db8d67c8-4997k_88a33ff0-1277-4284-9fe7-351f45c14df9/authorino/0.log"
Jul 23 23:57:16.935907 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:16.935867    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-sxwqj_2e7855e5-8651-4934-b6e4-43e4d8511ad3/manager/0.log"
Jul 23 23:57:17.414681 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:17.414653    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5fd4b8775f-hzccw_c3744dc4-6e0c-4dfc-893c-1947479a7ff7/copy-manifests/0.log"
Jul 23 23:57:17.433138 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:17.433112    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5fd4b8775f-hzccw_c3744dc4-6e0c-4dfc-893c-1947479a7ff7/manager/0.log"
Jul 23 23:57:19.206492 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:19.206418    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s5vzb_96ec0397-59ad-4135-a7b3-cfb4c116f779/util/0.log"
Jul 23 23:57:19.213324 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:19.213294    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s5vzb_96ec0397-59ad-4135-a7b3-cfb4c116f779/pull/0.log"
Jul 23 23:57:19.219457 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:19.219440    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s5vzb_96ec0397-59ad-4135-a7b3-cfb4c116f779/extract/0.log"
Jul 23 23:57:19.338679 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:19.338651    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pvffr_81d19c8e-0a24-4bdd-8d6f-4b83d1d54968/util/0.log"
Jul 23 23:57:19.345783 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:19.345753    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pvffr_81d19c8e-0a24-4bdd-8d6f-4b83d1d54968/pull/0.log"
Jul 23 23:57:19.352830 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:19.352810    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pvffr_81d19c8e-0a24-4bdd-8d6f-4b83d1d54968/extract/0.log"
Jul 23 23:57:19.463503 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:19.463419    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nzckn_dde480a2-3c65-4992-9070-febf7b7477a8/util/0.log"
Jul 23 23:57:19.469590 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:19.469566    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nzckn_dde480a2-3c65-4992-9070-febf7b7477a8/pull/0.log"
Jul 23 23:57:19.475622 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:19.475602    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nzckn_dde480a2-3c65-4992-9070-febf7b7477a8/extract/0.log"
Jul 23 23:57:19.604061 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:19.604029    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fhwbc_dcd3d034-aa51-460a-8274-7ca3ef0e7fd7/util/0.log"
Jul 23 23:57:19.612719 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:19.612694    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fhwbc_dcd3d034-aa51-460a-8274-7ca3ef0e7fd7/pull/0.log"
Jul 23 23:57:19.618783 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:19.618761    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fhwbc_dcd3d034-aa51-460a-8274-7ca3ef0e7fd7/extract/0.log"
Jul 23 23:57:19.744024 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:19.743929    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-86db8d67c8-4997k_88a33ff0-1277-4284-9fe7-351f45c14df9/authorino/0.log"
Jul 23 23:57:20.092443 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:20.092364    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-xpzb9_9a4cd733-8c3d-4d3f-b8d3-ec25cd8ce0e2/kuadrant-console-plugin/0.log"
Jul 23 23:57:20.473904 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:20.473876    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-bshbw_94abf77d-d985-4e91-aba8-937179a84c97/limitador/0.log"
Jul 23 23:57:21.212137 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:21.212108    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8b54995f6-sz5t4_d186cae4-08f8-465f-add7-507deb5699a2/kube-auth-proxy/0.log"
Jul 23 23:57:22.778176 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:22.778147    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-tg6lt_f534a349-dcfc-48c7-9c43-9f9732cfde69/main/0.log"
Jul 23 23:57:28.737984 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:28.737932    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gm46q/must-gather-rsgps"]
Jul 23 23:57:28.738459 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:28.738295    2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7036daa8-2a72-4d88-89f1-b872ed4e7804" containerName="test-curl-7647-139623459657168"
Jul 23 23:57:28.738459 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:28.738307    2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="7036daa8-2a72-4d88-89f1-b872ed4e7804" containerName="test-curl-7647-139623459657168"
Jul 23 23:57:28.738459 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:28.738373    2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="7036daa8-2a72-4d88-89f1-b872ed4e7804" containerName="test-curl-7647-139623459657168"
Jul 23 23:57:28.741579 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:28.741547    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gm46q/must-gather-rsgps"
Jul 23 23:57:28.747737 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:28.747708    2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gm46q\"/\"default-dockercfg-m5j7c\""
Jul 23 23:57:28.747884 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:28.747746    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gm46q\"/\"kube-root-ca.crt\""
Jul 23 23:57:28.747884 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:28.747709    2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gm46q\"/\"openshift-service-ca.crt\""
Jul 23 23:57:28.749599 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:28.749576    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gm46q/must-gather-rsgps"]
Jul 23 23:57:28.891821 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:28.891787    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/feee1aec-c20f-476f-8395-0a76325b138a-must-gather-output\") pod \"must-gather-rsgps\" (UID: \"feee1aec-c20f-476f-8395-0a76325b138a\") " pod="openshift-must-gather-gm46q/must-gather-rsgps"
Jul 23 23:57:28.892025 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:28.891828    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtdf6\" (UniqueName: \"kubernetes.io/projected/feee1aec-c20f-476f-8395-0a76325b138a-kube-api-access-vtdf6\") pod \"must-gather-rsgps\" (UID: \"feee1aec-c20f-476f-8395-0a76325b138a\") " pod="openshift-must-gather-gm46q/must-gather-rsgps"
Jul 23 23:57:28.993118 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:28.993025    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/feee1aec-c20f-476f-8395-0a76325b138a-must-gather-output\") pod \"must-gather-rsgps\" (UID: \"feee1aec-c20f-476f-8395-0a76325b138a\") " pod="openshift-must-gather-gm46q/must-gather-rsgps"
Jul 23 23:57:28.993118 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:28.993075    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtdf6\" (UniqueName: \"kubernetes.io/projected/feee1aec-c20f-476f-8395-0a76325b138a-kube-api-access-vtdf6\") pod \"must-gather-rsgps\" (UID: \"feee1aec-c20f-476f-8395-0a76325b138a\") " pod="openshift-must-gather-gm46q/must-gather-rsgps"
Jul 23 23:57:28.993461 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:28.993437    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/feee1aec-c20f-476f-8395-0a76325b138a-must-gather-output\") pod \"must-gather-rsgps\" (UID: \"feee1aec-c20f-476f-8395-0a76325b138a\") " pod="openshift-must-gather-gm46q/must-gather-rsgps"
Jul 23 23:57:29.001254 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:29.001231    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtdf6\" (UniqueName: \"kubernetes.io/projected/feee1aec-c20f-476f-8395-0a76325b138a-kube-api-access-vtdf6\") pod \"must-gather-rsgps\" (UID: \"feee1aec-c20f-476f-8395-0a76325b138a\") " pod="openshift-must-gather-gm46q/must-gather-rsgps"
Jul 23 23:57:29.055860 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:29.055828    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gm46q/must-gather-rsgps"
Jul 23 23:57:29.183478 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:29.183454    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gm46q/must-gather-rsgps"]
Jul 23 23:57:29.185043 ip-10-0-129-178 kubenswrapper[2548]: W0723 23:57:29.185016    2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfeee1aec_c20f_476f_8395_0a76325b138a.slice/crio-eba659795f09b53e1ffb4f733f6c1e3a24b2baecbdd2776023bbfb0bbc98988c WatchSource:0}: Error finding container eba659795f09b53e1ffb4f733f6c1e3a24b2baecbdd2776023bbfb0bbc98988c: Status 404 returned error can't find the container with id eba659795f09b53e1ffb4f733f6c1e3a24b2baecbdd2776023bbfb0bbc98988c
Jul 23 23:57:30.041542 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:30.041501    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gm46q/must-gather-rsgps" event={"ID":"feee1aec-c20f-476f-8395-0a76325b138a","Type":"ContainerStarted","Data":"77e23123a6c536874f3baf1734bc7e9c0ed30f7ca658e2e83edd3220c05ac43c"}
Jul 23 23:57:30.042066 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:30.041559    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gm46q/must-gather-rsgps" event={"ID":"feee1aec-c20f-476f-8395-0a76325b138a","Type":"ContainerStarted","Data":"eba659795f09b53e1ffb4f733f6c1e3a24b2baecbdd2776023bbfb0bbc98988c"}
Jul 23 23:57:31.049151 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:31.049107    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gm46q/must-gather-rsgps" event={"ID":"feee1aec-c20f-476f-8395-0a76325b138a","Type":"ContainerStarted","Data":"3e03f46324750e8243a48c4d72b2306a965f7a20f833c13a5eee2be38e62c1c2"}
Jul 23 23:57:31.064432 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:31.064380    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gm46q/must-gather-rsgps" podStartSLOduration=2.38262177 podStartE2EDuration="3.064364319s" totalImagesPullingTime="681.742549ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:57:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:57:31.06289564 +0000 UTC m=+2652.806624688" watchObservedRunningTime="2026-07-23 23:57:31.064364319 +0000 UTC m=+2652.808093398"
Jul 23 23:57:31.425059 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:31.425026    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-95x9c_f98e0ab3-2bd3-44a8-9132-6f3521075ba5/global-pull-secret-syncer/0.log"
Jul 23 23:57:31.621648 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:31.621607    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-b6n2c_54cba885-3128-45cb-a350-f05f7dc12dbe/konnectivity-agent/0.log"
Jul 23 23:57:31.665568 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:31.665527    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-178.ec2.internal_9ba0f66b58401a0fb93564ba1a377674/haproxy/0.log"
Jul 23 23:57:36.355666 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:36.355613    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s5vzb_96ec0397-59ad-4135-a7b3-cfb4c116f779/extract/0.log"
Jul 23 23:57:36.380172 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:36.380104    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s5vzb_96ec0397-59ad-4135-a7b3-cfb4c116f779/util/0.log"
Jul 23 23:57:36.407764 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:36.407718    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s5vzb_96ec0397-59ad-4135-a7b3-cfb4c116f779/pull/0.log"
Jul 23 23:57:36.435943 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:36.435896    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pvffr_81d19c8e-0a24-4bdd-8d6f-4b83d1d54968/extract/0.log"
Jul 23 23:57:36.463577 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:36.463544    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pvffr_81d19c8e-0a24-4bdd-8d6f-4b83d1d54968/util/0.log"
Jul 23 23:57:36.488020 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:36.487992    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pvffr_81d19c8e-0a24-4bdd-8d6f-4b83d1d54968/pull/0.log"
Jul 23 23:57:36.520318 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:36.520282    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nzckn_dde480a2-3c65-4992-9070-febf7b7477a8/extract/0.log"
Jul 23 23:57:36.543578 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:36.543549    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nzckn_dde480a2-3c65-4992-9070-febf7b7477a8/util/0.log"
Jul 23 23:57:36.568003 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:36.567942    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nzckn_dde480a2-3c65-4992-9070-febf7b7477a8/pull/0.log"
Jul 23 23:57:36.596231 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:36.596202    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fhwbc_dcd3d034-aa51-460a-8274-7ca3ef0e7fd7/extract/0.log"
Jul 23 23:57:36.619476 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:36.619402    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fhwbc_dcd3d034-aa51-460a-8274-7ca3ef0e7fd7/util/0.log"
Jul 23 23:57:36.644775 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:36.644735    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fhwbc_dcd3d034-aa51-460a-8274-7ca3ef0e7fd7/pull/0.log"
Jul 23 23:57:37.351948 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:37.351909    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-86db8d67c8-4997k_88a33ff0-1277-4284-9fe7-351f45c14df9/authorino/0.log"
Jul 23 23:57:37.428232 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:37.428125    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-xpzb9_9a4cd733-8c3d-4d3f-b8d3-ec25cd8ce0e2/kuadrant-console-plugin/0.log"
Jul 23 23:57:37.611194 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:37.611117    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-bshbw_94abf77d-d985-4e91-aba8-937179a84c97/limitador/0.log"
Jul 23 23:57:39.283510 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:39.283477    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_790669a6-8ee8-4a03-9a9c-a701fccb615f/alertmanager/0.log"
Jul 23 23:57:39.311838 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:39.311788    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_790669a6-8ee8-4a03-9a9c-a701fccb615f/config-reloader/0.log"
Jul 23 23:57:39.337106 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:39.337072    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_790669a6-8ee8-4a03-9a9c-a701fccb615f/kube-rbac-proxy-web/0.log"
Jul 23 23:57:39.360367 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:39.360335    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_790669a6-8ee8-4a03-9a9c-a701fccb615f/kube-rbac-proxy/0.log"
Jul 23 23:57:39.387722 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:39.387694    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_790669a6-8ee8-4a03-9a9c-a701fccb615f/kube-rbac-proxy-metric/0.log"
Jul 23 23:57:39.413936 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:39.413903    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_790669a6-8ee8-4a03-9a9c-a701fccb615f/prom-label-proxy/0.log"
Jul 23 23:57:39.440059 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:39.439966    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_790669a6-8ee8-4a03-9a9c-a701fccb615f/init-config-reloader/0.log"
Jul 23 23:57:39.589275 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:39.589200    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-mpdl4_59f469d8-44e2-4ad9-91d9-a3355df31183/monitoring-plugin/0.log"
Jul 23 23:57:39.687152 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:39.687122    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mbgws_a83f48b0-a080-4412-b457-4d06b0e963e8/node-exporter/0.log"
Jul 23 23:57:39.709831 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:39.709802    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mbgws_a83f48b0-a080-4412-b457-4d06b0e963e8/kube-rbac-proxy/0.log"
Jul 23 23:57:39.739148 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:39.739119    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mbgws_a83f48b0-a080-4412-b457-4d06b0e963e8/init-textfile/0.log"
Jul 23 23:57:40.197479 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:40.197450    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-54cdb95f56-7gnrn_ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1/thanos-query/0.log"
Jul 23 23:57:40.218138 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:40.218107    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-54cdb95f56-7gnrn_ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1/kube-rbac-proxy-web/0.log"
Jul 23 23:57:40.239198 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:40.239166    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-54cdb95f56-7gnrn_ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1/kube-rbac-proxy/0.log"
Jul 23 23:57:40.259456 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:40.259431    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-54cdb95f56-7gnrn_ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1/prom-label-proxy/0.log"
Jul 23 23:57:40.288263 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:40.288233    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-54cdb95f56-7gnrn_ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1/kube-rbac-proxy-rules/0.log"
Jul 23 23:57:40.315781 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:40.315739    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-54cdb95f56-7gnrn_ea87dce4-8d4f-4378-8f3c-eb7bba3b21f1/kube-rbac-proxy-metrics/0.log"
Jul 23 23:57:41.793900 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:41.793864    2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gm46q/perf-node-gather-daemonset-8c7dz"]
Jul 23 23:57:41.799203 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:41.799174    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-8c7dz"
Jul 23 23:57:41.807568 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:41.806100    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gm46q/perf-node-gather-daemonset-8c7dz"]
Jul 23 23:57:41.930480 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:41.930438    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2213d286-2061-46a1-9052-b99c1c14e2e7-proc\") pod \"perf-node-gather-daemonset-8c7dz\" (UID: \"2213d286-2061-46a1-9052-b99c1c14e2e7\") " pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-8c7dz"
Jul 23 23:57:41.930755 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:41.930725    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2213d286-2061-46a1-9052-b99c1c14e2e7-lib-modules\") pod \"perf-node-gather-daemonset-8c7dz\" (UID: \"2213d286-2061-46a1-9052-b99c1c14e2e7\") " pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-8c7dz"
Jul 23 23:57:41.930889 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:41.930813    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wd89\" (UniqueName: \"kubernetes.io/projected/2213d286-2061-46a1-9052-b99c1c14e2e7-kube-api-access-6wd89\") pod \"perf-node-gather-daemonset-8c7dz\" (UID: \"2213d286-2061-46a1-9052-b99c1c14e2e7\") " pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-8c7dz"
Jul 23 23:57:41.930889 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:41.930857    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2213d286-2061-46a1-9052-b99c1c14e2e7-sys\") pod \"perf-node-gather-daemonset-8c7dz\" (UID: \"2213d286-2061-46a1-9052-b99c1c14e2e7\") " pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-8c7dz"
Jul 23 23:57:41.931034 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:41.930890    2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2213d286-2061-46a1-9052-b99c1c14e2e7-podres\") pod \"perf-node-gather-daemonset-8c7dz\" (UID: \"2213d286-2061-46a1-9052-b99c1c14e2e7\") " pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-8c7dz"
Jul 23 23:57:42.032317 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:42.032283    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2213d286-2061-46a1-9052-b99c1c14e2e7-proc\") pod \"perf-node-gather-daemonset-8c7dz\" (UID: \"2213d286-2061-46a1-9052-b99c1c14e2e7\") " pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-8c7dz"
Jul 23 23:57:42.032317 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:42.032323    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2213d286-2061-46a1-9052-b99c1c14e2e7-lib-modules\") pod \"perf-node-gather-daemonset-8c7dz\" (UID: \"2213d286-2061-46a1-9052-b99c1c14e2e7\") " pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-8c7dz"
Jul 23 23:57:42.032583 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:42.032380    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wd89\" (UniqueName: \"kubernetes.io/projected/2213d286-2061-46a1-9052-b99c1c14e2e7-kube-api-access-6wd89\") pod \"perf-node-gather-daemonset-8c7dz\" (UID: \"2213d286-2061-46a1-9052-b99c1c14e2e7\") " pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-8c7dz"
Jul 23 23:57:42.032583 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:42.032428    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2213d286-2061-46a1-9052-b99c1c14e2e7-proc\") pod \"perf-node-gather-daemonset-8c7dz\" (UID: \"2213d286-2061-46a1-9052-b99c1c14e2e7\") " pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-8c7dz"
Jul 23 23:57:42.032583 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:42.032415    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2213d286-2061-46a1-9052-b99c1c14e2e7-sys\") pod \"perf-node-gather-daemonset-8c7dz\" (UID: \"2213d286-2061-46a1-9052-b99c1c14e2e7\") " pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-8c7dz"
Jul 23 23:57:42.032583 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:42.032485    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2213d286-2061-46a1-9052-b99c1c14e2e7-sys\") pod \"perf-node-gather-daemonset-8c7dz\" (UID: \"2213d286-2061-46a1-9052-b99c1c14e2e7\") " pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-8c7dz"
Jul 23 23:57:42.032583 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:42.032500    2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2213d286-2061-46a1-9052-b99c1c14e2e7-podres\") pod \"perf-node-gather-daemonset-8c7dz\" (UID: \"2213d286-2061-46a1-9052-b99c1c14e2e7\") " pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-8c7dz"
Jul 23 23:57:42.032583 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:42.032524    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2213d286-2061-46a1-9052-b99c1c14e2e7-lib-modules\") pod \"perf-node-gather-daemonset-8c7dz\" (UID: \"2213d286-2061-46a1-9052-b99c1c14e2e7\") " pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-8c7dz"
Jul 23 23:57:42.032826 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:42.032589    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2213d286-2061-46a1-9052-b99c1c14e2e7-podres\") pod \"perf-node-gather-daemonset-8c7dz\" (UID: \"2213d286-2061-46a1-9052-b99c1c14e2e7\") " pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-8c7dz"
Jul 23 23:57:42.041147 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:42.041108    2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wd89\" (UniqueName: \"kubernetes.io/projected/2213d286-2061-46a1-9052-b99c1c14e2e7-kube-api-access-6wd89\") pod \"perf-node-gather-daemonset-8c7dz\" (UID: \"2213d286-2061-46a1-9052-b99c1c14e2e7\") " pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-8c7dz"
Jul 23 23:57:42.112436 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:42.112349    2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-8c7dz"
Jul 23 23:57:42.302279 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:42.301931    2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gm46q/perf-node-gather-daemonset-8c7dz"]
Jul 23 23:57:42.631417 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:42.631339    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6b65d587b7-z6wc7_1e2072ae-1c14-4027-bfda-59fab38a2621/console/0.log"
Jul 23 23:57:43.153040 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:43.152993    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-8c7dz" event={"ID":"2213d286-2061-46a1-9052-b99c1c14e2e7","Type":"ContainerStarted","Data":"6dd3eaa8046aee362246fd6b34e1b7955b5e6533863a1fcb93eb837750eac71a"}
Jul 23 23:57:43.153632 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:43.153081    2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-8c7dz" event={"ID":"2213d286-2061-46a1-9052-b99c1c14e2e7","Type":"ContainerStarted","Data":"46ee857ae4bf2c7a03b3715c356070b267f2a5c71d318c43603f524e78b44c17"}
Jul 23 23:57:43.153632 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:43.153415    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-8c7dz"
Jul 23 23:57:43.168896 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:43.168846    2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-8c7dz" podStartSLOduration=2.168829686 podStartE2EDuration="2.168829686s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 23:57:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:57:43.168337141 +0000 UTC m=+2664.912066191" watchObservedRunningTime="2026-07-23 23:57:43.168829686 +0000 UTC m=+2664.912558734"
Jul 23 23:57:44.282515 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:44.282488    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-96p7w_ee636a48-3311-4ae5-a214-5c69fb0f4fc0/dns/0.log"
Jul 23 23:57:44.303830 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:44.303803    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-96p7w_ee636a48-3311-4ae5-a214-5c69fb0f4fc0/kube-rbac-proxy/0.log"
Jul 23 23:57:44.369260 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:44.369230    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mth7q_56add3c1-56d8-46a3-aaf7-88b91aad4640/dns-node-resolver/0.log"
Jul 23 23:57:44.917253 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:44.917191    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6ccb666b9d-s5gmb_9d88d4a3-c264-4f45-844a-6a2d39e1a1aa/registry/0.log"
Jul 23 23:57:44.983477 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:44.983442    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rdphw_ba2ce11c-4a0d-4a34-b106-c34b61838088/node-ca/0.log"
Jul 23 23:57:46.205920 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:46.205886    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8b54995f6-sz5t4_d186cae4-08f8-465f-add7-507deb5699a2/kube-auth-proxy/0.log"
Jul 23 23:57:47.102956 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:47.102925    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tqqp6_c2143110-0125-4cee-ac64-ac116965e143/serve-healthcheck-canary/0.log"
Jul 23 23:57:47.731334 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:47.731302    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-z8bsv_4bdbd7d3-94a5-4f5a-8bec-5da77e14deab/kube-rbac-proxy/0.log"
Jul 23 23:57:47.750730 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:47.750702    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-z8bsv_4bdbd7d3-94a5-4f5a-8bec-5da77e14deab/exporter/0.log"
Jul 23 23:57:47.773568 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:47.773539    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-z8bsv_4bdbd7d3-94a5-4f5a-8bec-5da77e14deab/extractor/0.log"
Jul 23 23:57:49.169285 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:49.169260    2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-8c7dz"
Jul 23 23:57:50.147865 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:50.147832    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-sxwqj_2e7855e5-8651-4934-b6e4-43e4d8511ad3/manager/0.log"
Jul 23 23:57:50.463838 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:50.463758    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5fd4b8775f-hzccw_c3744dc4-6e0c-4dfc-893c-1947479a7ff7/manager/0.log"
Jul 23 23:57:50.482064 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:50.482032    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5fd4b8775f-hzccw_c3744dc4-6e0c-4dfc-893c-1947479a7ff7/copy-manifests/0.log"
Jul 23 23:57:52.000255 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:52.000178    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-68dbcfb44d-d6rz4_ea8d4fb8-6b77-4e8b-9745-7c8bd9a4027f/manager/0.log"
Jul 23 23:57:52.266525 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:52.266427    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8b54995f6-sz5t4_d186cae4-08f8-465f-add7-507deb5699a2/kube-auth-proxy/0.log"
Jul 23 23:57:53.792911 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:53.792874    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8b54995f6-sz5t4_d186cae4-08f8-465f-add7-507deb5699a2/kube-auth-proxy/0.log"
Jul 23 23:57:58.348565 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:58.348538    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wk67r_6bdd18e7-c4ba-4506-8b81-0443611d06da/kube-multus-additional-cni-plugins/0.log"
Jul 23 23:57:58.370360 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:58.370336    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wk67r_6bdd18e7-c4ba-4506-8b81-0443611d06da/egress-router-binary-copy/0.log"
Jul 23 23:57:58.390405 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:58.390376    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wk67r_6bdd18e7-c4ba-4506-8b81-0443611d06da/cni-plugins/0.log"
Jul 23 23:57:58.410133 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:58.410083    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wk67r_6bdd18e7-c4ba-4506-8b81-0443611d06da/bond-cni-plugin/0.log"
Jul 23 23:57:58.429934 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:58.429907    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wk67r_6bdd18e7-c4ba-4506-8b81-0443611d06da/routeoverride-cni/0.log"
Jul 23 23:57:58.449884 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:58.449859    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wk67r_6bdd18e7-c4ba-4506-8b81-0443611d06da/whereabouts-cni-bincopy/0.log"
Jul 23 23:57:58.470221 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:58.470197    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wk67r_6bdd18e7-c4ba-4506-8b81-0443611d06da/whereabouts-cni/0.log"
Jul 23 23:57:58.513524 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:58.513486    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-bvgxv_7f715492-d857-4ac5-bb35-db1169ae239e/kube-multus/0.log"
Jul 23 23:57:58.614785 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:58.614704    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-fkdcf_5037079f-e554-410b-973a-adef25cd2493/network-metrics-daemon/0.log"
Jul 23 23:57:58.634170 ip-10-0-129-178 kubenswrapper[2548]: I0723 23:57:58.634140    2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-fkdcf_5037079f-e554-410b-973a-adef25cd2493/kube-rbac-proxy/0.log"