Jul 14 13:54:12.039935 ip-10-0-128-127 systemd[1]: Starting Kubernetes Kubelet...
Jul 14 13:54:12.408207 ip-10-0-128-127 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 14 13:54:12.408207 ip-10-0-128-127 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 14 13:54:12.408207 ip-10-0-128-127 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 14 13:54:12.408207 ip-10-0-128-127 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 14 13:54:12.408207 ip-10-0-128-127 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 14 13:54:12.410707 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.410577    2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 14 13:54:12.415386 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415360    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 13:54:12.415386 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415384    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 13:54:12.415386 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415389    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 13:54:12.415490 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415400    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 13:54:12.415490 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415404    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 13:54:12.415490 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415407    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 13:54:12.415490 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415410    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 13:54:12.415490 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415415    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 13:54:12.415490 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415418    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 13:54:12.415490 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415421    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 13:54:12.415490 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415424    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 13:54:12.415490 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415427    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 13:54:12.415490 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415430    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 13:54:12.415490 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415433    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 13:54:12.415490 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415436    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 13:54:12.415490 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415439    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 13:54:12.415490 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415442    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 13:54:12.415490 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415445    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 13:54:12.415490 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415448    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 13:54:12.415490 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415451    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 13:54:12.415490 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415455    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 13:54:12.415490 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415458    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 13:54:12.415490 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415460    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 13:54:12.416022 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415464    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 13:54:12.416022 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415466    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 13:54:12.416022 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415469    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 13:54:12.416022 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415472    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 13:54:12.416022 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415474    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 13:54:12.416022 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415478    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 13:54:12.416022 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415480    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 13:54:12.416022 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415483    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 13:54:12.416022 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415487    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 13:54:12.416022 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415490    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 13:54:12.416022 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415495    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 13:54:12.416022 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415500    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 13:54:12.416022 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415503    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 13:54:12.416022 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415507    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 13:54:12.416022 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415509    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 13:54:12.416022 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415512    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 13:54:12.416022 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415515    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 13:54:12.416022 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415518    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 13:54:12.416022 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415521    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 13:54:12.416553 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415524    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 13:54:12.416553 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415526    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 13:54:12.416553 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415529    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 13:54:12.416553 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415532    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 13:54:12.416553 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415534    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 13:54:12.416553 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415544    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 13:54:12.416553 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415548    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 13:54:12.416553 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415551    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 13:54:12.416553 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415553    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 13:54:12.416553 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415556    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 13:54:12.416553 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415558    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 13:54:12.416553 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415561    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 13:54:12.416553 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415564    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 13:54:12.416553 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415567    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 13:54:12.416553 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415571    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 13:54:12.416553 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415574    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 13:54:12.416553 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415577    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 13:54:12.416553 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415580    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 13:54:12.416553 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415583    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 13:54:12.416553 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415586    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 13:54:12.417102 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415589    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 13:54:12.417102 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415592    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 13:54:12.417102 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415596    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 13:54:12.417102 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415598    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 13:54:12.417102 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415602    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 13:54:12.417102 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415604    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 13:54:12.417102 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415607    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 13:54:12.417102 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415610    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 13:54:12.417102 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415613    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 13:54:12.417102 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415615    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 13:54:12.417102 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415619    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 13:54:12.417102 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415624    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 13:54:12.417102 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415628    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 13:54:12.417102 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415632    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 13:54:12.417102 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415635    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 13:54:12.417102 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415638    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 13:54:12.417102 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415641    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 13:54:12.417102 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415644    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 13:54:12.417102 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415647    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 13:54:12.417591 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415650    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 13:54:12.417591 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415653    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 13:54:12.417591 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415655    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 13:54:12.417591 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415658    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 13:54:12.417591 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415661    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 13:54:12.417591 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415676    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 13:54:12.417591 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415680    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 13:54:12.417591 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415686    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 13:54:12.417591 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415690    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 13:54:12.417591 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415693    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 13:54:12.417591 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415697    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 13:54:12.417591 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415700    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 13:54:12.417591 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415703    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 13:54:12.417591 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415706    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 13:54:12.417591 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415709    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 13:54:12.417591 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415712    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 13:54:12.417591 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415716    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 13:54:12.417591 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.415719    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 13:54:12.418080 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.417928    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 13:54:12.418080 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.417938    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 13:54:12.418080 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.417942    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 13:54:12.418080 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.417946    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 13:54:12.418080 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.417950    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 13:54:12.418080 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.417953    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 13:54:12.418080 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.417956    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 13:54:12.418080 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.417959    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 13:54:12.418080 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.417962    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 13:54:12.418080 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.417965    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 13:54:12.418080 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.417967    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 13:54:12.418080 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.417970    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 13:54:12.418080 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.417973    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 13:54:12.418080 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.417977    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 13:54:12.418080 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.417980    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 13:54:12.418080 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.417983    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 13:54:12.418080 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.417988    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 13:54:12.418080 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.417992    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 13:54:12.418080 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.417995    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 13:54:12.418080 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.417998    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 13:54:12.418616 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418002    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 13:54:12.418616 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418005    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 13:54:12.418616 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418008    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 13:54:12.418616 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418010    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 13:54:12.418616 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418013    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 13:54:12.418616 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418016    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 13:54:12.418616 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418019    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 13:54:12.418616 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418022    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 13:54:12.418616 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418025    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 13:54:12.418616 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418028    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 13:54:12.418616 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418031    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 13:54:12.418616 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418035    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 13:54:12.418616 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418037    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 13:54:12.418616 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418041    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 13:54:12.418616 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418044    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 13:54:12.418616 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418047    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 13:54:12.418616 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418049    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 13:54:12.418616 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418052    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 13:54:12.418616 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418055    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 13:54:12.418616 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418057    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 13:54:12.419209 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418061    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 13:54:12.419209 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418064    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 13:54:12.419209 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418067    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 13:54:12.419209 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418069    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 13:54:12.419209 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418073    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 13:54:12.419209 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418076    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 13:54:12.419209 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418079    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 13:54:12.419209 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418081    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 13:54:12.419209 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418084    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 13:54:12.419209 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418088    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 13:54:12.419209 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418091    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 13:54:12.419209 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418094    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 13:54:12.419209 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418097    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 13:54:12.419209 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418099    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 13:54:12.419209 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418102    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 13:54:12.419209 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418105    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 13:54:12.419209 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418109    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 13:54:12.419209 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418114    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 13:54:12.419209 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418117    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 13:54:12.419735 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418122    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 13:54:12.419735 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418126    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 13:54:12.419735 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418129    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 13:54:12.419735 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418132    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 13:54:12.419735 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418134    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 13:54:12.419735 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418138    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 13:54:12.419735 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418142    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 13:54:12.419735 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418146    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 13:54:12.419735 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418149    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 13:54:12.419735 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418152    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 13:54:12.419735 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418154    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 13:54:12.419735 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418157    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 13:54:12.419735 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418160    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 13:54:12.419735 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418163    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 13:54:12.419735 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418165    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 13:54:12.419735 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418168    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 13:54:12.419735 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418171    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 13:54:12.419735 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418174    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 13:54:12.419735 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418177    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 13:54:12.420233 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418180    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 13:54:12.420233 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418182    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 13:54:12.420233 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418185    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 13:54:12.420233 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418188    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 13:54:12.420233 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418190    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 13:54:12.420233 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418193    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 13:54:12.420233 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418195    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 13:54:12.420233 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418198    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 13:54:12.420233 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418201    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 13:54:12.420233 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418204    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 13:54:12.420233 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418206    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 13:54:12.420233 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418209    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 13:54:12.420233 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418212    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 13:54:12.420233 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418215    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 13:54:12.420233 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418218    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 13:54:12.420233 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418221    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 13:54:12.420233 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418224    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 13:54:12.420233 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418227    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 13:54:12.420233 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418230    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 13:54:12.420233 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418234    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 13:54:12.420766 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418236    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 13:54:12.420766 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418696    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 13:54:12.420766 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418702    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 13:54:12.420766 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418705    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 13:54:12.420766 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418708    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 13:54:12.420766 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418711    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 13:54:12.420766 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418714    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 13:54:12.420766 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418717    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 13:54:12.420766 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418719    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 13:54:12.420766 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418722    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 13:54:12.420766 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418725    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 13:54:12.420766 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418728    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 13:54:12.420766 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418731    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 13:54:12.420766 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418734    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 13:54:12.420766 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418737    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 13:54:12.420766 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418740    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 13:54:12.420766 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418743    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 13:54:12.420766 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418745    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 13:54:12.420766 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418748    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 13:54:12.420766 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418751    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 13:54:12.421289 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418754    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 13:54:12.421289 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418757    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 13:54:12.421289 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418760    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 13:54:12.421289 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418763    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 13:54:12.421289 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418766    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 13:54:12.421289 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418769    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 13:54:12.421289 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418772    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 13:54:12.421289 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418775    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 13:54:12.421289 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418778    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 13:54:12.421289 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418780    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 13:54:12.421289 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418783    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 13:54:12.421289 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418787    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 13:54:12.421289 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418790    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 13:54:12.421289 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418793    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 13:54:12.421289 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418798    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 13:54:12.421289 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418802    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 13:54:12.421289 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418805    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 13:54:12.421289 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418808    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 13:54:12.421289 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418811    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 13:54:12.421806 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418814    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 13:54:12.421806 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418817    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 13:54:12.421806 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418820    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 13:54:12.421806 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418823    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 13:54:12.421806 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418826    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 13:54:12.421806 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418828    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 13:54:12.421806 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418831    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 13:54:12.421806 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418834    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 13:54:12.421806 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418837    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 13:54:12.421806 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418839    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 13:54:12.421806 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418842    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 13:54:12.421806 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418845    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 13:54:12.421806 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418848    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 13:54:12.421806 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418851    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 13:54:12.421806 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418854    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 13:54:12.421806 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418856    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 13:54:12.421806 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418859    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 13:54:12.421806 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418862    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 13:54:12.421806 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418865    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 13:54:12.421806 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418867    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 13:54:12.422331 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418870    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 13:54:12.422331 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418873    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 13:54:12.422331 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418876    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 13:54:12.422331 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418878    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 13:54:12.422331 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418881    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 13:54:12.422331 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418885    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 13:54:12.422331 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418888    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 13:54:12.422331 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418891    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 13:54:12.422331 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418895    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 13:54:12.422331 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418899    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 13:54:12.422331 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418903    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 13:54:12.422331 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418906    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 13:54:12.422331 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418908    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 13:54:12.422331 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418911    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 13:54:12.422331 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418914    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 13:54:12.422331 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418916    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 13:54:12.422331 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418919    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 13:54:12.422331 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418922    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 13:54:12.422331 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418924    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 13:54:12.422331 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418927    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 13:54:12.422869 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418930    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 13:54:12.422869 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418932    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 13:54:12.422869 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418935    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 13:54:12.422869 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418938    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 13:54:12.422869 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418940    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 13:54:12.422869 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418943    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 13:54:12.422869 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418946    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 13:54:12.422869 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418949    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 13:54:12.422869 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418951    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 13:54:12.422869 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418954    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 13:54:12.422869 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418957    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 13:54:12.422869 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418960    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 13:54:12.422869 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418962    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 13:54:12.422869 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418966    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 13:54:12.422869 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418969    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 13:54:12.422869 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418974    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 13:54:12.422869 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418978    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 13:54:12.422869 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418981    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 13:54:12.422869 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418984    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 13:54:12.423366 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418988    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 13:54:12.423366 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.418990    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 13:54:12.423366 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419112    2576 flags.go:64] FLAG: --address="0.0.0.0"
Jul 14 13:54:12.423366 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419133    2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 14 13:54:12.423366 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419141    2576 flags.go:64] FLAG: --anonymous-auth="true"
Jul 14 13:54:12.423366 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419147    2576 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 14 13:54:12.423366 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419151    2576 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 14 13:54:12.423366 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419155    2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 14 13:54:12.423366 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419160    2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 14 13:54:12.423366 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419164    2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 14 13:54:12.423366 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419168    2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 14 13:54:12.423366 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419171    2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 14 13:54:12.423366 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419176    2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 14 13:54:12.423366 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419180    2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 14 13:54:12.423366 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419184    2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 14 13:54:12.423366 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419187    2576 flags.go:64] FLAG: --cgroup-root=""
Jul 14 13:54:12.423366 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419191    2576 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 14 13:54:12.423366 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419195    2576 flags.go:64] FLAG: --client-ca-file=""
Jul 14 13:54:12.423366 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419198    2576 flags.go:64] FLAG: --cloud-provider="external"
Jul 14 13:54:12.423366 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419201    2576 flags.go:64] FLAG: --cluster-dns="[]"
Jul 14 13:54:12.423366 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419206    2576 flags.go:64] FLAG: --cluster-domain=""
Jul 14 13:54:12.423366 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419209    2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 14 13:54:12.423366 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419212    2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 14 13:54:12.423366 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419216    2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 14 13:54:12.423996 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419220    2576 flags.go:64] FLAG: --container-log-max-files="5"
Jul 14 13:54:12.423996 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419224    2576 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 14 13:54:12.423996 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419228    2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 14 13:54:12.423996 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419231    2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 14 13:54:12.423996 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419235    2576 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 14 13:54:12.423996 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419239    2576 flags.go:64] FLAG: --contention-profiling="false"
Jul 14 13:54:12.423996 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419242    2576 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 14 13:54:12.423996 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419245    2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 14 13:54:12.423996 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419249    2576 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 14 13:54:12.423996 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419252    2576 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 14 13:54:12.423996 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419257    2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 14 13:54:12.423996 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419260    2576 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 14 13:54:12.423996 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419263    2576 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 14 13:54:12.423996 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419267    2576 flags.go:64] FLAG: --enable-load-reader="false"
Jul 14 13:54:12.423996 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419270    2576 flags.go:64] FLAG: --enable-server="true"
Jul 14 13:54:12.423996 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419274    2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 14 13:54:12.423996 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419284    2576 flags.go:64] FLAG: --event-burst="100"
Jul 14 13:54:12.423996 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419297    2576 flags.go:64] FLAG: --event-qps="50"
Jul 14 13:54:12.423996 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419300    2576 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 14 13:54:12.423996 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419304    2576 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 14 13:54:12.423996 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419307    2576 flags.go:64] FLAG: --eviction-hard=""
Jul 14 13:54:12.423996 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419312    2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 14 13:54:12.423996 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419315    2576 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 14 13:54:12.423996 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419319    2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 14 13:54:12.423996 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419323    2576 flags.go:64] FLAG: --eviction-soft=""
Jul 14 13:54:12.424678 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419326    2576 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 14 13:54:12.424678 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419330    2576 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 14 13:54:12.424678 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419333    2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 14 13:54:12.424678 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419336    2576 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 14 13:54:12.424678 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419340    2576 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 14 13:54:12.424678 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419344    2576 flags.go:64] FLAG: --fail-swap-on="true"
Jul 14 13:54:12.424678 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419347    2576 flags.go:64] FLAG: --feature-gates=""
Jul 14 13:54:12.424678 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419351    2576 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 14 13:54:12.424678 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419354    2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 14 13:54:12.424678 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419358    2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 14 13:54:12.424678 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419362    2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 14 13:54:12.424678 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419365    2576 flags.go:64] FLAG: --healthz-port="10248"
Jul 14 13:54:12.424678 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419368    2576 flags.go:64] FLAG: --help="false"
Jul 14 13:54:12.424678 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419372    2576 flags.go:64] FLAG: --hostname-override="ip-10-0-128-127.ec2.internal"
Jul 14 13:54:12.424678 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419375    2576 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 14 13:54:12.424678 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419379    2576 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 14 13:54:12.424678 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419383    2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 14 13:54:12.424678 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419386    2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 14 13:54:12.424678 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419390    2576 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 14 13:54:12.424678 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419393    2576 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 14 13:54:12.424678 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419397    2576 flags.go:64] FLAG: --image-service-endpoint=""
Jul 14 13:54:12.424678 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419400    2576 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 14 13:54:12.424678 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419404    2576 flags.go:64] FLAG: --kube-api-burst="100"
Jul 14 13:54:12.425341 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419407    2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 14 13:54:12.425341 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419412    2576 flags.go:64] FLAG: --kube-api-qps="50"
Jul 14 13:54:12.425341 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419415    2576 flags.go:64] FLAG: --kube-reserved=""
Jul 14 13:54:12.425341 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419418    2576 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 14 13:54:12.425341 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419422    2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 14 13:54:12.425341 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419425    2576 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 14 13:54:12.425341 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419429    2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 14 13:54:12.425341 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419432    2576 flags.go:64] FLAG: --lock-file=""
Jul 14 13:54:12.425341 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419438    2576 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 14 13:54:12.425341 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419441    2576 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 14 13:54:12.425341 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419445    2576 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 14 13:54:12.425341 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419451    2576 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 14 13:54:12.425341 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419455    2576 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 14 13:54:12.425341 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419458    2576 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 14 13:54:12.425341 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419461    2576 flags.go:64] FLAG: --logging-format="text"
Jul 14 13:54:12.425341 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419465    2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 14 13:54:12.425341 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419469    2576 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 14 13:54:12.425341 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419472    2576 flags.go:64] FLAG: --manifest-url=""
Jul 14 13:54:12.425341 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419475    2576 flags.go:64] FLAG: --manifest-url-header=""
Jul 14 13:54:12.425341 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419481    2576 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 14 13:54:12.425341 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419484    2576 flags.go:64] FLAG: --max-open-files="1000000"
Jul 14 13:54:12.425341 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419489    2576 flags.go:64] FLAG: --max-pods="110"
Jul 14 13:54:12.425341 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419492    2576 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 14 13:54:12.425341 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419496    2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 14 13:54:12.425341 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419499    2576 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 14 13:54:12.426091 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419503    2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 14 13:54:12.426091 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419508    2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 14 13:54:12.426091 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419512    2576 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 14 13:54:12.426091 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419515    2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 14 13:54:12.426091 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419524    2576 flags.go:64] FLAG: --node-status-max-images="50"
Jul 14 13:54:12.426091 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419527    2576 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 14 13:54:12.426091 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419531    2576 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 14 13:54:12.426091 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419535    2576 flags.go:64] FLAG: --pod-cidr=""
Jul 14 13:54:12.426091 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419538    2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 14 13:54:12.426091 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419545    2576 flags.go:64] FLAG: --pod-manifest-path=""
Jul 14 13:54:12.426091 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419548    2576 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 14 13:54:12.426091 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419552    2576 flags.go:64] FLAG: --pods-per-core="0"
Jul 14 13:54:12.426091 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419555    2576 flags.go:64] FLAG: --port="10250"
Jul 14 13:54:12.426091 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419559    2576 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 14 13:54:12.426091 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419565    2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-052eccee36bdd65e1"
Jul 14 13:54:12.426091 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419569    2576 flags.go:64] FLAG: --qos-reserved=""
Jul 14 13:54:12.426091 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419573    2576 flags.go:64] FLAG: --read-only-port="10255"
Jul 14 13:54:12.426091 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419576    2576 flags.go:64] FLAG: --register-node="true"
Jul 14 13:54:12.426091 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419580    2576 flags.go:64] FLAG: --register-with-taints=""
Jul 14 13:54:12.426091 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419584    2576 flags.go:64] FLAG: --registry-burst="10"
Jul 14 13:54:12.426091 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419587    2576 flags.go:64] FLAG: --registry-qps="5"
Jul 14 13:54:12.426091 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419590    2576 flags.go:64] FLAG: --reserved-cpus=""
Jul 14 13:54:12.426091 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419593    2576 flags.go:64] FLAG: --reserved-memory=""
Jul 14 13:54:12.426091 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419597    2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 14 13:54:12.426091 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419601    2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 14 13:54:12.426883 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419604    2576 flags.go:64] FLAG: --rotate-certificates="false"
Jul 14 13:54:12.426883 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419607    2576 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 14 13:54:12.426883 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419610    2576 flags.go:64] FLAG: --runonce="false"
Jul 14 13:54:12.426883 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419613    2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 14 13:54:12.426883 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419616    2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 14 13:54:12.426883 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419620    2576 flags.go:64] FLAG: --seccomp-default="false"
Jul 14 13:54:12.426883 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419623    2576 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 14 13:54:12.426883 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419626    2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 14 13:54:12.426883 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419630    2576 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 14 13:54:12.426883 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419634    2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 14 13:54:12.426883 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419638    2576 flags.go:64] FLAG: --storage-driver-password="root"
Jul 14 13:54:12.426883 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419641    2576 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 14 13:54:12.426883 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419644    2576 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 14 13:54:12.426883 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419647    2576 flags.go:64] FLAG: --storage-driver-user="root"
Jul 14 13:54:12.426883 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419650    2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 14 13:54:12.426883 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419654    2576 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 14 13:54:12.426883 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419657    2576 flags.go:64] FLAG: --system-cgroups=""
Jul 14 13:54:12.426883 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419660    2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 14 13:54:12.426883 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419680    2576 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 14 13:54:12.426883 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419684    2576 flags.go:64] FLAG: --tls-cert-file=""
Jul 14 13:54:12.426883 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419687    2576 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 14 13:54:12.426883 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419693    2576 flags.go:64] FLAG: --tls-min-version=""
Jul 14 13:54:12.426883 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419696    2576 flags.go:64] FLAG: --tls-private-key-file=""
Jul 14 13:54:12.426883 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419701    2576 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 14 13:54:12.426883 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419704    2576 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 14 13:54:12.428035 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419708    2576 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 14 13:54:12.428035 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419711    2576 flags.go:64] FLAG: --v="2"
Jul 14 13:54:12.428035 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419716    2576 flags.go:64] FLAG: --version="false"
Jul 14 13:54:12.428035 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419720    2576 flags.go:64] FLAG: --vmodule=""
Jul 14 13:54:12.428035 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419725    2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 14 13:54:12.428035 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.419729    2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 14 13:54:12.428035 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.419875    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 13:54:12.428035 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.419880    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 13:54:12.428035 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.419883    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 13:54:12.428035 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.419886    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 13:54:12.428035 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.419889    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 13:54:12.428035 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.419892    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 13:54:12.428035 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.419895    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 13:54:12.428035 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.419897    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 13:54:12.428035 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.419900    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 13:54:12.428035 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.419904    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 13:54:12.428035 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.419907    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 13:54:12.428035 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.419910    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 13:54:12.428035 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.419913    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 13:54:12.428035 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.419916    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 13:54:12.428035 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.419919    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 13:54:12.429016 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.419922    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 13:54:12.429016 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.419925    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 13:54:12.429016 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.419928    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 13:54:12.429016 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.419931    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 13:54:12.429016 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.419934    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 13:54:12.429016 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.419937    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 13:54:12.429016 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.419940    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 13:54:12.429016 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.419946    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 13:54:12.429016 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.419949    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 13:54:12.429016 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.419953    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 13:54:12.429016 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.419956    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 13:54:12.429016 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.419959    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 13:54:12.429016 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.419962    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 13:54:12.429016 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.419965    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 13:54:12.429016 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.419968    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 13:54:12.429016 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.419971    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 13:54:12.429016 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.419974    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 13:54:12.429016 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.419977    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 13:54:12.429016 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.419980    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 13:54:12.429867 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.419984    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 13:54:12.429867 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.419988    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 13:54:12.429867 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.419991    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 13:54:12.429867 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.419994    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 13:54:12.429867 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.419997    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 13:54:12.429867 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420000    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 13:54:12.429867 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420003    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 13:54:12.429867 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420005    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 13:54:12.429867 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420008    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 13:54:12.429867 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420011    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 13:54:12.429867 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420014    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 13:54:12.429867 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420017    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 13:54:12.429867 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420020    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 13:54:12.429867 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420022    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 13:54:12.429867 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420025    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 13:54:12.429867 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420028    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 13:54:12.429867 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420030    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 13:54:12.429867 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420034    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 13:54:12.429867 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420037    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 13:54:12.430860 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420040    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 13:54:12.430860 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420044    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 13:54:12.430860 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420047    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 13:54:12.430860 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420051    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 13:54:12.430860 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420054    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 13:54:12.430860 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420057    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 13:54:12.430860 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420059    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 13:54:12.430860 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420062    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 13:54:12.430860 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420065    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 13:54:12.430860 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420068    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 13:54:12.430860 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420071    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 13:54:12.430860 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420074    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 13:54:12.430860 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420077    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 13:54:12.430860 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420080    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 13:54:12.430860 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420083    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 13:54:12.430860 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420086    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 13:54:12.430860 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420088    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 13:54:12.430860 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420091    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 13:54:12.430860 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420094    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 13:54:12.430860 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420098    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 13:54:12.431410 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420103    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 13:54:12.431410 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420107    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 13:54:12.431410 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420110    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 13:54:12.431410 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420113    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 13:54:12.431410 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420117    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 13:54:12.431410 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420120    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 13:54:12.431410 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420123    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 13:54:12.431410 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420126    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 13:54:12.431410 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420129    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 13:54:12.431410 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420131    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 13:54:12.431410 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420134    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 13:54:12.431410 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420137    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 13:54:12.431410 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420139    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 13:54:12.431410 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420143    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 13:54:12.431410 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420146    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 13:54:12.431410 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420150    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 13:54:12.431410 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420153    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 13:54:12.431410 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420156    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 13:54:12.431410 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420158    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 13:54:12.431929 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420161    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 13:54:12.431929 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420164    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 13:54:12.431929 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420167    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 13:54:12.431929 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420170    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 13:54:12.431929 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420172    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 13:54:12.431929 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420175    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 13:54:12.431929 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.420178    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 13:54:12.431929 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.420776    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 14 13:54:12.431929 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.431197    2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 14 13:54:12.431929 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.431221    2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 14 13:54:12.431929 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.431244    2576 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 14 13:54:12.431929 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.431250    2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 14 13:54:12.431929 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431325    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 13:54:12.431929 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431332    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 13:54:12.431929 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431336    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 13:54:12.432335 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431340    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 13:54:12.432335 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431343    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 13:54:12.432335 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431346    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 13:54:12.432335 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431349    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 13:54:12.432335 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431351    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 13:54:12.432335 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431354    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 13:54:12.432335 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431357    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 13:54:12.432335 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431360    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 13:54:12.432335 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431363    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 13:54:12.432335 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431366    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 13:54:12.432335 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431370    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 13:54:12.432335 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431372    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 13:54:12.432335 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431376    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 13:54:12.432335 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431378    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 13:54:12.432335 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431381    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 13:54:12.432335 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431389    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 13:54:12.432335 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431392    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 13:54:12.432335 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431395    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 13:54:12.432335 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431399    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 13:54:12.432843 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431401    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 13:54:12.432843 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431404    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 13:54:12.432843 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431407    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 13:54:12.432843 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431410    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 13:54:12.432843 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431413    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 13:54:12.432843 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431416    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 13:54:12.432843 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431420    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 13:54:12.432843 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431424    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 13:54:12.432843 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431427    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 13:54:12.432843 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431430    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 13:54:12.432843 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431433    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 13:54:12.432843 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431436    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 13:54:12.432843 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431439    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 13:54:12.432843 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431442    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 13:54:12.432843 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431444    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 13:54:12.432843 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431447    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 13:54:12.432843 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431450    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 13:54:12.432843 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431453    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 13:54:12.432843 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431456    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 13:54:12.432843 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431458    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 13:54:12.433365 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431461    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 13:54:12.433365 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431464    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 13:54:12.433365 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431467    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 13:54:12.433365 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431470    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 13:54:12.433365 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431478    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 13:54:12.433365 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431482    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 13:54:12.433365 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431485    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 13:54:12.433365 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431488    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 13:54:12.433365 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431491    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 13:54:12.433365 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431494    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 13:54:12.433365 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431496    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 13:54:12.433365 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431499    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 13:54:12.433365 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431502    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 13:54:12.433365 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431505    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 13:54:12.433365 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431507    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 13:54:12.433365 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431511    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 13:54:12.433365 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431514    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 13:54:12.433365 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431517    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 13:54:12.433365 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431520    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 13:54:12.433365 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431524    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 13:54:12.433893 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431527    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 13:54:12.433893 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431530    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 13:54:12.433893 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431533    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 13:54:12.433893 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431536    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 13:54:12.433893 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431539    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 13:54:12.433893 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431542    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 13:54:12.433893 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431544    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 13:54:12.433893 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431547    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 13:54:12.433893 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431550    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 13:54:12.433893 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431553    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 13:54:12.433893 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431556    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 13:54:12.433893 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431558    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 13:54:12.433893 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431561    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 13:54:12.433893 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431564    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 13:54:12.433893 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431567    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 13:54:12.433893 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431571    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 13:54:12.433893 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431575    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 13:54:12.433893 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431583    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 13:54:12.433893 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431586    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 13:54:12.434395 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431589    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 13:54:12.434395 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431592    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 13:54:12.434395 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431595    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 13:54:12.434395 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431598    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 13:54:12.434395 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431601    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 13:54:12.434395 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431604    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 13:54:12.434395 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431607    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 13:54:12.434395 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431609    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 13:54:12.434395 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431612    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 13:54:12.434395 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431615    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 13:54:12.434395 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431618    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 13:54:12.434395 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431621    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 13:54:12.434395 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431625    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 13:54:12.434395 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431629    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 13:54:12.434395 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431633    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 13:54:12.434395 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431637    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 13:54:12.434395 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431640    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 13:54:12.434395 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431644    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 13:54:12.434885 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.431650    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 14 13:54:12.434885 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431803    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 13:54:12.434885 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431810    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 13:54:12.434885 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431813    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 13:54:12.434885 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431816    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 13:54:12.434885 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431820    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 13:54:12.434885 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431823    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 13:54:12.434885 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431826    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 13:54:12.434885 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431829    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 13:54:12.434885 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431832    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 13:54:12.434885 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431835    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 13:54:12.434885 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431838    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 13:54:12.434885 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431842    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 13:54:12.434885 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431850    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 13:54:12.434885 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431854    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 13:54:12.435280 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431856    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 13:54:12.435280 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431859    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 13:54:12.435280 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431862    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 13:54:12.435280 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431865    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 13:54:12.435280 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431868    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 13:54:12.435280 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431871    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 13:54:12.435280 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431874    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 13:54:12.435280 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431878    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 13:54:12.435280 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431882    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 13:54:12.435280 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431885    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 13:54:12.435280 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431888    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 13:54:12.435280 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431891    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 13:54:12.435280 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431894    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 13:54:12.435280 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431897    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 13:54:12.435280 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431900    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 13:54:12.435280 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431902    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 13:54:12.435280 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431905    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 13:54:12.435280 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431908    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 13:54:12.435280 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431912    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 13:54:12.435857 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431915    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 13:54:12.435857 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431919    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 13:54:12.435857 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431922    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 13:54:12.435857 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431925    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 13:54:12.435857 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431928    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 13:54:12.435857 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431931    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 13:54:12.435857 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431934    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 13:54:12.435857 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431937    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 13:54:12.435857 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431940    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 13:54:12.435857 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431943    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 13:54:12.435857 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431946    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 13:54:12.435857 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431948    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 13:54:12.435857 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431953    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 13:54:12.435857 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431956    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 13:54:12.435857 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431959    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 13:54:12.435857 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431962    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 13:54:12.435857 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431965    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 13:54:12.435857 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431967    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 13:54:12.435857 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431970    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 13:54:12.435857 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431973    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 13:54:12.436374 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431976    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 13:54:12.436374 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431978    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 13:54:12.436374 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431981    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 13:54:12.436374 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431984    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 13:54:12.436374 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431987    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 13:54:12.436374 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431990    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 13:54:12.436374 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431993    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 13:54:12.436374 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431996    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 13:54:12.436374 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.431998    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 13:54:12.436374 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.432001    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 13:54:12.436374 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.432004    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 13:54:12.436374 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.432007    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 13:54:12.436374 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.432010    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 13:54:12.436374 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.432012    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 13:54:12.436374 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.432015    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 13:54:12.436374 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.432018    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 13:54:12.436374 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.432021    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 13:54:12.436374 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.432023    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 13:54:12.436374 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.432026    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 13:54:12.436374 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.432028    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 13:54:12.436946 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.432031    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 13:54:12.436946 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.432034    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 13:54:12.436946 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.432037    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 13:54:12.436946 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.432039    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 13:54:12.436946 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.432042    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 13:54:12.436946 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.432045    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 13:54:12.436946 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.432048    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 13:54:12.436946 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.432051    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 13:54:12.436946 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.432054    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 13:54:12.436946 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.432057    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 13:54:12.436946 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.432060    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 13:54:12.436946 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.432062    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 13:54:12.436946 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.432065    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 13:54:12.436946 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.432068    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 13:54:12.436946 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.432070    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 13:54:12.436946 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.432073    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 13:54:12.436946 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.432077    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 13:54:12.436946 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.432080    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 13:54:12.436946 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.432084    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 13:54:12.437436 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.432087    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 13:54:12.437436 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.432090    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 13:54:12.437436 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.432093    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 13:54:12.437436 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.432096    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 13:54:12.437436 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.432100    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 13:54:12.437436 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.432103    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 13:54:12.437436 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:12.432106    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 13:54:12.437436 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.432111    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 14 13:54:12.437436 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.432294    2576 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 14 13:54:12.437436 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.436768    2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 14 13:54:12.438194 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.438181    2576 server.go:1019] "Starting client certificate rotation"
Jul 14 13:54:12.438311 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.438291    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 14 13:54:12.438354 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.438335    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 14 13:54:12.440451 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.440435    2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 14 13:54:12.446873 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.446854    2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 14 13:54:12.455767 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.455746    2576 log.go:25] "Validated CRI v1 runtime API"
Jul 14 13:54:12.458606 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.458585    2576 log.go:25] "Validated CRI v1 image API"
Jul 14 13:54:12.459617 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.459587    2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 14 13:54:12.463226 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.463202    2576 fs.go:135] Filesystem UUIDs: map[73e6a3dc-eb64-4656-9579-66bfc8ecd92c:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 eccd96c4-1431-4bab-9d28-638d99cef137:/dev/nvme0n1p3]
Jul 14 13:54:12.463292 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.463227    2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 14 13:54:12.466381 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.466357    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 14 13:54:12.469767 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.469616    2576 manager.go:216] Machine: {Timestamp:2026-07-14 13:54:12.469058782 +0000 UTC m=+0.319496279 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3105041 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec257ef6d9003dfdc166d7e6d1550650 SystemUUID:ec257ef6-d900-3dfd-c166-d7e6d1550650 BootID:6ab89c9c-6f10-4392-a1aa-9e1f40715299 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:14:dd:53:af:71 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:14:dd:53:af:71 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:22:2e:ed:8a:0c:b9 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 14 13:54:12.469767 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.469760    2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 14 13:54:12.469883 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.469858    2576 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 14 13:54:12.471445 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.471417    2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 14 13:54:12.471620 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.471446    2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-127.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 14 13:54:12.472231 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.472218    2576 topology_manager.go:138] "Creating topology manager with none policy"
Jul 14 13:54:12.472271 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.472234    2576 container_manager_linux.go:309] "Creating device plugin manager"
Jul 14 13:54:12.472271 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.472243    2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 14 13:54:12.473609 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.473588    2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 14 13:54:12.475274 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.475259    2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 14 13:54:12.478558 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.478541    2576 state_mem.go:36] "Initialized new in-memory state store"
Jul 14 13:54:12.478738 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.478727    2576 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 14 13:54:12.478772 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.478762    2576 kubelet.go:486] "Attempting to sync node with API server"
Jul 14 13:54:12.478801 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.478779    2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 14 13:54:12.478801 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.478792    2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 14 13:54:12.478871 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.478826    2576 kubelet.go:398] "Adding apiserver pod source"
Jul 14 13:54:12.478871 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.478840    2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 14 13:54:12.479362 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.479339    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 14 13:54:12.479484 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.479370    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 14 13:54:12.483412 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.483377    2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 14 13:54:12.485811 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.485792    2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 14 13:54:12.485897 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.485827    2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 14 13:54:12.486023 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.486012    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 14 13:54:12.486075 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.486028    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 14 13:54:12.486075 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.486035    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 14 13:54:12.486075 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.486041    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 14 13:54:12.486075 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.486050    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 14 13:54:12.486075 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.486055    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 14 13:54:12.486405 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.486395    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 14 13:54:12.486436 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.486405    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 14 13:54:12.486436 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.486413    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 14 13:54:12.486436 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.486419    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 14 13:54:12.486436 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.486428    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 14 13:54:12.486436 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.486437    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 14 13:54:12.487311 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.487300    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 14 13:54:12.487311 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.487311    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 14 13:54:12.490383 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.490355    2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-127.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 14 13:54:12.490494 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:12.490468    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 14 13:54:12.490751 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:12.490731    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-127.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 14 13:54:12.491273 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.491260    2576 server.go:1268] "Started kubelet"
Jul 14 13:54:12.491339 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.491305    2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 14 13:54:12.491591 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.491418    2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 14 13:54:12.491591 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.491550    2576 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 14 13:54:12.491775 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.491549    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qnmqp"
Jul 14 13:54:12.492423 ip-10-0-128-127 systemd[1]: Started Kubernetes Kubelet.
Jul 14 13:54:12.493739 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.493719    2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 14 13:54:12.494468 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.494453    2576 server.go:310] "Adding debug handlers to kubelet server"
Jul 14 13:54:12.497527 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:12.497500    2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 14 13:54:12.499707 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.499687    2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 14 13:54:12.499792 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.499685    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 14 13:54:12.499859 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:12.499818    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-127.ec2.internal\" not found"
Jul 14 13:54:12.500386 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.500262    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qnmqp"
Jul 14 13:54:12.500595 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.500574    2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 14 13:54:12.500714 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.500577    2576 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 14 13:54:12.500714 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.500633    2576 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 14 13:54:12.500912 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.500898    2576 reconstruct.go:97] "Volume reconstruction finished"
Jul 14 13:54:12.501031 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.501017    2576 reconciler.go:29] "Reconciler: start to sync state"
Jul 14 13:54:12.501087 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:12.499551    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-127.ec2.internal.18c22c6dcccdb9c4  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-127.ec2.internal,UID:ip-10-0-128-127.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-127.ec2.internal,},FirstTimestamp:2026-07-14 13:54:12.491229636 +0000 UTC m=+0.341667133,LastTimestamp:2026-07-14 13:54:12.491229636 +0000 UTC m=+0.341667133,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-127.ec2.internal,}"
Jul 14 13:54:12.501296 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.501278    2576 factory.go:55] Registering systemd factory
Jul 14 13:54:12.501344 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.501304    2576 factory.go:223] Registration of the systemd container factory successfully
Jul 14 13:54:12.501548 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.501519    2576 factory.go:153] Registering CRI-O factory
Jul 14 13:54:12.501548 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.501539    2576 factory.go:223] Registration of the crio container factory successfully
Jul 14 13:54:12.501708 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.501581    2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 14 13:54:12.501708 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.501600    2576 factory.go:103] Registering Raw factory
Jul 14 13:54:12.501708 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.501610    2576 manager.go:1196] Started watching for new ooms in manager
Jul 14 13:54:12.502138 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.502044    2576 manager.go:349] Starting recovery of all containers
Jul 14 13:54:12.505929 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:12.505888    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 14 13:54:12.506088 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:12.506060    2576 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-127.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 14 13:54:12.515414 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.515179    2576 manager.go:354] Recovery completed
Jul 14 13:54:12.520178 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.520159    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 13:54:12.522475 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.522456    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-127.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 13:54:12.522592 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.522493    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-127.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 13:54:12.522592 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.522509    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-127.ec2.internal" event="NodeHasSufficientPID"
Jul 14 13:54:12.523127 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.523107    2576 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 14 13:54:12.523127 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.523119    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 13:54:12.523127 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.523127    2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 14 13:54:12.523261 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.523148    2576 state_mem.go:36] "Initialized new in-memory state store"
Jul 14 13:54:12.523847 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.523831    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-127.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 13:54:12.523903 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.523862    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-127.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 13:54:12.523936 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.523909    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-127.ec2.internal" event="NodeHasSufficientPID"
Jul 14 13:54:12.524527 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.524515    2576 policy_none.go:49] "None policy: Start"
Jul 14 13:54:12.524562 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.524533    2576 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 14 13:54:12.524562 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.524556    2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 14 13:54:12.525953 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.525936    2576 policy_none.go:47] "Start"
Jul 14 13:54:12.562357 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.562336    2576 manager.go:337] "Starting Device Plugin manager"
Jul 14 13:54:12.562495 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:12.562382    2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 14 13:54:12.562495 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.562396    2576 server.go:87] "Starting device plugin registration server"
Jul 14 13:54:12.562732 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.562717    2576 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 14 13:54:12.562798 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.562734    2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 14 13:54:12.562858 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.562843    2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 14 13:54:12.563000 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.562986    2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 14 13:54:12.563000 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.563000    2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 14 13:54:12.563518 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:12.563495    2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 14 13:54:12.563591 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:12.563554    2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-127.ec2.internal\" not found"
Jul 14 13:54:12.665447 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.665406    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 13:54:12.667629 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.666800    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-127.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 13:54:12.667629 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.666829    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-127.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 13:54:12.667629 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.666839    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-127.ec2.internal" event="NodeHasSufficientPID"
Jul 14 13:54:12.667629 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.666868    2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-127.ec2.internal"
Jul 14 13:54:12.675391 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.675364    2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-127.ec2.internal"
Jul 14 13:54:12.675515 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:12.675403    2576 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-127.ec2.internal\": node \"ip-10-0-128-127.ec2.internal\" not found"
Jul 14 13:54:12.681411 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.681341    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 14 13:54:12.682773 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.682754    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 14 13:54:12.682851 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.682796    2576 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 14 13:54:12.682851 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.682825    2576 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 14 13:54:12.682969 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:12.682870    2576 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 14 13:54:12.684959 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.684939    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 13:54:12.689810 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:12.689787    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-127.ec2.internal\" not found"
Jul 14 13:54:12.783810 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.783732    2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-127.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-127.ec2.internal"]
Jul 14 13:54:12.784006 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.783852    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 13:54:12.785575 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.785552    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-127.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 13:54:12.785739 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.785591    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-127.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 13:54:12.785739 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.785606    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-127.ec2.internal" event="NodeHasSufficientPID"
Jul 14 13:54:12.787966 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.787947    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 13:54:12.788087 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.788058    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-127.ec2.internal"
Jul 14 13:54:12.788137 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.788094    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 13:54:12.788778 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.788762    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-127.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 13:54:12.788867 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.788787    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-127.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 13:54:12.788867 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.788797    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-127.ec2.internal" event="NodeHasSufficientPID"
Jul 14 13:54:12.788867 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.788764    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-127.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 13:54:12.788985 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.788871    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-127.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 13:54:12.788985 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.788884    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-127.ec2.internal" event="NodeHasSufficientPID"
Jul 14 13:54:12.790854 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:12.790833    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-127.ec2.internal\" not found"
Jul 14 13:54:12.791128 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.791112    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-127.ec2.internal"
Jul 14 13:54:12.791233 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.791141    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 13:54:12.791872 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.791854    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-127.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 13:54:12.791961 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.791880    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-127.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 13:54:12.791961 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.791890    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-127.ec2.internal" event="NodeHasSufficientPID"
Jul 14 13:54:12.802935 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.802904    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cbb95fe8aa83faa678c273911aee96f3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-127.ec2.internal\" (UID: \"cbb95fe8aa83faa678c273911aee96f3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-127.ec2.internal"
Jul 14 13:54:12.803090 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.802940    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cbb95fe8aa83faa678c273911aee96f3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-127.ec2.internal\" (UID: \"cbb95fe8aa83faa678c273911aee96f3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-127.ec2.internal"
Jul 14 13:54:12.803090 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.802958    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8c0fbe03dcb71398a547458685400965-config\") pod \"kube-apiserver-proxy-ip-10-0-128-127.ec2.internal\" (UID: \"8c0fbe03dcb71398a547458685400965\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-127.ec2.internal"
Jul 14 13:54:12.815758 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:12.815730    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-127.ec2.internal\" not found" node="ip-10-0-128-127.ec2.internal"
Jul 14 13:54:12.821088 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:12.821069    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-127.ec2.internal\" not found" node="ip-10-0-128-127.ec2.internal"
Jul 14 13:54:12.891307 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:12.891258    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-127.ec2.internal\" not found"
Jul 14 13:54:12.903634 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.903585    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cbb95fe8aa83faa678c273911aee96f3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-127.ec2.internal\" (UID: \"cbb95fe8aa83faa678c273911aee96f3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-127.ec2.internal"
Jul 14 13:54:12.903724 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.903639    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cbb95fe8aa83faa678c273911aee96f3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-127.ec2.internal\" (UID: \"cbb95fe8aa83faa678c273911aee96f3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-127.ec2.internal"
Jul 14 13:54:12.903724 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.903659    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8c0fbe03dcb71398a547458685400965-config\") pod \"kube-apiserver-proxy-ip-10-0-128-127.ec2.internal\" (UID: \"8c0fbe03dcb71398a547458685400965\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-127.ec2.internal"
Jul 14 13:54:12.903724 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.903687    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8c0fbe03dcb71398a547458685400965-config\") pod \"kube-apiserver-proxy-ip-10-0-128-127.ec2.internal\" (UID: \"8c0fbe03dcb71398a547458685400965\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-127.ec2.internal"
Jul 14 13:54:12.903724 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.903701    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cbb95fe8aa83faa678c273911aee96f3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-127.ec2.internal\" (UID: \"cbb95fe8aa83faa678c273911aee96f3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-127.ec2.internal"
Jul 14 13:54:12.903869 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:12.903701    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cbb95fe8aa83faa678c273911aee96f3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-127.ec2.internal\" (UID: \"cbb95fe8aa83faa678c273911aee96f3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-127.ec2.internal"
Jul 14 13:54:12.991725 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:12.991645    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-127.ec2.internal\" not found"
Jul 14 13:54:13.098283 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:13.098244    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-127.ec2.internal\" not found"
Jul 14 13:54:13.118288 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.118237    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-127.ec2.internal"
Jul 14 13:54:13.125952 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.125928    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-127.ec2.internal"
Jul 14 13:54:13.200521 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:13.200473    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-127.ec2.internal\" not found"
Jul 14 13:54:13.306738 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:13.306710    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-127.ec2.internal\" not found"
Jul 14 13:54:13.407564 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:13.407519    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-127.ec2.internal\" not found"
Jul 14 13:54:13.437275 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.437235    2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 14 13:54:13.437863 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.437414    2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 14 13:54:13.452285 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.452257    2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 13:54:13.481563 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.481526    2576 apiserver.go:52] "Watching apiserver"
Jul 14 13:54:13.491612 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.491584    2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 14 13:54:13.491998 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.491974    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-9pk9w","openshift-ovn-kubernetes/ovnkube-node-sh6rt","kube-system/global-pull-secret-syncer-5m6xn","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r5kpb","openshift-cluster-node-tuning-operator/tuned-tfd72","openshift-dns/node-resolver-jfpgf","openshift-image-registry/node-ca-5pjcr","openshift-multus/multus-additional-cni-plugins-qhw7d","openshift-multus/multus-jbr68","openshift-multus/network-metrics-daemon-486sw","kube-system/konnectivity-agent-65w26","openshift-network-diagnostics/network-check-target-22z2l"]
Jul 14 13:54:13.494578 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.494553    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9pk9w"
Jul 14 13:54:13.497897 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.497874    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-mcbsn\""
Jul 14 13:54:13.498058 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.498036    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.498058 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.498046    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 14 13:54:13.498189 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.498034    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 14 13:54:13.498247 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.498210    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 14 13:54:13.499982 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.499963    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 14 13:54:13.500098 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.499962    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-127.ec2.internal"
Jul 14 13:54:13.500413 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.500391    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 14 13:54:13.500508 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.500400    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dfmvs\""
Jul 14 13:54:13.500508 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.500473    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:54:13.500617 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:13.500540    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5m6xn" podUID="02b53875-f2af-42e5-a311-74b8301e8b62"
Jul 14 13:54:13.500845 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.500817    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 14 13:54:13.500946 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.500847    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 14 13:54:13.500946 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.500899    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 14 13:54:13.501271 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.501256    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 14 13:54:13.501373 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.501355    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 14 13:54:13.503016 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.502999    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r5kpb"
Jul 14 13:54:13.504890 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.504852    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-13 13:49:12 +0000 UTC" deadline="2027-12-22 16:53:28.459474515 +0000 UTC"
Jul 14 13:54:13.504890 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.504887    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12626h59m14.954589444s"
Jul 14 13:54:13.505128 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.505116    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 14 13:54:13.505253 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.505237    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 14 13:54:13.505253 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.505245    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 14 13:54:13.505404 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.505387    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2jzxx\""
Jul 14 13:54:13.505456 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.505394    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.507658 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.507638    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 14 13:54:13.507658 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.507653    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jfpgf"
Jul 14 13:54:13.507842 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.507695    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5600e3e1-e775-482e-bc72-76d732552705-ovnkube-config\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.507842 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.507706    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 14 13:54:13.507842 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.507729    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-etc-openvswitch\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.507842 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.507753    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-run-ovn\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.507842 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.507807    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-node-log\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.507842 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.507836    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5600e3e1-e775-482e-bc72-76d732552705-env-overrides\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.508142 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.507860    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/02b53875-f2af-42e5-a311-74b8301e8b62-dbus\") pod \"global-pull-secret-syncer-5m6xn\" (UID: \"02b53875-f2af-42e5-a311-74b8301e8b62\") " pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:54:13.508142 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.507883    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7a539262-9b04-4fbf-ae2b-f922f910e471-socket-dir\") pod \"aws-ebs-csi-driver-node-r5kpb\" (UID: \"7a539262-9b04-4fbf-ae2b-f922f910e471\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r5kpb"
Jul 14 13:54:13.508142 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.507908    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-run-openvswitch\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.508142 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.507935    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-systemd-units\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.508142 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.507955    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-host-run-netns\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.508142 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.507970    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lddt\" (UniqueName: \"kubernetes.io/projected/5600e3e1-e775-482e-bc72-76d732552705-kube-api-access-2lddt\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.508142 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.507988    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7a539262-9b04-4fbf-ae2b-f922f910e471-etc-selinux\") pod \"aws-ebs-csi-driver-node-r5kpb\" (UID: \"7a539262-9b04-4fbf-ae2b-f922f910e471\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r5kpb"
Jul 14 13:54:13.508142 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.508011    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-run-systemd\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.508142 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.508048    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-5gcwf\""
Jul 14 13:54:13.508142 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.508062    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a9b528fa-6f75-402e-8948-190262b8325a-host-slash\") pod \"iptables-alerter-9pk9w\" (UID: \"a9b528fa-6f75-402e-8948-190262b8325a\") " pod="openshift-network-operator/iptables-alerter-9pk9w"
Jul 14 13:54:13.508142 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.508087    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-host-kubelet\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.508142 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.508102    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-log-socket\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.508142 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.508123    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7a539262-9b04-4fbf-ae2b-f922f910e471-device-dir\") pod \"aws-ebs-csi-driver-node-r5kpb\" (UID: \"7a539262-9b04-4fbf-ae2b-f922f910e471\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r5kpb"
Jul 14 13:54:13.508142 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.508149    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7wkx\" (UniqueName: \"kubernetes.io/projected/7a539262-9b04-4fbf-ae2b-f922f910e471-kube-api-access-z7wkx\") pod \"aws-ebs-csi-driver-node-r5kpb\" (UID: \"7a539262-9b04-4fbf-ae2b-f922f910e471\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r5kpb"
Jul 14 13:54:13.508732 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.508171    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a9b528fa-6f75-402e-8948-190262b8325a-iptables-alerter-script\") pod \"iptables-alerter-9pk9w\" (UID: \"a9b528fa-6f75-402e-8948-190262b8325a\") " pod="openshift-network-operator/iptables-alerter-9pk9w"
Jul 14 13:54:13.508732 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.508190    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-host-run-ovn-kubernetes\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.508732 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.508206    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5600e3e1-e775-482e-bc72-76d732552705-ovn-node-metrics-cert\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.508732 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.508229    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/02b53875-f2af-42e5-a311-74b8301e8b62-kubelet-config\") pod \"global-pull-secret-syncer-5m6xn\" (UID: \"02b53875-f2af-42e5-a311-74b8301e8b62\") " pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:54:13.508732 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.508257    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7a539262-9b04-4fbf-ae2b-f922f910e471-sys-fs\") pod \"aws-ebs-csi-driver-node-r5kpb\" (UID: \"7a539262-9b04-4fbf-ae2b-f922f910e471\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r5kpb"
Jul 14 13:54:13.508732 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.508299    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-host-slash\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.508732 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.508331    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-host-cni-netd\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.508732 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.508361    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.508732 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.508379    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-host-cni-bin\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.508732 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.508397    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/02b53875-f2af-42e5-a311-74b8301e8b62-original-pull-secret\") pod \"global-pull-secret-syncer-5m6xn\" (UID: \"02b53875-f2af-42e5-a311-74b8301e8b62\") " pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:54:13.508732 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.508414    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7a539262-9b04-4fbf-ae2b-f922f910e471-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r5kpb\" (UID: \"7a539262-9b04-4fbf-ae2b-f922f910e471\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r5kpb"
Jul 14 13:54:13.508732 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.508431    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7a539262-9b04-4fbf-ae2b-f922f910e471-registration-dir\") pod \"aws-ebs-csi-driver-node-r5kpb\" (UID: \"7a539262-9b04-4fbf-ae2b-f922f910e471\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r5kpb"
Jul 14 13:54:13.508732 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.508448    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5600e3e1-e775-482e-bc72-76d732552705-ovnkube-script-lib\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.508732 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.508464    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-var-lib-openvswitch\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.508732 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.508480    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njsk9\" (UniqueName: \"kubernetes.io/projected/a9b528fa-6f75-402e-8948-190262b8325a-kube-api-access-njsk9\") pod \"iptables-alerter-9pk9w\" (UID: \"a9b528fa-6f75-402e-8948-190262b8325a\") " pod="openshift-network-operator/iptables-alerter-9pk9w"
Jul 14 13:54:13.509309 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.509051    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 14 13:54:13.509309 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.509112    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-127.ec2.internal"
Jul 14 13:54:13.509895 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.509879    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 14 13:54:13.510226 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.510171    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5pjcr"
Jul 14 13:54:13.510357 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.510340    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5q2nw\""
Jul 14 13:54:13.510413 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.510368    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 14 13:54:13.512237 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.512219    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-nsq8f\""
Jul 14 13:54:13.512559 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.512540    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 14 13:54:13.512693 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.512666    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 14 13:54:13.512693 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.512681    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qhw7d"
Jul 14 13:54:13.513011 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.512964    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 14 13:54:13.515201 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.515178    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 14 13:54:13.515365 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.515344    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 14 13:54:13.515365 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.515354    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 14 13:54:13.515535 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.515394    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 14 13:54:13.515535 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.515519    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 14 13:54:13.515661 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.515600    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kqkvd\""
Jul 14 13:54:13.515661 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.515640    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.515765 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.515737    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 14 13:54:13.517944 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.517919    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 14 13:54:13.517944 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.517937    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-6wc74\""
Jul 14 13:54:13.518117 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.518064    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:54:13.518165 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:13.518131    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-486sw" podUID="188bee4a-9fbe-4f95-8829-a91ab584ce4e"
Jul 14 13:54:13.519787 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.519756    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 14 13:54:13.520386 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.520370    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-65w26"
Jul 14 13:54:13.522720 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.522702    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 14 13:54:13.523249 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.523237    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 14 13:54:13.523400 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.523385    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-128-127.ec2.internal"]
Jul 14 13:54:13.523444 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.523428    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-127.ec2.internal"]
Jul 14 13:54:13.523514 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.523500    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-dw48j\""
Jul 14 13:54:13.523554 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.523532    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:54:13.523613 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:13.523591    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22z2l" podUID="421bfc78-19a7-4327-8fde-0b7a9927bb4e"
Jul 14 13:54:13.547313 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.547289    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-8khn2"
Jul 14 13:54:13.556343 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.556309    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-8khn2"
Jul 14 13:54:13.601661 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.601617    2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 14 13:54:13.608893 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.608860    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-system-cni-dir\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.609068 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.608932    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-multus-cni-dir\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.609068 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.608971    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-cni-binary-copy\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.609068 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.609003    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-etc-kubernetes\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.609068 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.609050    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c53cb4b0-7a90-4dba-b53d-df505d61bf19-system-cni-dir\") pod \"multus-additional-cni-plugins-qhw7d\" (UID: \"c53cb4b0-7a90-4dba-b53d-df505d61bf19\") " pod="openshift-multus/multus-additional-cni-plugins-qhw7d"
Jul 14 13:54:13.609305 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.609081    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c53cb4b0-7a90-4dba-b53d-df505d61bf19-cnibin\") pod \"multus-additional-cni-plugins-qhw7d\" (UID: \"c53cb4b0-7a90-4dba-b53d-df505d61bf19\") " pod="openshift-multus/multus-additional-cni-plugins-qhw7d"
Jul 14 13:54:13.609305 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.609113    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5600e3e1-e775-482e-bc72-76d732552705-env-overrides\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.609305 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.609153    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/02b53875-f2af-42e5-a311-74b8301e8b62-dbus\") pod \"global-pull-secret-syncer-5m6xn\" (UID: \"02b53875-f2af-42e5-a311-74b8301e8b62\") " pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:54:13.609305 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.609199    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c53cb4b0-7a90-4dba-b53d-df505d61bf19-os-release\") pod \"multus-additional-cni-plugins-qhw7d\" (UID: \"c53cb4b0-7a90-4dba-b53d-df505d61bf19\") " pod="openshift-multus/multus-additional-cni-plugins-qhw7d"
Jul 14 13:54:13.609305 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.609228    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-run-openvswitch\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.609305 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.609255    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-systemd-units\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.609305 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.609284    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-host-run-netns\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.609305 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.609290    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/02b53875-f2af-42e5-a311-74b8301e8b62-dbus\") pod \"global-pull-secret-syncer-5m6xn\" (UID: \"02b53875-f2af-42e5-a311-74b8301e8b62\") " pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:54:13.609699 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.609310    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lddt\" (UniqueName: \"kubernetes.io/projected/5600e3e1-e775-482e-bc72-76d732552705-kube-api-access-2lddt\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.609699 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.609315    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-run-openvswitch\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.609699 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.609335    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e589fbdd-065a-4025-8862-43ca4000fa4c-lib-modules\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.609699 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.609357    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e589fbdd-065a-4025-8862-43ca4000fa4c-tmp\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.609699 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.609369    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-systemd-units\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.609699 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.609374    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-host-run-netns\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.609699 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.609381    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-host-run-netns\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.609699 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.609417    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c53cb4b0-7a90-4dba-b53d-df505d61bf19-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qhw7d\" (UID: \"c53cb4b0-7a90-4dba-b53d-df505d61bf19\") " pod="openshift-multus/multus-additional-cni-plugins-qhw7d"
Jul 14 13:54:13.609699 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.609447    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e589fbdd-065a-4025-8862-43ca4000fa4c-etc-systemd\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.609699 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.609473    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a5c1fc8-edd8-4cb9-8afb-81970bbd10c6-tmp-dir\") pod \"node-resolver-jfpgf\" (UID: \"5a5c1fc8-edd8-4cb9-8afb-81970bbd10c6\") " pod="openshift-dns/node-resolver-jfpgf"
Jul 14 13:54:13.609699 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.609499    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-run-systemd\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.609699 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.609531    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a9b528fa-6f75-402e-8948-190262b8325a-host-slash\") pod \"iptables-alerter-9pk9w\" (UID: \"a9b528fa-6f75-402e-8948-190262b8325a\") " pod="openshift-network-operator/iptables-alerter-9pk9w"
Jul 14 13:54:13.609699 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.609559    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-log-socket\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.609699 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.609584    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7a539262-9b04-4fbf-ae2b-f922f910e471-device-dir\") pod \"aws-ebs-csi-driver-node-r5kpb\" (UID: \"7a539262-9b04-4fbf-ae2b-f922f910e471\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r5kpb"
Jul 14 13:54:13.609699 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.609613    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7wkx\" (UniqueName: \"kubernetes.io/projected/7a539262-9b04-4fbf-ae2b-f922f910e471-kube-api-access-z7wkx\") pod \"aws-ebs-csi-driver-node-r5kpb\" (UID: \"7a539262-9b04-4fbf-ae2b-f922f910e471\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r5kpb"
Jul 14 13:54:13.609699 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.609674    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e589fbdd-065a-4025-8862-43ca4000fa4c-var-lib-kubelet\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.609699 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.609687    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5600e3e1-e775-482e-bc72-76d732552705-env-overrides\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.610589 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.609699    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-host-run-ovn-kubernetes\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.610589 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.609762    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5600e3e1-e775-482e-bc72-76d732552705-ovn-node-metrics-cert\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.610589 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.609804    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a9b528fa-6f75-402e-8948-190262b8325a-host-slash\") pod \"iptables-alerter-9pk9w\" (UID: \"a9b528fa-6f75-402e-8948-190262b8325a\") " pod="openshift-network-operator/iptables-alerter-9pk9w"
Jul 14 13:54:13.610589 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.609880    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-run-systemd\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.610589 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.609901    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7k49l\" (UniqueName: \"kubernetes.io/projected/8a9c4e5c-fb07-4089-9d9c-f01896cd6e25-kube-api-access-7k49l\") pod \"node-ca-5pjcr\" (UID: \"8a9c4e5c-fb07-4089-9d9c-f01896cd6e25\") " pod="openshift-image-registry/node-ca-5pjcr"
Jul 14 13:54:13.610589 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.609927    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-log-socket\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.610589 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.609974    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-host-run-ovn-kubernetes\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.610589 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.610017    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-cnibin\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.610589 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.610043    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-os-release\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.610589 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.610042    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7a539262-9b04-4fbf-ae2b-f922f910e471-device-dir\") pod \"aws-ebs-csi-driver-node-r5kpb\" (UID: \"7a539262-9b04-4fbf-ae2b-f922f910e471\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r5kpb"
Jul 14 13:54:13.610589 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.610114    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-multus-daemon-config\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.610589 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.610151    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b23d0d9c-a401-4327-9167-db15cbf7095e-konnectivity-ca\") pod \"konnectivity-agent-65w26\" (UID: \"b23d0d9c-a401-4327-9167-db15cbf7095e\") " pod="kube-system/konnectivity-agent-65w26"
Jul 14 13:54:13.610589 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.610213    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5a5c1fc8-edd8-4cb9-8afb-81970bbd10c6-hosts-file\") pod \"node-resolver-jfpgf\" (UID: \"5a5c1fc8-edd8-4cb9-8afb-81970bbd10c6\") " pod="openshift-dns/node-resolver-jfpgf"
Jul 14 13:54:13.610589 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.610230    2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 14 13:54:13.610589 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.610284    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-host-slash\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.610589 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.610253    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-host-slash\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.610589 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.610326    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-host-var-lib-cni-bin\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.610589 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.610345    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b23d0d9c-a401-4327-9167-db15cbf7095e-agent-certs\") pod \"konnectivity-agent-65w26\" (UID: \"b23d0d9c-a401-4327-9167-db15cbf7095e\") " pod="kube-system/konnectivity-agent-65w26"
Jul 14 13:54:13.611736 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.610370    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/02b53875-f2af-42e5-a311-74b8301e8b62-original-pull-secret\") pod \"global-pull-secret-syncer-5m6xn\" (UID: \"02b53875-f2af-42e5-a311-74b8301e8b62\") " pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:54:13.611736 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.610396    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8a9c4e5c-fb07-4089-9d9c-f01896cd6e25-host\") pod \"node-ca-5pjcr\" (UID: \"8a9c4e5c-fb07-4089-9d9c-f01896cd6e25\") " pod="openshift-image-registry/node-ca-5pjcr"
Jul 14 13:54:13.611736 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.610436    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-host-var-lib-cni-multus\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.611736 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:13.610503    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 13:54:13.611736 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.610570    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-hostroot\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.611736 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:13.610621    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02b53875-f2af-42e5-a311-74b8301e8b62-original-pull-secret podName:02b53875-f2af-42e5-a311-74b8301e8b62 nodeName:}" failed. No retries permitted until 2026-07-14 13:54:14.110564586 +0000 UTC m=+1.961002111 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/02b53875-f2af-42e5-a311-74b8301e8b62-original-pull-secret") pod "global-pull-secret-syncer-5m6xn" (UID: "02b53875-f2af-42e5-a311-74b8301e8b62") : object "kube-system"/"original-pull-secret" not registered
Jul 14 13:54:13.611736 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.610658    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjcnq\" (UniqueName: \"kubernetes.io/projected/c53cb4b0-7a90-4dba-b53d-df505d61bf19-kube-api-access-hjcnq\") pod \"multus-additional-cni-plugins-qhw7d\" (UID: \"c53cb4b0-7a90-4dba-b53d-df505d61bf19\") " pod="openshift-multus/multus-additional-cni-plugins-qhw7d"
Jul 14 13:54:13.611736 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.610692    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5600e3e1-e775-482e-bc72-76d732552705-ovnkube-script-lib\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.611736 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.610727    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njsk9\" (UniqueName: \"kubernetes.io/projected/a9b528fa-6f75-402e-8948-190262b8325a-kube-api-access-njsk9\") pod \"iptables-alerter-9pk9w\" (UID: \"a9b528fa-6f75-402e-8948-190262b8325a\") " pod="openshift-network-operator/iptables-alerter-9pk9w"
Jul 14 13:54:13.611736 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.610776    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c53cb4b0-7a90-4dba-b53d-df505d61bf19-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qhw7d\" (UID: \"c53cb4b0-7a90-4dba-b53d-df505d61bf19\") " pod="openshift-multus/multus-additional-cni-plugins-qhw7d"
Jul 14 13:54:13.611736 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.610809    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-etc-openvswitch\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.611736 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.610852    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-run-ovn\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.611736 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.610878    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-node-log\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.611736 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.610905    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7a539262-9b04-4fbf-ae2b-f922f910e471-socket-dir\") pod \"aws-ebs-csi-driver-node-r5kpb\" (UID: \"7a539262-9b04-4fbf-ae2b-f922f910e471\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r5kpb"
Jul 14 13:54:13.611736 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.610931    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-multus-conf-dir\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.611736 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.610958    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e589fbdd-065a-4025-8862-43ca4000fa4c-etc-sysctl-d\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.612610 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.610986    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7a539262-9b04-4fbf-ae2b-f922f910e471-etc-selinux\") pod \"aws-ebs-csi-driver-node-r5kpb\" (UID: \"7a539262-9b04-4fbf-ae2b-f922f910e471\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r5kpb"
Jul 14 13:54:13.612610 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.611014    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqjm4\" (UniqueName: \"kubernetes.io/projected/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-kube-api-access-jqjm4\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.612610 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.611042    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-multus-socket-dir-parent\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.612610 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.611068    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c53cb4b0-7a90-4dba-b53d-df505d61bf19-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qhw7d\" (UID: \"c53cb4b0-7a90-4dba-b53d-df505d61bf19\") " pod="openshift-multus/multus-additional-cni-plugins-qhw7d"
Jul 14 13:54:13.612610 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.611151    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-etc-openvswitch\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.612610 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.611211    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/188bee4a-9fbe-4f95-8829-a91ab584ce4e-metrics-certs\") pod \"network-metrics-daemon-486sw\" (UID: \"188bee4a-9fbe-4f95-8829-a91ab584ce4e\") " pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:54:13.612610 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.611221    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-run-ovn\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.612610 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.611252    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blm59\" (UniqueName: \"kubernetes.io/projected/e589fbdd-065a-4025-8862-43ca4000fa4c-kube-api-access-blm59\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.612610 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.611265    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-node-log\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.612610 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.611277    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-host-run-multus-certs\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.612610 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.611334    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-host-kubelet\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.612610 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.611376    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-host-run-k8s-cni-cncf-io\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.612610 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.611384    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7a539262-9b04-4fbf-ae2b-f922f910e471-socket-dir\") pod \"aws-ebs-csi-driver-node-r5kpb\" (UID: \"7a539262-9b04-4fbf-ae2b-f922f910e471\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r5kpb"
Jul 14 13:54:13.612610 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.611397    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxh8k\" (UniqueName: \"kubernetes.io/projected/188bee4a-9fbe-4f95-8829-a91ab584ce4e-kube-api-access-lxh8k\") pod \"network-metrics-daemon-486sw\" (UID: \"188bee4a-9fbe-4f95-8829-a91ab584ce4e\") " pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:54:13.612610 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.611441    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e589fbdd-065a-4025-8862-43ca4000fa4c-etc-sysctl-conf\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.612610 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.611466    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-host-kubelet\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.612610 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.611473    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7a539262-9b04-4fbf-ae2b-f922f910e471-etc-selinux\") pod \"aws-ebs-csi-driver-node-r5kpb\" (UID: \"7a539262-9b04-4fbf-ae2b-f922f910e471\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r5kpb"
Jul 14 13:54:13.613178 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.611471    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e589fbdd-065a-4025-8862-43ca4000fa4c-etc-tuned\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.613178 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.611537    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sl8qs\" (UniqueName: \"kubernetes.io/projected/5a5c1fc8-edd8-4cb9-8afb-81970bbd10c6-kube-api-access-sl8qs\") pod \"node-resolver-jfpgf\" (UID: \"5a5c1fc8-edd8-4cb9-8afb-81970bbd10c6\") " pod="openshift-dns/node-resolver-jfpgf"
Jul 14 13:54:13.613178 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.611567    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a9b528fa-6f75-402e-8948-190262b8325a-iptables-alerter-script\") pod \"iptables-alerter-9pk9w\" (UID: \"a9b528fa-6f75-402e-8948-190262b8325a\") " pod="openshift-network-operator/iptables-alerter-9pk9w"
Jul 14 13:54:13.613178 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.611594    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/02b53875-f2af-42e5-a311-74b8301e8b62-kubelet-config\") pod \"global-pull-secret-syncer-5m6xn\" (UID: \"02b53875-f2af-42e5-a311-74b8301e8b62\") " pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:54:13.613178 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.611634    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7a539262-9b04-4fbf-ae2b-f922f910e471-sys-fs\") pod \"aws-ebs-csi-driver-node-r5kpb\" (UID: \"7a539262-9b04-4fbf-ae2b-f922f910e471\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r5kpb"
Jul 14 13:54:13.613178 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.611655    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-host-var-lib-kubelet\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.613178 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.611716    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e589fbdd-065a-4025-8862-43ca4000fa4c-etc-kubernetes\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.613178 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.611738    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e589fbdd-065a-4025-8862-43ca4000fa4c-sys\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.613178 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.611762    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e589fbdd-065a-4025-8862-43ca4000fa4c-host\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.613178 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.611786    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-host-cni-netd\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.613178 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.611813    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.613178 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.611847    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c53cb4b0-7a90-4dba-b53d-df505d61bf19-cni-binary-copy\") pod \"multus-additional-cni-plugins-qhw7d\" (UID: \"c53cb4b0-7a90-4dba-b53d-df505d61bf19\") " pod="openshift-multus/multus-additional-cni-plugins-qhw7d"
Jul 14 13:54:13.613178 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.611865    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5600e3e1-e775-482e-bc72-76d732552705-ovnkube-script-lib\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.613178 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.611871    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e589fbdd-065a-4025-8862-43ca4000fa4c-run\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.613178 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.611894    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-host-cni-bin\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.613178 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.611962    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/02b53875-f2af-42e5-a311-74b8301e8b62-kubelet-config\") pod \"global-pull-secret-syncer-5m6xn\" (UID: \"02b53875-f2af-42e5-a311-74b8301e8b62\") " pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:54:13.613178 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.612000    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-host-cni-netd\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.613769 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.612014    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.613769 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.612051    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-host-cni-bin\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.613769 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.612078    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7a539262-9b04-4fbf-ae2b-f922f910e471-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r5kpb\" (UID: \"7a539262-9b04-4fbf-ae2b-f922f910e471\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r5kpb"
Jul 14 13:54:13.613769 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.612096    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7a539262-9b04-4fbf-ae2b-f922f910e471-sys-fs\") pod \"aws-ebs-csi-driver-node-r5kpb\" (UID: \"7a539262-9b04-4fbf-ae2b-f922f910e471\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r5kpb"
Jul 14 13:54:13.613769 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.612147    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7a539262-9b04-4fbf-ae2b-f922f910e471-registration-dir\") pod \"aws-ebs-csi-driver-node-r5kpb\" (UID: \"7a539262-9b04-4fbf-ae2b-f922f910e471\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r5kpb"
Jul 14 13:54:13.613769 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.612177    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7a539262-9b04-4fbf-ae2b-f922f910e471-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r5kpb\" (UID: \"7a539262-9b04-4fbf-ae2b-f922f910e471\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r5kpb"
Jul 14 13:54:13.613769 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.612214    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8a9c4e5c-fb07-4089-9d9c-f01896cd6e25-serviceca\") pod \"node-ca-5pjcr\" (UID: \"8a9c4e5c-fb07-4089-9d9c-f01896cd6e25\") " pod="openshift-image-registry/node-ca-5pjcr"
Jul 14 13:54:13.613769 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.612242    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f274l\" (UniqueName: \"kubernetes.io/projected/421bfc78-19a7-4327-8fde-0b7a9927bb4e-kube-api-access-f274l\") pod \"network-check-target-22z2l\" (UID: \"421bfc78-19a7-4327-8fde-0b7a9927bb4e\") " pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:54:13.613769 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.612278    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e589fbdd-065a-4025-8862-43ca4000fa4c-etc-modprobe-d\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.613769 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.612305    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e589fbdd-065a-4025-8862-43ca4000fa4c-etc-sysconfig\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.613769 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.612320    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7a539262-9b04-4fbf-ae2b-f922f910e471-registration-dir\") pod \"aws-ebs-csi-driver-node-r5kpb\" (UID: \"7a539262-9b04-4fbf-ae2b-f922f910e471\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r5kpb"
Jul 14 13:54:13.613769 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.612336    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-var-lib-openvswitch\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.613769 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.612364    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5600e3e1-e775-482e-bc72-76d732552705-ovnkube-config\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.613769 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.612370    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5600e3e1-e775-482e-bc72-76d732552705-var-lib-openvswitch\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.613769 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.612396    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a9b528fa-6f75-402e-8948-190262b8325a-iptables-alerter-script\") pod \"iptables-alerter-9pk9w\" (UID: \"a9b528fa-6f75-402e-8948-190262b8325a\") " pod="openshift-network-operator/iptables-alerter-9pk9w"
Jul 14 13:54:13.613769 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.612823    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5600e3e1-e775-482e-bc72-76d732552705-ovnkube-config\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.614385 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.614048    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5600e3e1-e775-482e-bc72-76d732552705-ovn-node-metrics-cert\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.621573 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.621546    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7wkx\" (UniqueName: \"kubernetes.io/projected/7a539262-9b04-4fbf-ae2b-f922f910e471-kube-api-access-z7wkx\") pod \"aws-ebs-csi-driver-node-r5kpb\" (UID: \"7a539262-9b04-4fbf-ae2b-f922f910e471\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r5kpb"
Jul 14 13:54:13.621573 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.621557    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lddt\" (UniqueName: \"kubernetes.io/projected/5600e3e1-e775-482e-bc72-76d732552705-kube-api-access-2lddt\") pod \"ovnkube-node-sh6rt\" (UID: \"5600e3e1-e775-482e-bc72-76d732552705\") " pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.623768 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.623745    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njsk9\" (UniqueName: \"kubernetes.io/projected/a9b528fa-6f75-402e-8948-190262b8325a-kube-api-access-njsk9\") pod \"iptables-alerter-9pk9w\" (UID: \"a9b528fa-6f75-402e-8948-190262b8325a\") " pod="openshift-network-operator/iptables-alerter-9pk9w"
Jul 14 13:54:13.712860 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.712820    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-host-var-lib-kubelet\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.712860 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.712861    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e589fbdd-065a-4025-8862-43ca4000fa4c-etc-kubernetes\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.713134 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.712884    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e589fbdd-065a-4025-8862-43ca4000fa4c-sys\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.713134 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.712907    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e589fbdd-065a-4025-8862-43ca4000fa4c-host\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.713134 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.712929    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c53cb4b0-7a90-4dba-b53d-df505d61bf19-cni-binary-copy\") pod \"multus-additional-cni-plugins-qhw7d\" (UID: \"c53cb4b0-7a90-4dba-b53d-df505d61bf19\") " pod="openshift-multus/multus-additional-cni-plugins-qhw7d"
Jul 14 13:54:13.713134 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.712953    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-host-var-lib-kubelet\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.713134 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.712969    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e589fbdd-065a-4025-8862-43ca4000fa4c-etc-kubernetes\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.713134 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.712980    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e589fbdd-065a-4025-8862-43ca4000fa4c-sys\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.713134 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.712986    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e589fbdd-065a-4025-8862-43ca4000fa4c-host\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.713134 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713010    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e589fbdd-065a-4025-8862-43ca4000fa4c-run\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.713134 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713043    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8a9c4e5c-fb07-4089-9d9c-f01896cd6e25-serviceca\") pod \"node-ca-5pjcr\" (UID: \"8a9c4e5c-fb07-4089-9d9c-f01896cd6e25\") " pod="openshift-image-registry/node-ca-5pjcr"
Jul 14 13:54:13.713134 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713062    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f274l\" (UniqueName: \"kubernetes.io/projected/421bfc78-19a7-4327-8fde-0b7a9927bb4e-kube-api-access-f274l\") pod \"network-check-target-22z2l\" (UID: \"421bfc78-19a7-4327-8fde-0b7a9927bb4e\") " pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:54:13.713134 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713087    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e589fbdd-065a-4025-8862-43ca4000fa4c-etc-modprobe-d\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.713134 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713104    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e589fbdd-065a-4025-8862-43ca4000fa4c-etc-sysconfig\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.713134 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713095    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e589fbdd-065a-4025-8862-43ca4000fa4c-run\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.713786 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713174    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-system-cni-dir\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.713786 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713124    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-system-cni-dir\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.713786 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713211    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e589fbdd-065a-4025-8862-43ca4000fa4c-etc-sysconfig\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.713786 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713218    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-multus-cni-dir\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.713786 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713245    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-cni-binary-copy\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.713786 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713267    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-multus-cni-dir\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.713786 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713272    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-etc-kubernetes\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.713786 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713305    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-etc-kubernetes\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.713786 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713300    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c53cb4b0-7a90-4dba-b53d-df505d61bf19-system-cni-dir\") pod \"multus-additional-cni-plugins-qhw7d\" (UID: \"c53cb4b0-7a90-4dba-b53d-df505d61bf19\") " pod="openshift-multus/multus-additional-cni-plugins-qhw7d"
Jul 14 13:54:13.713786 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713270    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e589fbdd-065a-4025-8862-43ca4000fa4c-etc-modprobe-d\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.713786 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713345    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c53cb4b0-7a90-4dba-b53d-df505d61bf19-cnibin\") pod \"multus-additional-cni-plugins-qhw7d\" (UID: \"c53cb4b0-7a90-4dba-b53d-df505d61bf19\") " pod="openshift-multus/multus-additional-cni-plugins-qhw7d"
Jul 14 13:54:13.713786 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713376    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c53cb4b0-7a90-4dba-b53d-df505d61bf19-os-release\") pod \"multus-additional-cni-plugins-qhw7d\" (UID: \"c53cb4b0-7a90-4dba-b53d-df505d61bf19\") " pod="openshift-multus/multus-additional-cni-plugins-qhw7d"
Jul 14 13:54:13.713786 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713384    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c53cb4b0-7a90-4dba-b53d-df505d61bf19-system-cni-dir\") pod \"multus-additional-cni-plugins-qhw7d\" (UID: \"c53cb4b0-7a90-4dba-b53d-df505d61bf19\") " pod="openshift-multus/multus-additional-cni-plugins-qhw7d"
Jul 14 13:54:13.713786 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713405    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e589fbdd-065a-4025-8862-43ca4000fa4c-lib-modules\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.713786 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713395    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c53cb4b0-7a90-4dba-b53d-df505d61bf19-cnibin\") pod \"multus-additional-cni-plugins-qhw7d\" (UID: \"c53cb4b0-7a90-4dba-b53d-df505d61bf19\") " pod="openshift-multus/multus-additional-cni-plugins-qhw7d"
Jul 14 13:54:13.713786 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713441    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e589fbdd-065a-4025-8862-43ca4000fa4c-tmp\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.713786 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713478    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-host-run-netns\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.713786 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713496    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c53cb4b0-7a90-4dba-b53d-df505d61bf19-os-release\") pod \"multus-additional-cni-plugins-qhw7d\" (UID: \"c53cb4b0-7a90-4dba-b53d-df505d61bf19\") " pod="openshift-multus/multus-additional-cni-plugins-qhw7d"
Jul 14 13:54:13.714737 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713507    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c53cb4b0-7a90-4dba-b53d-df505d61bf19-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qhw7d\" (UID: \"c53cb4b0-7a90-4dba-b53d-df505d61bf19\") " pod="openshift-multus/multus-additional-cni-plugins-qhw7d"
Jul 14 13:54:13.714737 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713533    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e589fbdd-065a-4025-8862-43ca4000fa4c-etc-systemd\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.714737 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713537    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e589fbdd-065a-4025-8862-43ca4000fa4c-lib-modules\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.714737 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713543    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-host-run-netns\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.714737 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713553    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8a9c4e5c-fb07-4089-9d9c-f01896cd6e25-serviceca\") pod \"node-ca-5pjcr\" (UID: \"8a9c4e5c-fb07-4089-9d9c-f01896cd6e25\") " pod="openshift-image-registry/node-ca-5pjcr"
Jul 14 13:54:13.714737 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713569    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a5c1fc8-edd8-4cb9-8afb-81970bbd10c6-tmp-dir\") pod \"node-resolver-jfpgf\" (UID: \"5a5c1fc8-edd8-4cb9-8afb-81970bbd10c6\") " pod="openshift-dns/node-resolver-jfpgf"
Jul 14 13:54:13.714737 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713626    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c53cb4b0-7a90-4dba-b53d-df505d61bf19-cni-binary-copy\") pod \"multus-additional-cni-plugins-qhw7d\" (UID: \"c53cb4b0-7a90-4dba-b53d-df505d61bf19\") " pod="openshift-multus/multus-additional-cni-plugins-qhw7d"
Jul 14 13:54:13.714737 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713675    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e589fbdd-065a-4025-8862-43ca4000fa4c-etc-systemd\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.714737 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713693    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e589fbdd-065a-4025-8862-43ca4000fa4c-var-lib-kubelet\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.714737 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713712    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c53cb4b0-7a90-4dba-b53d-df505d61bf19-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qhw7d\" (UID: \"c53cb4b0-7a90-4dba-b53d-df505d61bf19\") " pod="openshift-multus/multus-additional-cni-plugins-qhw7d"
Jul 14 13:54:13.714737 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713732    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7k49l\" (UniqueName: \"kubernetes.io/projected/8a9c4e5c-fb07-4089-9d9c-f01896cd6e25-kube-api-access-7k49l\") pod \"node-ca-5pjcr\" (UID: \"8a9c4e5c-fb07-4089-9d9c-f01896cd6e25\") " pod="openshift-image-registry/node-ca-5pjcr"
Jul 14 13:54:13.714737 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713754    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e589fbdd-065a-4025-8862-43ca4000fa4c-var-lib-kubelet\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.714737 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713762    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-cnibin\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.714737 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713787    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-os-release\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.714737 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713804    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-cnibin\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.714737 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713812    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-multus-daemon-config\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.714737 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713839    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b23d0d9c-a401-4327-9167-db15cbf7095e-konnectivity-ca\") pod \"konnectivity-agent-65w26\" (UID: \"b23d0d9c-a401-4327-9167-db15cbf7095e\") " pod="kube-system/konnectivity-agent-65w26"
Jul 14 13:54:13.714737 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713864    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5a5c1fc8-edd8-4cb9-8afb-81970bbd10c6-hosts-file\") pod \"node-resolver-jfpgf\" (UID: \"5a5c1fc8-edd8-4cb9-8afb-81970bbd10c6\") " pod="openshift-dns/node-resolver-jfpgf"
Jul 14 13:54:13.715473 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713865    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-cni-binary-copy\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.715473 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713870    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-os-release\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.715473 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713889    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-host-var-lib-cni-bin\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.715473 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713920    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a5c1fc8-edd8-4cb9-8afb-81970bbd10c6-tmp-dir\") pod \"node-resolver-jfpgf\" (UID: \"5a5c1fc8-edd8-4cb9-8afb-81970bbd10c6\") " pod="openshift-dns/node-resolver-jfpgf"
Jul 14 13:54:13.715473 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713931    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b23d0d9c-a401-4327-9167-db15cbf7095e-agent-certs\") pod \"konnectivity-agent-65w26\" (UID: \"b23d0d9c-a401-4327-9167-db15cbf7095e\") " pod="kube-system/konnectivity-agent-65w26"
Jul 14 13:54:13.715473 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713932    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-host-var-lib-cni-bin\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.715473 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713995    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8a9c4e5c-fb07-4089-9d9c-f01896cd6e25-host\") pod \"node-ca-5pjcr\" (UID: \"8a9c4e5c-fb07-4089-9d9c-f01896cd6e25\") " pod="openshift-image-registry/node-ca-5pjcr"
Jul 14 13:54:13.715473 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.714022    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-host-var-lib-cni-multus\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.715473 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.713992    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5a5c1fc8-edd8-4cb9-8afb-81970bbd10c6-hosts-file\") pod \"node-resolver-jfpgf\" (UID: \"5a5c1fc8-edd8-4cb9-8afb-81970bbd10c6\") " pod="openshift-dns/node-resolver-jfpgf"
Jul 14 13:54:13.715473 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.714049    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-hostroot\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.715473 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.714094    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjcnq\" (UniqueName: \"kubernetes.io/projected/c53cb4b0-7a90-4dba-b53d-df505d61bf19-kube-api-access-hjcnq\") pod \"multus-additional-cni-plugins-qhw7d\" (UID: \"c53cb4b0-7a90-4dba-b53d-df505d61bf19\") " pod="openshift-multus/multus-additional-cni-plugins-qhw7d"
Jul 14 13:54:13.715473 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.714048    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8a9c4e5c-fb07-4089-9d9c-f01896cd6e25-host\") pod \"node-ca-5pjcr\" (UID: \"8a9c4e5c-fb07-4089-9d9c-f01896cd6e25\") " pod="openshift-image-registry/node-ca-5pjcr"
Jul 14 13:54:13.715473 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.714106    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-hostroot\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.715473 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.714096    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-host-var-lib-cni-multus\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.715473 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.714125    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c53cb4b0-7a90-4dba-b53d-df505d61bf19-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qhw7d\" (UID: \"c53cb4b0-7a90-4dba-b53d-df505d61bf19\") " pod="openshift-multus/multus-additional-cni-plugins-qhw7d"
Jul 14 13:54:13.715473 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.714175    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-multus-conf-dir\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.715473 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.714202    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e589fbdd-065a-4025-8862-43ca4000fa4c-etc-sysctl-d\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.715473 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.714247    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqjm4\" (UniqueName: \"kubernetes.io/projected/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-kube-api-access-jqjm4\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.716211 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.714274    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-multus-socket-dir-parent\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.716211 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.714280    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-multus-daemon-config\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.716211 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.714302    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c53cb4b0-7a90-4dba-b53d-df505d61bf19-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qhw7d\" (UID: \"c53cb4b0-7a90-4dba-b53d-df505d61bf19\") " pod="openshift-multus/multus-additional-cni-plugins-qhw7d"
Jul 14 13:54:13.716211 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.714346    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/188bee4a-9fbe-4f95-8829-a91ab584ce4e-metrics-certs\") pod \"network-metrics-daemon-486sw\" (UID: \"188bee4a-9fbe-4f95-8829-a91ab584ce4e\") " pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:54:13.716211 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.714360    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-multus-conf-dir\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.716211 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.714373    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blm59\" (UniqueName: \"kubernetes.io/projected/e589fbdd-065a-4025-8862-43ca4000fa4c-kube-api-access-blm59\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.716211 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.714417    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-host-run-multus-certs\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.716211 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.714446    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-host-run-k8s-cni-cncf-io\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.716211 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.714453    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b23d0d9c-a401-4327-9167-db15cbf7095e-konnectivity-ca\") pod \"konnectivity-agent-65w26\" (UID: \"b23d0d9c-a401-4327-9167-db15cbf7095e\") " pod="kube-system/konnectivity-agent-65w26"
Jul 14 13:54:13.716211 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:13.714474    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 13:54:13.716211 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.714474    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxh8k\" (UniqueName: \"kubernetes.io/projected/188bee4a-9fbe-4f95-8829-a91ab584ce4e-kube-api-access-lxh8k\") pod \"network-metrics-daemon-486sw\" (UID: \"188bee4a-9fbe-4f95-8829-a91ab584ce4e\") " pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:54:13.716211 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.714517    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e589fbdd-065a-4025-8862-43ca4000fa4c-etc-sysctl-conf\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.716211 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.714528    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-host-run-multus-certs\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.716211 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.714417    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e589fbdd-065a-4025-8862-43ca4000fa4c-etc-sysctl-d\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.716211 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:13.714584    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/188bee4a-9fbe-4f95-8829-a91ab584ce4e-metrics-certs podName:188bee4a-9fbe-4f95-8829-a91ab584ce4e nodeName:}" failed. No retries permitted until 2026-07-14 13:54:14.214525451 +0000 UTC m=+2.064962948 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/188bee4a-9fbe-4f95-8829-a91ab584ce4e-metrics-certs") pod "network-metrics-daemon-486sw" (UID: "188bee4a-9fbe-4f95-8829-a91ab584ce4e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 13:54:13.716211 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.714702    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-multus-socket-dir-parent\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.716211 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.714704    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c53cb4b0-7a90-4dba-b53d-df505d61bf19-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qhw7d\" (UID: \"c53cb4b0-7a90-4dba-b53d-df505d61bf19\") " pod="openshift-multus/multus-additional-cni-plugins-qhw7d"
Jul 14 13:54:13.717135 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.714755    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e589fbdd-065a-4025-8862-43ca4000fa4c-etc-tuned\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.717135 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.714799    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl8qs\" (UniqueName: \"kubernetes.io/projected/5a5c1fc8-edd8-4cb9-8afb-81970bbd10c6-kube-api-access-sl8qs\") pod \"node-resolver-jfpgf\" (UID: \"5a5c1fc8-edd8-4cb9-8afb-81970bbd10c6\") " pod="openshift-dns/node-resolver-jfpgf"
Jul 14 13:54:13.717135 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.714952    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e589fbdd-065a-4025-8862-43ca4000fa4c-etc-sysctl-conf\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.717135 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.715194    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-host-run-k8s-cni-cncf-io\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.717135 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.715346    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c53cb4b0-7a90-4dba-b53d-df505d61bf19-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qhw7d\" (UID: \"c53cb4b0-7a90-4dba-b53d-df505d61bf19\") " pod="openshift-multus/multus-additional-cni-plugins-qhw7d"
Jul 14 13:54:13.717135 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.716140    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e589fbdd-065a-4025-8862-43ca4000fa4c-tmp\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.717135 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.716490    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b23d0d9c-a401-4327-9167-db15cbf7095e-agent-certs\") pod \"konnectivity-agent-65w26\" (UID: \"b23d0d9c-a401-4327-9167-db15cbf7095e\") " pod="kube-system/konnectivity-agent-65w26"
Jul 14 13:54:13.717135 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.716758    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e589fbdd-065a-4025-8862-43ca4000fa4c-etc-tuned\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.722641 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:13.722614    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 13:54:13.722800 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:13.722646    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 13:54:13.722800 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:13.722661    2576 projected.go:196] Error preparing data for projected volume kube-api-access-f274l for pod openshift-network-diagnostics/network-check-target-22z2l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 13:54:13.722800 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:13.722787    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/421bfc78-19a7-4327-8fde-0b7a9927bb4e-kube-api-access-f274l podName:421bfc78-19a7-4327-8fde-0b7a9927bb4e nodeName:}" failed. No retries permitted until 2026-07-14 13:54:14.222764643 +0000 UTC m=+2.073202156 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-f274l" (UniqueName: "kubernetes.io/projected/421bfc78-19a7-4327-8fde-0b7a9927bb4e-kube-api-access-f274l") pod "network-check-target-22z2l" (UID: "421bfc78-19a7-4327-8fde-0b7a9927bb4e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 13:54:13.724785 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.724749    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7k49l\" (UniqueName: \"kubernetes.io/projected/8a9c4e5c-fb07-4089-9d9c-f01896cd6e25-kube-api-access-7k49l\") pod \"node-ca-5pjcr\" (UID: \"8a9c4e5c-fb07-4089-9d9c-f01896cd6e25\") " pod="openshift-image-registry/node-ca-5pjcr"
Jul 14 13:54:13.725235 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.725206    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjcnq\" (UniqueName: \"kubernetes.io/projected/c53cb4b0-7a90-4dba-b53d-df505d61bf19-kube-api-access-hjcnq\") pod \"multus-additional-cni-plugins-qhw7d\" (UID: \"c53cb4b0-7a90-4dba-b53d-df505d61bf19\") " pod="openshift-multus/multus-additional-cni-plugins-qhw7d"
Jul 14 13:54:13.725357 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.725210    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxh8k\" (UniqueName: \"kubernetes.io/projected/188bee4a-9fbe-4f95-8829-a91ab584ce4e-kube-api-access-lxh8k\") pod \"network-metrics-daemon-486sw\" (UID: \"188bee4a-9fbe-4f95-8829-a91ab584ce4e\") " pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:54:13.725477 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.725461    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqjm4\" (UniqueName: \"kubernetes.io/projected/9e9684f1-f646-4a49-a4c8-23d51e4a94e3-kube-api-access-jqjm4\") pod \"multus-jbr68\" (UID: \"9e9684f1-f646-4a49-a4c8-23d51e4a94e3\") " pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.725591 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.725574    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blm59\" (UniqueName: \"kubernetes.io/projected/e589fbdd-065a-4025-8862-43ca4000fa4c-kube-api-access-blm59\") pod \"tuned-tfd72\" (UID: \"e589fbdd-065a-4025-8862-43ca4000fa4c\") " pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.725859 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.725838    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sl8qs\" (UniqueName: \"kubernetes.io/projected/5a5c1fc8-edd8-4cb9-8afb-81970bbd10c6-kube-api-access-sl8qs\") pod \"node-resolver-jfpgf\" (UID: \"5a5c1fc8-edd8-4cb9-8afb-81970bbd10c6\") " pod="openshift-dns/node-resolver-jfpgf"
Jul 14 13:54:13.829709 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.829580    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9pk9w"
Jul 14 13:54:13.844225 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.844195    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:13.859407 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.859372    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r5kpb"
Jul 14 13:54:13.877230 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.877190    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tfd72"
Jul 14 13:54:13.894704 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.894662    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jfpgf"
Jul 14 13:54:13.910260 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.910202    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5pjcr"
Jul 14 13:54:13.931881 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.931848    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qhw7d"
Jul 14 13:54:13.942351 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.942325    2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 13:54:13.948019 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.947987    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jbr68"
Jul 14 13:54:13.952149 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:13.952130    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-65w26"
Jul 14 13:54:14.003459 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:14.003424    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbb95fe8aa83faa678c273911aee96f3.slice/crio-f77982809d473b89b045b457d6974457c3d63fc47e60ab00d1d8d449c76a56da WatchSource:0}: Error finding container f77982809d473b89b045b457d6974457c3d63fc47e60ab00d1d8d449c76a56da: Status 404 returned error can't find the container with id f77982809d473b89b045b457d6974457c3d63fc47e60ab00d1d8d449c76a56da
Jul 14 13:54:14.007893 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:14.007866    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 13:54:14.013148 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:14.013117    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c0fbe03dcb71398a547458685400965.slice/crio-2e55463995f1a72ba27c883ffc115a5c7904cfdc6755523583dfbc1cd2413818 WatchSource:0}: Error finding container 2e55463995f1a72ba27c883ffc115a5c7904cfdc6755523583dfbc1cd2413818: Status 404 returned error can't find the container with id 2e55463995f1a72ba27c883ffc115a5c7904cfdc6755523583dfbc1cd2413818
Jul 14 13:54:14.083579 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:14.083480    2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 13:54:14.118170 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:14.118130    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/02b53875-f2af-42e5-a311-74b8301e8b62-original-pull-secret\") pod \"global-pull-secret-syncer-5m6xn\" (UID: \"02b53875-f2af-42e5-a311-74b8301e8b62\") " pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:54:14.118375 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:14.118288    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 13:54:14.118375 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:14.118365    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02b53875-f2af-42e5-a311-74b8301e8b62-original-pull-secret podName:02b53875-f2af-42e5-a311-74b8301e8b62 nodeName:}" failed. No retries permitted until 2026-07-14 13:54:15.118343193 +0000 UTC m=+2.968780703 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/02b53875-f2af-42e5-a311-74b8301e8b62-original-pull-secret") pod "global-pull-secret-syncer-5m6xn" (UID: "02b53875-f2af-42e5-a311-74b8301e8b62") : object "kube-system"/"original-pull-secret" not registered
Jul 14 13:54:14.172750 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:14.172712    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5600e3e1_e775_482e_bc72_76d732552705.slice/crio-d8d8373020d641081da0750067d9a728e11050200e6cbbc9133f1b46959875df WatchSource:0}: Error finding container d8d8373020d641081da0750067d9a728e11050200e6cbbc9133f1b46959875df: Status 404 returned error can't find the container with id d8d8373020d641081da0750067d9a728e11050200e6cbbc9133f1b46959875df
Jul 14 13:54:14.212050 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:14.212016    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9b528fa_6f75_402e_8948_190262b8325a.slice/crio-9d86d5562dc920512c603b34f7870e1535a20b1968bb4b10dde510737b226c9c WatchSource:0}: Error finding container 9d86d5562dc920512c603b34f7870e1535a20b1968bb4b10dde510737b226c9c: Status 404 returned error can't find the container with id 9d86d5562dc920512c603b34f7870e1535a20b1968bb4b10dde510737b226c9c
Jul 14 13:54:14.218941 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:14.218915    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/188bee4a-9fbe-4f95-8829-a91ab584ce4e-metrics-certs\") pod \"network-metrics-daemon-486sw\" (UID: \"188bee4a-9fbe-4f95-8829-a91ab584ce4e\") " pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:54:14.219177 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:14.219077    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 13:54:14.219177 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:14.219142    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/188bee4a-9fbe-4f95-8829-a91ab584ce4e-metrics-certs podName:188bee4a-9fbe-4f95-8829-a91ab584ce4e nodeName:}" failed. No retries permitted until 2026-07-14 13:54:15.219125554 +0000 UTC m=+3.069563054 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/188bee4a-9fbe-4f95-8829-a91ab584ce4e-metrics-certs") pod "network-metrics-daemon-486sw" (UID: "188bee4a-9fbe-4f95-8829-a91ab584ce4e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 13:54:14.231985 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:14.231949    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e9684f1_f646_4a49_a4c8_23d51e4a94e3.slice/crio-9f5dea97c70c6fa3dd0f1846ebb6b98f2a02019300e52f66d29ac8280ce10903 WatchSource:0}: Error finding container 9f5dea97c70c6fa3dd0f1846ebb6b98f2a02019300e52f66d29ac8280ce10903: Status 404 returned error can't find the container with id 9f5dea97c70c6fa3dd0f1846ebb6b98f2a02019300e52f66d29ac8280ce10903
Jul 14 13:54:14.300519 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:14.300479    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc53cb4b0_7a90_4dba_b53d_df505d61bf19.slice/crio-dd261f344b29eb1420094b31c46ea0df50519fa4cb634158bf43dd90565d57db WatchSource:0}: Error finding container dd261f344b29eb1420094b31c46ea0df50519fa4cb634158bf43dd90565d57db: Status 404 returned error can't find the container with id dd261f344b29eb1420094b31c46ea0df50519fa4cb634158bf43dd90565d57db
Jul 14 13:54:14.320244 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:14.320204    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f274l\" (UniqueName: \"kubernetes.io/projected/421bfc78-19a7-4327-8fde-0b7a9927bb4e-kube-api-access-f274l\") pod \"network-check-target-22z2l\" (UID: \"421bfc78-19a7-4327-8fde-0b7a9927bb4e\") " pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:54:14.320433 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:14.320377    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 13:54:14.320433 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:14.320403    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 13:54:14.320433 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:14.320414    2576 projected.go:196] Error preparing data for projected volume kube-api-access-f274l for pod openshift-network-diagnostics/network-check-target-22z2l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 13:54:14.320619 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:14.320470    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/421bfc78-19a7-4327-8fde-0b7a9927bb4e-kube-api-access-f274l podName:421bfc78-19a7-4327-8fde-0b7a9927bb4e nodeName:}" failed. No retries permitted until 2026-07-14 13:54:15.320453464 +0000 UTC m=+3.170890948 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-f274l" (UniqueName: "kubernetes.io/projected/421bfc78-19a7-4327-8fde-0b7a9927bb4e-kube-api-access-f274l") pod "network-check-target-22z2l" (UID: "421bfc78-19a7-4327-8fde-0b7a9927bb4e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 13:54:14.323411 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:14.323363    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a9c4e5c_fb07_4089_9d9c_f01896cd6e25.slice/crio-3313bd0b897e9e6e580135250224e65ed4b32dba9792091680f38ad052ff8714 WatchSource:0}: Error finding container 3313bd0b897e9e6e580135250224e65ed4b32dba9792091680f38ad052ff8714: Status 404 returned error can't find the container with id 3313bd0b897e9e6e580135250224e65ed4b32dba9792091680f38ad052ff8714
Jul 14 13:54:14.401689 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:14.401659    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a539262_9b04_4fbf_ae2b_f922f910e471.slice/crio-1a62b40fe3aebc451b6e8fd36be26bdfebb60a78d01f47e005cc33476efc3f50 WatchSource:0}: Error finding container 1a62b40fe3aebc451b6e8fd36be26bdfebb60a78d01f47e005cc33476efc3f50: Status 404 returned error can't find the container with id 1a62b40fe3aebc451b6e8fd36be26bdfebb60a78d01f47e005cc33476efc3f50
Jul 14 13:54:14.451493 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:14.451460    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode589fbdd_065a_4025_8862_43ca4000fa4c.slice/crio-8ecc80acbff6618ccfe0566906f9ffb7703f2399ab3dca61ca0798fc07942953 WatchSource:0}: Error finding container 8ecc80acbff6618ccfe0566906f9ffb7703f2399ab3dca61ca0798fc07942953: Status 404 returned error can't find the container with id 8ecc80acbff6618ccfe0566906f9ffb7703f2399ab3dca61ca0798fc07942953
Jul 14 13:54:14.497333 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:14.497296    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a5c1fc8_edd8_4cb9_8afb_81970bbd10c6.slice/crio-bae03f8d0a110a09f6df176812517547bc7cfd8ea9ff7bd1f06c5df495bfbd54 WatchSource:0}: Error finding container bae03f8d0a110a09f6df176812517547bc7cfd8ea9ff7bd1f06c5df495bfbd54: Status 404 returned error can't find the container with id bae03f8d0a110a09f6df176812517547bc7cfd8ea9ff7bd1f06c5df495bfbd54
Jul 14 13:54:14.525868 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:14.525838    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 13:54:14.560162 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:14.560126    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 13:49:13 +0000 UTC" deadline="2028-03-24 17:27:41.077725738 +0000 UTC"
Jul 14 13:54:14.560162 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:14.560158    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14859h33m26.517571798s"
Jul 14 13:54:14.628537 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:54:14.628503    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb23d0d9c_a401_4327_9167_db15cbf7095e.slice/crio-253df59870678a1efa2c3b3b21997baca43bb435f331a76992365bdca18c8597 WatchSource:0}: Error finding container 253df59870678a1efa2c3b3b21997baca43bb435f331a76992365bdca18c8597: Status 404 returned error can't find the container with id 253df59870678a1efa2c3b3b21997baca43bb435f331a76992365bdca18c8597
Jul 14 13:54:14.686387 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:14.686155    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:54:14.686387 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:14.686283    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22z2l" podUID="421bfc78-19a7-4327-8fde-0b7a9927bb4e"
Jul 14 13:54:14.692062 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:14.692007    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-65w26" event={"ID":"b23d0d9c-a401-4327-9167-db15cbf7095e","Type":"ContainerStarted","Data":"253df59870678a1efa2c3b3b21997baca43bb435f331a76992365bdca18c8597"}
Jul 14 13:54:14.694102 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:14.694070    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jfpgf" event={"ID":"5a5c1fc8-edd8-4cb9-8afb-81970bbd10c6","Type":"ContainerStarted","Data":"bae03f8d0a110a09f6df176812517547bc7cfd8ea9ff7bd1f06c5df495bfbd54"}
Jul 14 13:54:14.696064 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:14.696030    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tfd72" event={"ID":"e589fbdd-065a-4025-8862-43ca4000fa4c","Type":"ContainerStarted","Data":"8ecc80acbff6618ccfe0566906f9ffb7703f2399ab3dca61ca0798fc07942953"}
Jul 14 13:54:14.702794 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:14.702743    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r5kpb" event={"ID":"7a539262-9b04-4fbf-ae2b-f922f910e471","Type":"ContainerStarted","Data":"1a62b40fe3aebc451b6e8fd36be26bdfebb60a78d01f47e005cc33476efc3f50"}
Jul 14 13:54:14.709684 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:14.709645    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qhw7d" event={"ID":"c53cb4b0-7a90-4dba-b53d-df505d61bf19","Type":"ContainerStarted","Data":"dd261f344b29eb1420094b31c46ea0df50519fa4cb634158bf43dd90565d57db"}
Jul 14 13:54:14.711288 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:14.711212    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jbr68" event={"ID":"9e9684f1-f646-4a49-a4c8-23d51e4a94e3","Type":"ContainerStarted","Data":"9f5dea97c70c6fa3dd0f1846ebb6b98f2a02019300e52f66d29ac8280ce10903"}
Jul 14 13:54:14.713722 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:14.713652    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9pk9w" event={"ID":"a9b528fa-6f75-402e-8948-190262b8325a","Type":"ContainerStarted","Data":"9d86d5562dc920512c603b34f7870e1535a20b1968bb4b10dde510737b226c9c"}
Jul 14 13:54:14.716881 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:14.716852    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt" event={"ID":"5600e3e1-e775-482e-bc72-76d732552705","Type":"ContainerStarted","Data":"d8d8373020d641081da0750067d9a728e11050200e6cbbc9133f1b46959875df"}
Jul 14 13:54:14.719415 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:14.719388    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5pjcr" event={"ID":"8a9c4e5c-fb07-4089-9d9c-f01896cd6e25","Type":"ContainerStarted","Data":"3313bd0b897e9e6e580135250224e65ed4b32dba9792091680f38ad052ff8714"}
Jul 14 13:54:14.722982 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:14.722954    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-127.ec2.internal" event={"ID":"8c0fbe03dcb71398a547458685400965","Type":"ContainerStarted","Data":"2e55463995f1a72ba27c883ffc115a5c7904cfdc6755523583dfbc1cd2413818"}
Jul 14 13:54:14.725941 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:14.725910    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-127.ec2.internal" event={"ID":"cbb95fe8aa83faa678c273911aee96f3","Type":"ContainerStarted","Data":"f77982809d473b89b045b457d6974457c3d63fc47e60ab00d1d8d449c76a56da"}
Jul 14 13:54:15.127409 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:15.127368    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/02b53875-f2af-42e5-a311-74b8301e8b62-original-pull-secret\") pod \"global-pull-secret-syncer-5m6xn\" (UID: \"02b53875-f2af-42e5-a311-74b8301e8b62\") " pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:54:15.127619 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:15.127512    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 13:54:15.127619 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:15.127577    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02b53875-f2af-42e5-a311-74b8301e8b62-original-pull-secret podName:02b53875-f2af-42e5-a311-74b8301e8b62 nodeName:}" failed. No retries permitted until 2026-07-14 13:54:17.1275582 +0000 UTC m=+4.977995688 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/02b53875-f2af-42e5-a311-74b8301e8b62-original-pull-secret") pod "global-pull-secret-syncer-5m6xn" (UID: "02b53875-f2af-42e5-a311-74b8301e8b62") : object "kube-system"/"original-pull-secret" not registered
Jul 14 13:54:15.228245 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:15.228176    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/188bee4a-9fbe-4f95-8829-a91ab584ce4e-metrics-certs\") pod \"network-metrics-daemon-486sw\" (UID: \"188bee4a-9fbe-4f95-8829-a91ab584ce4e\") " pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:54:15.228432 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:15.228334    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 13:54:15.228503 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:15.228441    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/188bee4a-9fbe-4f95-8829-a91ab584ce4e-metrics-certs podName:188bee4a-9fbe-4f95-8829-a91ab584ce4e nodeName:}" failed. No retries permitted until 2026-07-14 13:54:17.228384803 +0000 UTC m=+5.078822290 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/188bee4a-9fbe-4f95-8829-a91ab584ce4e-metrics-certs") pod "network-metrics-daemon-486sw" (UID: "188bee4a-9fbe-4f95-8829-a91ab584ce4e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 13:54:15.329286 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:15.329246    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f274l\" (UniqueName: \"kubernetes.io/projected/421bfc78-19a7-4327-8fde-0b7a9927bb4e-kube-api-access-f274l\") pod \"network-check-target-22z2l\" (UID: \"421bfc78-19a7-4327-8fde-0b7a9927bb4e\") " pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:54:15.329482 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:15.329418    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 13:54:15.329482 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:15.329439    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 13:54:15.329482 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:15.329453    2576 projected.go:196] Error preparing data for projected volume kube-api-access-f274l for pod openshift-network-diagnostics/network-check-target-22z2l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 13:54:15.329662 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:15.329515    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/421bfc78-19a7-4327-8fde-0b7a9927bb4e-kube-api-access-f274l podName:421bfc78-19a7-4327-8fde-0b7a9927bb4e nodeName:}" failed. No retries permitted until 2026-07-14 13:54:17.329495408 +0000 UTC m=+5.179932900 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-f274l" (UniqueName: "kubernetes.io/projected/421bfc78-19a7-4327-8fde-0b7a9927bb4e-kube-api-access-f274l") pod "network-check-target-22z2l" (UID: "421bfc78-19a7-4327-8fde-0b7a9927bb4e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 13:54:15.560916 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:15.560821    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 13:49:13 +0000 UTC" deadline="2028-01-17 18:30:49.08701284 +0000 UTC"
Jul 14 13:54:15.560916 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:15.560887    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13252h36m33.526129808s"
Jul 14 13:54:15.683875 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:15.683847    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:54:15.684069 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:15.683982    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-486sw" podUID="188bee4a-9fbe-4f95-8829-a91ab584ce4e"
Jul 14 13:54:15.684397 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:15.684377    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:54:15.684518 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:15.684488    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5m6xn" podUID="02b53875-f2af-42e5-a311-74b8301e8b62"
Jul 14 13:54:16.683865 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:16.683639    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:54:16.684322 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:16.683976    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22z2l" podUID="421bfc78-19a7-4327-8fde-0b7a9927bb4e"
Jul 14 13:54:17.145079 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:17.144997    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/02b53875-f2af-42e5-a311-74b8301e8b62-original-pull-secret\") pod \"global-pull-secret-syncer-5m6xn\" (UID: \"02b53875-f2af-42e5-a311-74b8301e8b62\") " pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:54:17.145312 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:17.145175    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 13:54:17.145312 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:17.145242    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02b53875-f2af-42e5-a311-74b8301e8b62-original-pull-secret podName:02b53875-f2af-42e5-a311-74b8301e8b62 nodeName:}" failed. No retries permitted until 2026-07-14 13:54:21.145221663 +0000 UTC m=+8.995659150 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/02b53875-f2af-42e5-a311-74b8301e8b62-original-pull-secret") pod "global-pull-secret-syncer-5m6xn" (UID: "02b53875-f2af-42e5-a311-74b8301e8b62") : object "kube-system"/"original-pull-secret" not registered
Jul 14 13:54:17.245628 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:17.245570    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/188bee4a-9fbe-4f95-8829-a91ab584ce4e-metrics-certs\") pod \"network-metrics-daemon-486sw\" (UID: \"188bee4a-9fbe-4f95-8829-a91ab584ce4e\") " pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:54:17.245836 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:17.245741    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 13:54:17.245836 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:17.245810    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/188bee4a-9fbe-4f95-8829-a91ab584ce4e-metrics-certs podName:188bee4a-9fbe-4f95-8829-a91ab584ce4e nodeName:}" failed. No retries permitted until 2026-07-14 13:54:21.245789345 +0000 UTC m=+9.096226829 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/188bee4a-9fbe-4f95-8829-a91ab584ce4e-metrics-certs") pod "network-metrics-daemon-486sw" (UID: "188bee4a-9fbe-4f95-8829-a91ab584ce4e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 13:54:17.346799 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:17.346756    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f274l\" (UniqueName: \"kubernetes.io/projected/421bfc78-19a7-4327-8fde-0b7a9927bb4e-kube-api-access-f274l\") pod \"network-check-target-22z2l\" (UID: \"421bfc78-19a7-4327-8fde-0b7a9927bb4e\") " pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:54:17.347459 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:17.346997    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 13:54:17.347459 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:17.347027    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 13:54:17.347459 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:17.347041    2576 projected.go:196] Error preparing data for projected volume kube-api-access-f274l for pod openshift-network-diagnostics/network-check-target-22z2l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 13:54:17.347459 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:17.347108    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/421bfc78-19a7-4327-8fde-0b7a9927bb4e-kube-api-access-f274l podName:421bfc78-19a7-4327-8fde-0b7a9927bb4e nodeName:}" failed. No retries permitted until 2026-07-14 13:54:21.347088851 +0000 UTC m=+9.197526357 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-f274l" (UniqueName: "kubernetes.io/projected/421bfc78-19a7-4327-8fde-0b7a9927bb4e-kube-api-access-f274l") pod "network-check-target-22z2l" (UID: "421bfc78-19a7-4327-8fde-0b7a9927bb4e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 13:54:17.683781 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:17.683744    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:54:17.683973 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:17.683910    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-486sw" podUID="188bee4a-9fbe-4f95-8829-a91ab584ce4e"
Jul 14 13:54:17.684357 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:17.684314    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:54:17.684502 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:17.684409    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5m6xn" podUID="02b53875-f2af-42e5-a311-74b8301e8b62"
Jul 14 13:54:18.684632 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:18.683796    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:54:18.684632 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:18.683935    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22z2l" podUID="421bfc78-19a7-4327-8fde-0b7a9927bb4e"
Jul 14 13:54:19.683347 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:19.683312    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:54:19.683536 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:19.683445    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-486sw" podUID="188bee4a-9fbe-4f95-8829-a91ab584ce4e"
Jul 14 13:54:19.683829 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:19.683312    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:54:19.683943 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:19.683924    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5m6xn" podUID="02b53875-f2af-42e5-a311-74b8301e8b62"
Jul 14 13:54:20.683426 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:20.683352    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:54:20.683875 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:20.683496    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22z2l" podUID="421bfc78-19a7-4327-8fde-0b7a9927bb4e"
Jul 14 13:54:21.181706 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:21.181156    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/02b53875-f2af-42e5-a311-74b8301e8b62-original-pull-secret\") pod \"global-pull-secret-syncer-5m6xn\" (UID: \"02b53875-f2af-42e5-a311-74b8301e8b62\") " pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:54:21.181706 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:21.181285    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 13:54:21.181706 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:21.181359    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02b53875-f2af-42e5-a311-74b8301e8b62-original-pull-secret podName:02b53875-f2af-42e5-a311-74b8301e8b62 nodeName:}" failed. No retries permitted until 2026-07-14 13:54:29.181337681 +0000 UTC m=+17.031775173 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/02b53875-f2af-42e5-a311-74b8301e8b62-original-pull-secret") pod "global-pull-secret-syncer-5m6xn" (UID: "02b53875-f2af-42e5-a311-74b8301e8b62") : object "kube-system"/"original-pull-secret" not registered
Jul 14 13:54:21.282919 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:21.282417    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/188bee4a-9fbe-4f95-8829-a91ab584ce4e-metrics-certs\") pod \"network-metrics-daemon-486sw\" (UID: \"188bee4a-9fbe-4f95-8829-a91ab584ce4e\") " pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:54:21.282919 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:21.282533    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 13:54:21.282919 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:21.282604    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/188bee4a-9fbe-4f95-8829-a91ab584ce4e-metrics-certs podName:188bee4a-9fbe-4f95-8829-a91ab584ce4e nodeName:}" failed. No retries permitted until 2026-07-14 13:54:29.282574651 +0000 UTC m=+17.133012132 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/188bee4a-9fbe-4f95-8829-a91ab584ce4e-metrics-certs") pod "network-metrics-daemon-486sw" (UID: "188bee4a-9fbe-4f95-8829-a91ab584ce4e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 13:54:21.383073 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:21.383040    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f274l\" (UniqueName: \"kubernetes.io/projected/421bfc78-19a7-4327-8fde-0b7a9927bb4e-kube-api-access-f274l\") pod \"network-check-target-22z2l\" (UID: \"421bfc78-19a7-4327-8fde-0b7a9927bb4e\") " pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:54:21.383272 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:21.383234    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 13:54:21.383272 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:21.383262    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 13:54:21.383272 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:21.383274    2576 projected.go:196] Error preparing data for projected volume kube-api-access-f274l for pod openshift-network-diagnostics/network-check-target-22z2l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 13:54:21.383423 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:21.383338    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/421bfc78-19a7-4327-8fde-0b7a9927bb4e-kube-api-access-f274l podName:421bfc78-19a7-4327-8fde-0b7a9927bb4e nodeName:}" failed. No retries permitted until 2026-07-14 13:54:29.38331934 +0000 UTC m=+17.233756825 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-f274l" (UniqueName: "kubernetes.io/projected/421bfc78-19a7-4327-8fde-0b7a9927bb4e-kube-api-access-f274l") pod "network-check-target-22z2l" (UID: "421bfc78-19a7-4327-8fde-0b7a9927bb4e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 13:54:21.683823 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:21.683787    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:54:21.684315 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:21.683930    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-486sw" podUID="188bee4a-9fbe-4f95-8829-a91ab584ce4e"
Jul 14 13:54:21.684315 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:21.684112    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:54:21.684315 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:21.684252    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5m6xn" podUID="02b53875-f2af-42e5-a311-74b8301e8b62"
Jul 14 13:54:22.685286 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:22.684833    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:54:22.685286 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:22.684948    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22z2l" podUID="421bfc78-19a7-4327-8fde-0b7a9927bb4e"
Jul 14 13:54:23.684088 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:23.684029    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:54:23.684282 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:23.684043    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:54:23.684282 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:23.684176    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5m6xn" podUID="02b53875-f2af-42e5-a311-74b8301e8b62"
Jul 14 13:54:23.684510 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:23.684304    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-486sw" podUID="188bee4a-9fbe-4f95-8829-a91ab584ce4e"
Jul 14 13:54:24.684133 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:24.684095    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:54:24.684521 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:24.684230    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22z2l" podUID="421bfc78-19a7-4327-8fde-0b7a9927bb4e"
Jul 14 13:54:25.683177 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:25.683141    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:54:25.683397 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:25.683141    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:54:25.683397 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:25.683280    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-486sw" podUID="188bee4a-9fbe-4f95-8829-a91ab584ce4e"
Jul 14 13:54:25.683397 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:25.683346    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5m6xn" podUID="02b53875-f2af-42e5-a311-74b8301e8b62"
Jul 14 13:54:26.683652 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:26.683558    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:54:26.684080 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:26.683694    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22z2l" podUID="421bfc78-19a7-4327-8fde-0b7a9927bb4e"
Jul 14 13:54:27.683848 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:27.683811    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:54:27.684323 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:27.683948    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-486sw" podUID="188bee4a-9fbe-4f95-8829-a91ab584ce4e"
Jul 14 13:54:27.684323 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:27.683996    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:54:27.684323 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:27.684096    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5m6xn" podUID="02b53875-f2af-42e5-a311-74b8301e8b62"
Jul 14 13:54:28.683532 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:28.683480    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:54:28.683743 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:28.683664    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22z2l" podUID="421bfc78-19a7-4327-8fde-0b7a9927bb4e"
Jul 14 13:54:29.246179 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:29.246138    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/02b53875-f2af-42e5-a311-74b8301e8b62-original-pull-secret\") pod \"global-pull-secret-syncer-5m6xn\" (UID: \"02b53875-f2af-42e5-a311-74b8301e8b62\") " pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:54:29.246711 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:29.246309    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 13:54:29.246711 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:29.246401    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02b53875-f2af-42e5-a311-74b8301e8b62-original-pull-secret podName:02b53875-f2af-42e5-a311-74b8301e8b62 nodeName:}" failed. No retries permitted until 2026-07-14 13:54:45.246377715 +0000 UTC m=+33.096815216 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/02b53875-f2af-42e5-a311-74b8301e8b62-original-pull-secret") pod "global-pull-secret-syncer-5m6xn" (UID: "02b53875-f2af-42e5-a311-74b8301e8b62") : object "kube-system"/"original-pull-secret" not registered
Jul 14 13:54:29.346634 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:29.346582    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/188bee4a-9fbe-4f95-8829-a91ab584ce4e-metrics-certs\") pod \"network-metrics-daemon-486sw\" (UID: \"188bee4a-9fbe-4f95-8829-a91ab584ce4e\") " pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:54:29.346825 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:29.346755    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 13:54:29.346891 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:29.346838    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/188bee4a-9fbe-4f95-8829-a91ab584ce4e-metrics-certs podName:188bee4a-9fbe-4f95-8829-a91ab584ce4e nodeName:}" failed. No retries permitted until 2026-07-14 13:54:45.346816077 +0000 UTC m=+33.197253575 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/188bee4a-9fbe-4f95-8829-a91ab584ce4e-metrics-certs") pod "network-metrics-daemon-486sw" (UID: "188bee4a-9fbe-4f95-8829-a91ab584ce4e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 13:54:29.447209 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:29.447164    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f274l\" (UniqueName: \"kubernetes.io/projected/421bfc78-19a7-4327-8fde-0b7a9927bb4e-kube-api-access-f274l\") pod \"network-check-target-22z2l\" (UID: \"421bfc78-19a7-4327-8fde-0b7a9927bb4e\") " pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:54:29.447381 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:29.447314    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 13:54:29.447381 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:29.447333    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 13:54:29.447381 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:29.447342    2576 projected.go:196] Error preparing data for projected volume kube-api-access-f274l for pod openshift-network-diagnostics/network-check-target-22z2l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 13:54:29.447519 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:29.447406    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/421bfc78-19a7-4327-8fde-0b7a9927bb4e-kube-api-access-f274l podName:421bfc78-19a7-4327-8fde-0b7a9927bb4e nodeName:}" failed. No retries permitted until 2026-07-14 13:54:45.447388638 +0000 UTC m=+33.297826118 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-f274l" (UniqueName: "kubernetes.io/projected/421bfc78-19a7-4327-8fde-0b7a9927bb4e-kube-api-access-f274l") pod "network-check-target-22z2l" (UID: "421bfc78-19a7-4327-8fde-0b7a9927bb4e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 13:54:29.683981 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:29.683940    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:54:29.684179 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:29.683984    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:54:29.684179 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:29.684077    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-486sw" podUID="188bee4a-9fbe-4f95-8829-a91ab584ce4e"
Jul 14 13:54:29.684301 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:29.684207    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5m6xn" podUID="02b53875-f2af-42e5-a311-74b8301e8b62"
Jul 14 13:54:30.684088 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:30.684044    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:54:30.684554 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:30.684152    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22z2l" podUID="421bfc78-19a7-4327-8fde-0b7a9927bb4e"
Jul 14 13:54:31.683317 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:31.683273    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:54:31.683496 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:31.683285    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:54:31.683496 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:31.683421    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-486sw" podUID="188bee4a-9fbe-4f95-8829-a91ab584ce4e"
Jul 14 13:54:31.683577 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:31.683497    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5m6xn" podUID="02b53875-f2af-42e5-a311-74b8301e8b62"
Jul 14 13:54:32.686092 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:32.686060    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:54:32.686493 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:32.686178    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22z2l" podUID="421bfc78-19a7-4327-8fde-0b7a9927bb4e"
Jul 14 13:54:33.683220 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:33.683186    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:54:33.683405 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:33.683233    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:54:33.683405 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:33.683331    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-486sw" podUID="188bee4a-9fbe-4f95-8829-a91ab584ce4e"
Jul 14 13:54:33.683529 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:33.683436    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5m6xn" podUID="02b53875-f2af-42e5-a311-74b8301e8b62"
Jul 14 13:54:34.683564 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:34.683530    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:54:34.683944 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:34.683655    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22z2l" podUID="421bfc78-19a7-4327-8fde-0b7a9927bb4e"
Jul 14 13:54:35.683484 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:35.683266    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:54:35.683641 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:35.683279    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:54:35.684078 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:35.683673    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-486sw" podUID="188bee4a-9fbe-4f95-8829-a91ab584ce4e"
Jul 14 13:54:35.684078 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:35.683784    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5m6xn" podUID="02b53875-f2af-42e5-a311-74b8301e8b62"
Jul 14 13:54:35.783340 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:35.783102    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jfpgf" event={"ID":"5a5c1fc8-edd8-4cb9-8afb-81970bbd10c6","Type":"ContainerStarted","Data":"e68dd8de62eaade3a7a233f6124061a8d150a6cf2692a5d18bc1a3ae242819db"}
Jul 14 13:54:35.785472 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:35.785378    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tfd72" event={"ID":"e589fbdd-065a-4025-8862-43ca4000fa4c","Type":"ContainerStarted","Data":"cbfe48cb52de424a9529933858bfdb5b1c7846355eab6533f12ef6ec37a2c6d3"}
Jul 14 13:54:35.787114 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:35.787089    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r5kpb" event={"ID":"7a539262-9b04-4fbf-ae2b-f922f910e471","Type":"ContainerStarted","Data":"052d8f906d0c11b7131d5c56caaffed22334d50ec3bb7cbc81e4f8a11235a81d"}
Jul 14 13:54:35.788569 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:35.788541    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qhw7d" event={"ID":"c53cb4b0-7a90-4dba-b53d-df505d61bf19","Type":"ContainerStarted","Data":"ecb38490d3af2e34f217d76915ccca550e39f5231ae4bd6d20f0cd18ce816c8e"}
Jul 14 13:54:35.790071 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:35.790020    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jbr68" event={"ID":"9e9684f1-f646-4a49-a4c8-23d51e4a94e3","Type":"ContainerStarted","Data":"0a5d38316e53c717021cb586baec698d3843dc613bc32419765fe31184175569"}
Jul 14 13:54:35.792416 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:35.792395    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sh6rt_5600e3e1-e775-482e-bc72-76d732552705/ovn-acl-logging/0.log"
Jul 14 13:54:35.792794 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:35.792741    2576 generic.go:356] "Generic (PLEG): container finished" podID="5600e3e1-e775-482e-bc72-76d732552705" containerID="05b5675674e4235f0861797f6bd625dd578f5c4445ffe5f311cf8eb8625d6169" exitCode=1
Jul 14 13:54:35.792861 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:35.792802    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt" event={"ID":"5600e3e1-e775-482e-bc72-76d732552705","Type":"ContainerStarted","Data":"026b65371b6663988b8fdce1e7e724ca924f5e4a0a4bce5d1cfdfcf0c6fcf3bb"}
Jul 14 13:54:35.792917 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:35.792860    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt" event={"ID":"5600e3e1-e775-482e-bc72-76d732552705","Type":"ContainerStarted","Data":"b35c9d87bda81a721aebc99c2da5be280e7d9627f4e0b1fabb1202cfb39d46e2"}
Jul 14 13:54:35.792917 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:35.792893    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt" event={"ID":"5600e3e1-e775-482e-bc72-76d732552705","Type":"ContainerStarted","Data":"28539b863b22a2532bc153ac98352cad7f5a0ca3f02bf14c1ed8ddb9ae953472"}
Jul 14 13:54:35.793018 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:35.792926    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt" event={"ID":"5600e3e1-e775-482e-bc72-76d732552705","Type":"ContainerDied","Data":"05b5675674e4235f0861797f6bd625dd578f5c4445ffe5f311cf8eb8625d6169"}
Jul 14 13:54:35.793018 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:35.792964    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt" event={"ID":"5600e3e1-e775-482e-bc72-76d732552705","Type":"ContainerStarted","Data":"35fb8b056dd23999c8709f2b9f88d865e4ca82c50411e6761599c58a109456f6"}
Jul 14 13:54:35.794319 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:35.794294    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5pjcr" event={"ID":"8a9c4e5c-fb07-4089-9d9c-f01896cd6e25","Type":"ContainerStarted","Data":"30ed865d426b1ae8dda941e9500c229d7fb4119fcde0c8742d0b69c829582ad2"}
Jul 14 13:54:35.795806 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:35.795769    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-127.ec2.internal" event={"ID":"8c0fbe03dcb71398a547458685400965","Type":"ContainerStarted","Data":"41bc82ef4b8e624f5d20bccae18949ef6c29208230b0a08136f8c4b516403c14"}
Jul 14 13:54:35.797322 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:35.797280    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-127.ec2.internal" event={"ID":"cbb95fe8aa83faa678c273911aee96f3","Type":"ContainerStarted","Data":"89ab7ab7345cfa65a75d2d721bf61061ea8c06590447dc2b5604280be48074ab"}
Jul 14 13:54:35.798572 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:35.798552    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-65w26" event={"ID":"b23d0d9c-a401-4327-9167-db15cbf7095e","Type":"ContainerStarted","Data":"d66d9abe890e6fec6c5e9ec080f606e0039b6d3126d1a36ddb10d060e2499031"}
Jul 14 13:54:35.799385 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:35.799344    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-jfpgf" podStartSLOduration=3.530635138 podStartE2EDuration="23.799332113s" totalImagesPullingTime="20.268696975s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:54:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:54:35.799165594 +0000 UTC m=+23.649603099" watchObservedRunningTime="2026-07-14 13:54:35.799332113 +0000 UTC m=+23.649769640"
Jul 14 13:54:35.858483 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:35.858402    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-65w26" podStartSLOduration=3.823433081 podStartE2EDuration="23.858380688s" totalImagesPullingTime="20.034947607s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:54:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:54:35.858037493 +0000 UTC m=+23.708474997" watchObservedRunningTime="2026-07-14 13:54:35.858380688 +0000 UTC m=+23.708818193"
Jul 14 13:54:35.881392 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:35.881057    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-tfd72" podStartSLOduration=3.568333214 podStartE2EDuration="23.881037181s" totalImagesPullingTime="20.312703967s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:54:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:54:35.880301739 +0000 UTC m=+23.730739244" watchObservedRunningTime="2026-07-14 13:54:35.881037181 +0000 UTC m=+23.731474684"
Jul 14 13:54:35.900524 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:35.900477    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-jbr68" podStartSLOduration=3.351789929 podStartE2EDuration="23.900461934s" totalImagesPullingTime="20.548672005s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:54:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:54:35.899917724 +0000 UTC m=+23.750355230" watchObservedRunningTime="2026-07-14 13:54:35.900461934 +0000 UTC m=+23.750899437"
Jul 14 13:54:35.920056 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:35.920005    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-127.ec2.internal" podStartSLOduration=22.919987597 podStartE2EDuration="22.919987597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:54:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:54:35.919175511 +0000 UTC m=+23.769613017" watchObservedRunningTime="2026-07-14 13:54:35.919987597 +0000 UTC m=+23.770425099"
Jul 14 13:54:35.939237 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:35.939183    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5pjcr" podStartSLOduration=3.59719763 podStartE2EDuration="23.939164382s" totalImagesPullingTime="20.341966752s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:54:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:54:35.938765221 +0000 UTC m=+23.789202725" watchObservedRunningTime="2026-07-14 13:54:35.939164382 +0000 UTC m=+23.789601885"
Jul 14 13:54:36.074497 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:36.074461    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:36.222251 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:36.222212    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:36.683267 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:36.683239    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:54:36.683451 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:36.683348    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22z2l" podUID="421bfc78-19a7-4327-8fde-0b7a9927bb4e"
Jul 14 13:54:36.803064 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:36.803027    2576 generic.go:356] "Generic (PLEG): container finished" podID="c53cb4b0-7a90-4dba-b53d-df505d61bf19" containerID="ecb38490d3af2e34f217d76915ccca550e39f5231ae4bd6d20f0cd18ce816c8e" exitCode=0
Jul 14 13:54:36.803656 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:36.803116    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qhw7d" event={"ID":"c53cb4b0-7a90-4dba-b53d-df505d61bf19","Type":"ContainerDied","Data":"ecb38490d3af2e34f217d76915ccca550e39f5231ae4bd6d20f0cd18ce816c8e"}
Jul 14 13:54:36.804541 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:36.804519    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9pk9w" event={"ID":"a9b528fa-6f75-402e-8948-190262b8325a","Type":"ContainerStarted","Data":"3ad5ab6bf08112e11f2007fe2fcc204d78051eb51390aee65dbbf910a7e9b490"}
Jul 14 13:54:36.809797 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:36.809777    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sh6rt_5600e3e1-e775-482e-bc72-76d732552705/ovn-acl-logging/0.log"
Jul 14 13:54:36.810166 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:36.810145    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt" event={"ID":"5600e3e1-e775-482e-bc72-76d732552705","Type":"ContainerStarted","Data":"3ac1aeca5107aeb52853e4ece4d16afdbda710d4e1dddffc83171755375f02ae"}
Jul 14 13:54:36.811927 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:36.811900    2576 generic.go:356] "Generic (PLEG): container finished" podID="cbb95fe8aa83faa678c273911aee96f3" containerID="89ab7ab7345cfa65a75d2d721bf61061ea8c06590447dc2b5604280be48074ab" exitCode=0
Jul 14 13:54:36.812016 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:36.811957    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-127.ec2.internal" event={"ID":"cbb95fe8aa83faa678c273911aee96f3","Type":"ContainerDied","Data":"89ab7ab7345cfa65a75d2d721bf61061ea8c06590447dc2b5604280be48074ab"}
Jul 14 13:54:36.812016 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:36.811992    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-127.ec2.internal" event={"ID":"cbb95fe8aa83faa678c273911aee96f3","Type":"ContainerStarted","Data":"28e8fd6c4d635eab58d66e25c8a9cc473823186711fc9ba32a62ecd9aef2a01a"}
Jul 14 13:54:36.861953 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:36.861904    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-9pk9w" podStartSLOduration=4.336232848 podStartE2EDuration="24.861889203s" totalImagesPullingTime="20.525656355s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:54:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:54:36.861613735 +0000 UTC m=+24.712051229" watchObservedRunningTime="2026-07-14 13:54:36.861889203 +0000 UTC m=+24.712326743"
Jul 14 13:54:36.883304 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:36.883249    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-127.ec2.internal" podStartSLOduration=23.883233731 podStartE2EDuration="23.883233731s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:54:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:54:36.883012648 +0000 UTC m=+24.733450150" watchObservedRunningTime="2026-07-14 13:54:36.883233731 +0000 UTC m=+24.733671230"
Jul 14 13:54:36.908845 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:36.908820    2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 14 13:54:37.140502 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:37.140473    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:37.232031 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:37.232001    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:37.578050 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:37.577888    2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-14T13:54:36.908840458Z","UUID":"2c9d4df7-ce30-47c9-a9b4-060d3fbfcd16","Handler":null,"Name":"","Endpoint":""}
Jul 14 13:54:37.579931 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:37.579905    2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 14 13:54:37.580115 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:37.579945    2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 14 13:54:37.683689 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:37.683656    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:54:37.683689 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:37.683693    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:54:37.683908 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:37.683774    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-486sw" podUID="188bee4a-9fbe-4f95-8829-a91ab584ce4e"
Jul 14 13:54:37.683957 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:37.683909    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5m6xn" podUID="02b53875-f2af-42e5-a311-74b8301e8b62"
Jul 14 13:54:37.817120 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:37.817060    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r5kpb" event={"ID":"7a539262-9b04-4fbf-ae2b-f922f910e471","Type":"ContainerStarted","Data":"c626d8c2d0443744914136d9f3130473ab7fb41a72b244b4264e487af1ecddb9"}
Jul 14 13:54:38.258902 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:38.258651    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:38.359729 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:38.359691    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:38.463207 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:38.463176    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:38.683405 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:38.683325    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:54:38.683660 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:38.683448    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22z2l" podUID="421bfc78-19a7-4327-8fde-0b7a9927bb4e"
Jul 14 13:54:38.700090 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:38.700048    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:38.796375 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:38.796334    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:38.822356 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:38.822308    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r5kpb" event={"ID":"7a539262-9b04-4fbf-ae2b-f922f910e471","Type":"ContainerStarted","Data":"d9362e2830facd5f0f9a3964edd93259c2d2aeeb369e16adad075530b6caed28"}
Jul 14 13:54:38.825717 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:38.825695    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sh6rt_5600e3e1-e775-482e-bc72-76d732552705/ovn-acl-logging/0.log"
Jul 14 13:54:38.826093 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:38.826073    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt" event={"ID":"5600e3e1-e775-482e-bc72-76d732552705","Type":"ContainerStarted","Data":"96eaa804ac5c3907d826921415b49d763608608f63ae3db57d41d2f6791e59dc"}
Jul 14 13:54:38.850312 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:38.850260    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r5kpb" podStartSLOduration=4.063077998 podStartE2EDuration="26.850240962s" totalImagesPullingTime="22.787162964s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:54:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:54:38.849735847 +0000 UTC m=+26.700173350" watchObservedRunningTime="2026-07-14 13:54:38.850240962 +0000 UTC m=+26.700678467"
Jul 14 13:54:38.894257 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:38.894214    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:38.994941 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:38.994849    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:39.216181 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:39.216143    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:39.315467 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:39.315425    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:39.605666 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:39.605562    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jfpgf_5a5c1fc8-edd8-4cb9-8afb-81970bbd10c6/dns-node-resolver/0.log"
Jul 14 13:54:39.683648 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:39.683608    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:54:39.683825 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:39.683750    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-486sw" podUID="188bee4a-9fbe-4f95-8829-a91ab584ce4e"
Jul 14 13:54:39.683825 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:39.683790    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:54:39.683927 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:39.683865    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5m6xn" podUID="02b53875-f2af-42e5-a311-74b8301e8b62"
Jul 14 13:54:39.784985 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:39.784942    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:39.891586 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:39.891506    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:40.205340 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:40.205268    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5pjcr_8a9c4e5c-fb07-4089-9d9c-f01896cd6e25/node-ca/0.log"
Jul 14 13:54:40.662020 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:40.661985    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-65w26"
Jul 14 13:54:40.662785 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:40.662767    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-65w26"
Jul 14 13:54:40.684357 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:40.684319    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:54:40.684563 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:40.684450    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22z2l" podUID="421bfc78-19a7-4327-8fde-0b7a9927bb4e"
Jul 14 13:54:40.830252 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:40.830222    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-65w26"
Jul 14 13:54:40.830741 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:40.830721    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-65w26"
Jul 14 13:54:40.842327 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:40.842295    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:41.143284 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:41.143252    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:41.684023 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:41.683786    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:54:41.684023 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:41.683812    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:54:41.684246 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:41.684079    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-486sw" podUID="188bee4a-9fbe-4f95-8829-a91ab584ce4e"
Jul 14 13:54:41.684246 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:41.684118    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5m6xn" podUID="02b53875-f2af-42e5-a311-74b8301e8b62"
Jul 14 13:54:41.832452 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:41.832422    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:41.834489 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:41.834456    2576 generic.go:356] "Generic (PLEG): container finished" podID="c53cb4b0-7a90-4dba-b53d-df505d61bf19" containerID="243098d2bd4ab38021bf7710a1b093f71f5874b56b55c389e548d6a1701ab747" exitCode=0
Jul 14 13:54:41.834669 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:41.834569    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qhw7d" event={"ID":"c53cb4b0-7a90-4dba-b53d-df505d61bf19","Type":"ContainerDied","Data":"243098d2bd4ab38021bf7710a1b093f71f5874b56b55c389e548d6a1701ab747"}
Jul 14 13:54:41.838255 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:41.838237    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sh6rt_5600e3e1-e775-482e-bc72-76d732552705/ovn-acl-logging/0.log"
Jul 14 13:54:41.838651 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:41.838624    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt" event={"ID":"5600e3e1-e775-482e-bc72-76d732552705","Type":"ContainerStarted","Data":"b811fd84eaf0ad29a4a8c6dc3f628d537cafc26736dca72958301832e11fc74f"}
Jul 14 13:54:41.839109 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:41.839091    2576 scope.go:117] "RemoveContainer" containerID="05b5675674e4235f0861797f6bd625dd578f5c4445ffe5f311cf8eb8625d6169"
Jul 14 13:54:42.684367 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:42.684333    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:54:42.684828 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:42.684418    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22z2l" podUID="421bfc78-19a7-4327-8fde-0b7a9927bb4e"
Jul 14 13:54:42.828264 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:42.828226    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:42.844373 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:42.844339    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sh6rt_5600e3e1-e775-482e-bc72-76d732552705/ovn-acl-logging/0.log"
Jul 14 13:54:42.844792 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:42.844767    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt" event={"ID":"5600e3e1-e775-482e-bc72-76d732552705","Type":"ContainerStarted","Data":"9609a0d3c9ff710505f050ed72757d065328762ba3833a602edc1a116f4b5195"}
Jul 14 13:54:42.844901 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:42.844885    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:42.844961 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:42.844951    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:42.845177 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:42.845140    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:42.860859 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:42.860830    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:42.861008 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:42.860942    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:54:42.880046 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:42.879994    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt" podStartSLOduration=9.840062987 podStartE2EDuration="30.879976988s" totalImagesPullingTime="21.039914001s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:54:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:54:42.878581041 +0000 UTC m=+30.729018538" watchObservedRunningTime="2026-07-14 13:54:42.879976988 +0000 UTC m=+30.730414490"
Jul 14 13:54:43.021496 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:43.021453    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:43.119299 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:43.119260    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:43.234929 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:43.234829    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:43.324706 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:43.324671    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:43.544383 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:43.544339    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:43.683478 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:43.683441    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:54:43.683687 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:43.683549    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-486sw" podUID="188bee4a-9fbe-4f95-8829-a91ab584ce4e"
Jul 14 13:54:43.683687 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:43.683569    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:54:43.683803 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:43.683696    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5m6xn" podUID="02b53875-f2af-42e5-a311-74b8301e8b62"
Jul 14 13:54:43.848920 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:43.848838    2576 generic.go:356] "Generic (PLEG): container finished" podID="c53cb4b0-7a90-4dba-b53d-df505d61bf19" containerID="6df53b98ea75134283a96ebb5157ed03f889147ba11db405c8a31aace9631920" exitCode=0
Jul 14 13:54:43.849285 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:43.848913    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qhw7d" event={"ID":"c53cb4b0-7a90-4dba-b53d-df505d61bf19","Type":"ContainerDied","Data":"6df53b98ea75134283a96ebb5157ed03f889147ba11db405c8a31aace9631920"}
Jul 14 13:54:44.265188 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:44.265102    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:44.532106 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:44.532075    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:44.683844 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:44.683808    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:54:44.684021 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:44.683937    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22z2l" podUID="421bfc78-19a7-4327-8fde-0b7a9927bb4e"
Jul 14 13:54:44.811074 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:44.810984    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:44.930727 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:44.930685    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:45.133441 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:45.133403    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:45.262544 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:45.262509    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/02b53875-f2af-42e5-a311-74b8301e8b62-original-pull-secret\") pod \"global-pull-secret-syncer-5m6xn\" (UID: \"02b53875-f2af-42e5-a311-74b8301e8b62\") " pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:54:45.262738 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:45.262667    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 13:54:45.262782 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:45.262737    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02b53875-f2af-42e5-a311-74b8301e8b62-original-pull-secret podName:02b53875-f2af-42e5-a311-74b8301e8b62 nodeName:}" failed. No retries permitted until 2026-07-14 13:55:17.262720289 +0000 UTC m=+65.113157774 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/02b53875-f2af-42e5-a311-74b8301e8b62-original-pull-secret") pod "global-pull-secret-syncer-5m6xn" (UID: "02b53875-f2af-42e5-a311-74b8301e8b62") : object "kube-system"/"original-pull-secret" not registered
Jul 14 13:54:45.343995 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:45.343903    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:45.362886 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:45.362848    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/188bee4a-9fbe-4f95-8829-a91ab584ce4e-metrics-certs\") pod \"network-metrics-daemon-486sw\" (UID: \"188bee4a-9fbe-4f95-8829-a91ab584ce4e\") " pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:54:45.363020 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:45.363002    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 13:54:45.363078 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:45.363069    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/188bee4a-9fbe-4f95-8829-a91ab584ce4e-metrics-certs podName:188bee4a-9fbe-4f95-8829-a91ab584ce4e nodeName:}" failed. No retries permitted until 2026-07-14 13:55:17.363052634 +0000 UTC m=+65.213490116 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/188bee4a-9fbe-4f95-8829-a91ab584ce4e-metrics-certs") pod "network-metrics-daemon-486sw" (UID: "188bee4a-9fbe-4f95-8829-a91ab584ce4e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 13:54:45.463788 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:45.463757    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f274l\" (UniqueName: \"kubernetes.io/projected/421bfc78-19a7-4327-8fde-0b7a9927bb4e-kube-api-access-f274l\") pod \"network-check-target-22z2l\" (UID: \"421bfc78-19a7-4327-8fde-0b7a9927bb4e\") " pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:54:45.463967 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:45.463946    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 13:54:45.464010 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:45.463972    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 13:54:45.464010 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:45.463984    2576 projected.go:196] Error preparing data for projected volume kube-api-access-f274l for pod openshift-network-diagnostics/network-check-target-22z2l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 13:54:45.464078 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:45.464041    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/421bfc78-19a7-4327-8fde-0b7a9927bb4e-kube-api-access-f274l podName:421bfc78-19a7-4327-8fde-0b7a9927bb4e nodeName:}" failed. No retries permitted until 2026-07-14 13:55:17.464025261 +0000 UTC m=+65.314462742 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-f274l" (UniqueName: "kubernetes.io/projected/421bfc78-19a7-4327-8fde-0b7a9927bb4e-kube-api-access-f274l") pod "network-check-target-22z2l" (UID: "421bfc78-19a7-4327-8fde-0b7a9927bb4e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 13:54:45.683270 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:45.683190    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:54:45.683270 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:45.683230    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:54:45.683438 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:45.683306    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5m6xn" podUID="02b53875-f2af-42e5-a311-74b8301e8b62"
Jul 14 13:54:45.683471 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:45.683439    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-486sw" podUID="188bee4a-9fbe-4f95-8829-a91ab584ce4e"
Jul 14 13:54:45.751652 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:45.751621    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:45.859258 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:45.859223    2576 generic.go:356] "Generic (PLEG): container finished" podID="c53cb4b0-7a90-4dba-b53d-df505d61bf19" containerID="d769d8beb947e7aa65d9a8ff31637df4296f42fe73efad12af90d3b144c2f0df" exitCode=0
Jul 14 13:54:45.859462 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:45.859277    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qhw7d" event={"ID":"c53cb4b0-7a90-4dba-b53d-df505d61bf19","Type":"ContainerDied","Data":"d769d8beb947e7aa65d9a8ff31637df4296f42fe73efad12af90d3b144c2f0df"}
Jul 14 13:54:46.540135 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:46.539908    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:46.632869 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:46.632814    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:46.687609 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:46.687569    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:54:46.687795 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:46.687685    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22z2l" podUID="421bfc78-19a7-4327-8fde-0b7a9927bb4e"
Jul 14 13:54:46.734929 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:46.734893    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:47.136969 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:47.136929    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:47.324119 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:47.324078    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:47.425152 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:47.425067    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:47.683288 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:47.683205    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:54:47.683288 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:47.683227    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:54:47.683775 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:47.683352    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-486sw" podUID="188bee4a-9fbe-4f95-8829-a91ab584ce4e"
Jul 14 13:54:47.683775 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:47.683483    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5m6xn" podUID="02b53875-f2af-42e5-a311-74b8301e8b62"
Jul 14 13:54:47.932817 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:47.932774    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:48.326329 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:48.326293    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:48.436575 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:48.436533    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:48.546659 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:48.546619    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:48.607360 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:48.607267    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:48.684051 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:48.684011    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:54:48.684511 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:48.684161    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22z2l" podUID="421bfc78-19a7-4327-8fde-0b7a9927bb4e"
Jul 14 13:54:49.497431 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:49.497379    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:49.597412 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:49.597308    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:49.683335 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:49.683307    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:54:49.683487 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:49.683405    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-486sw" podUID="188bee4a-9fbe-4f95-8829-a91ab584ce4e"
Jul 14 13:54:49.683487 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:49.683462    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:54:49.683586 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:49.683565    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5m6xn" podUID="02b53875-f2af-42e5-a311-74b8301e8b62"
Jul 14 13:54:49.872286 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:49.872229    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qhw7d" event={"ID":"c53cb4b0-7a90-4dba-b53d-df505d61bf19","Type":"ContainerStarted","Data":"91d85c65972e6e4a3626ded183049c06932efc59b7d82511b95a0cafd3db9c99"}
Jul 14 13:54:49.971905 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:49.971873    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:50.136184 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:50.136147    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:50.542732 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:50.542694    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:50.683663 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:50.683628    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:54:50.683820 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:50.683741    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22z2l" podUID="421bfc78-19a7-4327-8fde-0b7a9927bb4e"
Jul 14 13:54:50.711377 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:50.711329    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:50.785194 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:50.785164    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:50.865079 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:50.864988    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:50.877394 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:50.877356    2576 generic.go:356] "Generic (PLEG): container finished" podID="c53cb4b0-7a90-4dba-b53d-df505d61bf19" containerID="91d85c65972e6e4a3626ded183049c06932efc59b7d82511b95a0cafd3db9c99" exitCode=0
Jul 14 13:54:50.877873 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:50.877414    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qhw7d" event={"ID":"c53cb4b0-7a90-4dba-b53d-df505d61bf19","Type":"ContainerDied","Data":"91d85c65972e6e4a3626ded183049c06932efc59b7d82511b95a0cafd3db9c99"}
Jul 14 13:54:51.001644 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:51.001608    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:51.135152 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:51.135063    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:51.233361 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:51.233320    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:51.536125 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:51.536094    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:51.636717 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:51.636670    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:51.683786 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:51.683749    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:54:51.683898 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:51.683792    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:54:51.683959 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:51.683907    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-486sw" podUID="188bee4a-9fbe-4f95-8829-a91ab584ce4e"
Jul 14 13:54:51.684040 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:51.684019    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5m6xn" podUID="02b53875-f2af-42e5-a311-74b8301e8b62"
Jul 14 13:54:51.882650 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:51.882543    2576 generic.go:356] "Generic (PLEG): container finished" podID="c53cb4b0-7a90-4dba-b53d-df505d61bf19" containerID="838d8e4fb2c29c418bd32690aa1dc84272046e955343c26cf58ae7836a3bdf45" exitCode=0
Jul 14 13:54:51.882650 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:51.882631    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qhw7d" event={"ID":"c53cb4b0-7a90-4dba-b53d-df505d61bf19","Type":"ContainerDied","Data":"838d8e4fb2c29c418bd32690aa1dc84272046e955343c26cf58ae7836a3bdf45"}
Jul 14 13:54:51.932910 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:51.932865    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:51.993020 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:51.992980    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:52.127530 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:52.127494    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:52.428390 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:52.428335    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:52.528733 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:52.528697    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:52.684515 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:52.684432    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:54:52.684676 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:52.684513    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22z2l" podUID="421bfc78-19a7-4327-8fde-0b7a9927bb4e"
Jul 14 13:54:52.783718 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:52.783665    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:52.888067 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:52.888031    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qhw7d" event={"ID":"c53cb4b0-7a90-4dba-b53d-df505d61bf19","Type":"ContainerStarted","Data":"ec6aaf0ac3bef044e0d67e55ba2c9cea04c3c20c619e8b5dc266b0305b82dbb7"}
Jul 14 13:54:52.915290 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:52.915241    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qhw7d" podStartSLOduration=9.801523011 podStartE2EDuration="40.915222529s" totalImagesPullingTime="31.113699518s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:54:12 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:54:52.913711159 +0000 UTC m=+40.764148663" watchObservedRunningTime="2026-07-14 13:54:52.915222529 +0000 UTC m=+40.765660032"
Jul 14 13:54:53.134816 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:53.134783    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:53.312869 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:53.312838    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:53.413800 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:53.413702    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:53.514831 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:53.514792    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:53.614510 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:53.614470    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:53.683517 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:53.683430    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:54:53.683693 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:53.683429    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:54:53.683693 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:53.683538    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5m6xn" podUID="02b53875-f2af-42e5-a311-74b8301e8b62"
Jul 14 13:54:53.683693 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:53.683639    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-486sw" podUID="188bee4a-9fbe-4f95-8829-a91ab584ce4e"
Jul 14 13:54:53.753205 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:53.753169    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:53.939822 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:53.939722    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:54.683286 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:54.683245    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:54:54.683446 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:54.683373    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22z2l" podUID="421bfc78-19a7-4327-8fde-0b7a9927bb4e"
Jul 14 13:54:55.297733 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:55.297696    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:55.369652 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:55.369610    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:55.683247 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:55.683164    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:54:55.683404 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:55.683265    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-486sw" podUID="188bee4a-9fbe-4f95-8829-a91ab584ce4e"
Jul 14 13:54:55.683404 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:55.683294    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:54:55.683404 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:55.683341    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5m6xn" podUID="02b53875-f2af-42e5-a311-74b8301e8b62"
Jul 14 13:54:56.683245 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:56.683208    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:54:56.683753 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:56.683311    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22z2l" podUID="421bfc78-19a7-4327-8fde-0b7a9927bb4e"
Jul 14 13:54:57.410142 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:57.410107    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:54:57.683451 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:57.683364    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:54:57.683832 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:57.683364    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:54:57.683832 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:57.683483    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5m6xn" podUID="02b53875-f2af-42e5-a311-74b8301e8b62"
Jul 14 13:54:57.683832 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:57.683549    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-486sw" podUID="188bee4a-9fbe-4f95-8829-a91ab584ce4e"
Jul 14 13:54:58.683814 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:58.683778    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:54:58.684211 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:58.683928    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22z2l" podUID="421bfc78-19a7-4327-8fde-0b7a9927bb4e"
Jul 14 13:54:59.683798 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:59.683758    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:54:59.684023 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:54:59.683758    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:54:59.684023 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:59.683868    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-486sw" podUID="188bee4a-9fbe-4f95-8829-a91ab584ce4e"
Jul 14 13:54:59.684023 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:54:59.683943    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5m6xn" podUID="02b53875-f2af-42e5-a311-74b8301e8b62"
Jul 14 13:55:00.684058 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:00.684019    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:55:00.684544 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:55:00.684122    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22z2l" podUID="421bfc78-19a7-4327-8fde-0b7a9927bb4e"
Jul 14 13:55:01.683788 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:01.683751    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:55:01.683970 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:01.683752    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:55:01.683970 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:55:01.683856    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-486sw" podUID="188bee4a-9fbe-4f95-8829-a91ab584ce4e"
Jul 14 13:55:01.683970 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:55:01.683918    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5m6xn" podUID="02b53875-f2af-42e5-a311-74b8301e8b62"
Jul 14 13:55:02.134921 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:02.134882    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:02.305169 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:02.305139    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:02.400475 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:02.400358    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:02.483350 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:02.483313    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:02.684535 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:02.684460    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:55:02.684683 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:55:02.684552    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22z2l" podUID="421bfc78-19a7-4327-8fde-0b7a9927bb4e"
Jul 14 13:55:03.158148 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:03.158109    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5m6xn"]
Jul 14 13:55:03.158909 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:03.158289    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:55:03.158909 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:55:03.158396    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5m6xn" podUID="02b53875-f2af-42e5-a311-74b8301e8b62"
Jul 14 13:55:03.163137 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:03.163082    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-22z2l"]
Jul 14 13:55:03.163532 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:03.163493    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:55:03.163689 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:55:03.163620    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22z2l" podUID="421bfc78-19a7-4327-8fde-0b7a9927bb4e"
Jul 14 13:55:03.163884 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:03.163790    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-486sw"]
Jul 14 13:55:03.163995 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:03.163913    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:55:03.164056 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:55:03.164023    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-486sw" podUID="188bee4a-9fbe-4f95-8829-a91ab584ce4e"
Jul 14 13:55:04.683680 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:04.683488    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:55:04.684169 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:04.683493    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:55:04.684169 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:55:04.683754    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5m6xn" podUID="02b53875-f2af-42e5-a311-74b8301e8b62"
Jul 14 13:55:04.684169 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:04.683522    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:55:04.684169 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:55:04.683865    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-486sw" podUID="188bee4a-9fbe-4f95-8829-a91ab584ce4e"
Jul 14 13:55:04.684169 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:55:04.683904    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22z2l" podUID="421bfc78-19a7-4327-8fde-0b7a9927bb4e"
Jul 14 13:55:06.683650 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:06.683576    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:55:06.684051 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:06.683577    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:55:06.684051 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:55:06.683753    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5m6xn" podUID="02b53875-f2af-42e5-a311-74b8301e8b62"
Jul 14 13:55:06.684051 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:06.683627    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:55:06.684051 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:55:06.683803    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-486sw" podUID="188bee4a-9fbe-4f95-8829-a91ab584ce4e"
Jul 14 13:55:06.684051 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:55:06.683894    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22z2l" podUID="421bfc78-19a7-4327-8fde-0b7a9927bb4e"
Jul 14 13:55:06.795339 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:06.795297    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:06.897205 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:06.897119    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:06.948642 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:06.948613    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-127.ec2.internal" event="NodeReady"
Jul 14 13:55:06.948837 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:06.948749    2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 14 13:55:06.987614 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:06.987560    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:07.003025 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.002991    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6d9fd7cf6c-pk2t8"]
Jul 14 13:55:07.004876 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.004854    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d9fd7cf6c-pk2t8"
Jul 14 13:55:07.007315 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.007282    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-d4hld\""
Jul 14 13:55:07.007480 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.007460    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 14 13:55:07.007664 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.007646    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 14 13:55:07.007762 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.007747    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 14 13:55:07.014361 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.014338    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 14 13:55:07.022773 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.022744    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d9fd7cf6c-pk2t8"]
Jul 14 13:55:07.085332 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.085301    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-vvtxj"]
Jul 14 13:55:07.086787 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.086772    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vvtxj"
Jul 14 13:55:07.089009 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.088988    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-q6rx4"]
Jul 14 13:55:07.089950 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.089913    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rvh4f\""
Jul 14 13:55:07.090062 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.090051    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 14 13:55:07.090462 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.090432    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 14 13:55:07.090462 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.090435    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-q6rx4"
Jul 14 13:55:07.090739 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.090720    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 14 13:55:07.093257 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.093223    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 14 13:55:07.093413 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.093296    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 14 13:55:07.093753 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.093730    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 14 13:55:07.093986 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.093968    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rr67m\""
Jul 14 13:55:07.094066 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.094008    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 14 13:55:07.098402 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.098352    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vvtxj"]
Jul 14 13:55:07.110846 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.110818    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-q6rx4"]
Jul 14 13:55:07.123336 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.123306    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/886aaf44-5bd7-4262-bda7-f622787aeb6c-registry-certificates\") pod \"image-registry-6d9fd7cf6c-pk2t8\" (UID: \"886aaf44-5bd7-4262-bda7-f622787aeb6c\") " pod="openshift-image-registry/image-registry-6d9fd7cf6c-pk2t8"
Jul 14 13:55:07.123506 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.123351    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/886aaf44-5bd7-4262-bda7-f622787aeb6c-installation-pull-secrets\") pod \"image-registry-6d9fd7cf6c-pk2t8\" (UID: \"886aaf44-5bd7-4262-bda7-f622787aeb6c\") " pod="openshift-image-registry/image-registry-6d9fd7cf6c-pk2t8"
Jul 14 13:55:07.123506 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.123368    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4bbb\" (UniqueName: \"kubernetes.io/projected/886aaf44-5bd7-4262-bda7-f622787aeb6c-kube-api-access-b4bbb\") pod \"image-registry-6d9fd7cf6c-pk2t8\" (UID: \"886aaf44-5bd7-4262-bda7-f622787aeb6c\") " pod="openshift-image-registry/image-registry-6d9fd7cf6c-pk2t8"
Jul 14 13:55:07.123506 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.123388    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/886aaf44-5bd7-4262-bda7-f622787aeb6c-image-registry-private-configuration\") pod \"image-registry-6d9fd7cf6c-pk2t8\" (UID: \"886aaf44-5bd7-4262-bda7-f622787aeb6c\") " pod="openshift-image-registry/image-registry-6d9fd7cf6c-pk2t8"
Jul 14 13:55:07.123506 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.123439    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/886aaf44-5bd7-4262-bda7-f622787aeb6c-ca-trust-extracted\") pod \"image-registry-6d9fd7cf6c-pk2t8\" (UID: \"886aaf44-5bd7-4262-bda7-f622787aeb6c\") " pod="openshift-image-registry/image-registry-6d9fd7cf6c-pk2t8"
Jul 14 13:55:07.123506 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.123500    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/886aaf44-5bd7-4262-bda7-f622787aeb6c-registry-tls\") pod \"image-registry-6d9fd7cf6c-pk2t8\" (UID: \"886aaf44-5bd7-4262-bda7-f622787aeb6c\") " pod="openshift-image-registry/image-registry-6d9fd7cf6c-pk2t8"
Jul 14 13:55:07.123706 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.123529    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/886aaf44-5bd7-4262-bda7-f622787aeb6c-bound-sa-token\") pod \"image-registry-6d9fd7cf6c-pk2t8\" (UID: \"886aaf44-5bd7-4262-bda7-f622787aeb6c\") " pod="openshift-image-registry/image-registry-6d9fd7cf6c-pk2t8"
Jul 14 13:55:07.123706 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.123578    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/886aaf44-5bd7-4262-bda7-f622787aeb6c-trusted-ca\") pod \"image-registry-6d9fd7cf6c-pk2t8\" (UID: \"886aaf44-5bd7-4262-bda7-f622787aeb6c\") " pod="openshift-image-registry/image-registry-6d9fd7cf6c-pk2t8"
Jul 14 13:55:07.188428 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.188352    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-hhmmt"]
Jul 14 13:55:07.189952 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.189937    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hhmmt"
Jul 14 13:55:07.192577 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.192530    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 14 13:55:07.192739 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.192723    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 14 13:55:07.192811 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.192794    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mh5f9\""
Jul 14 13:55:07.206396 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.206368    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hhmmt"]
Jul 14 13:55:07.224461 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.224424    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/886aaf44-5bd7-4262-bda7-f622787aeb6c-ca-trust-extracted\") pod \"image-registry-6d9fd7cf6c-pk2t8\" (UID: \"886aaf44-5bd7-4262-bda7-f622787aeb6c\") " pod="openshift-image-registry/image-registry-6d9fd7cf6c-pk2t8"
Jul 14 13:55:07.224461 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.224466    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f2371b29-9e56-469a-ac86-7ead875c7127-cert\") pod \"ingress-canary-vvtxj\" (UID: \"f2371b29-9e56-469a-ac86-7ead875c7127\") " pod="openshift-ingress-canary/ingress-canary-vvtxj"
Jul 14 13:55:07.224722 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.224501    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/886aaf44-5bd7-4262-bda7-f622787aeb6c-registry-tls\") pod \"image-registry-6d9fd7cf6c-pk2t8\" (UID: \"886aaf44-5bd7-4262-bda7-f622787aeb6c\") " pod="openshift-image-registry/image-registry-6d9fd7cf6c-pk2t8"
Jul 14 13:55:07.224722 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.224522    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/62788698-f167-42d8-b1e9-cc61cca44012-data-volume\") pod \"insights-runtime-extractor-q6rx4\" (UID: \"62788698-f167-42d8-b1e9-cc61cca44012\") " pod="openshift-insights/insights-runtime-extractor-q6rx4"
Jul 14 13:55:07.224722 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.224549    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/886aaf44-5bd7-4262-bda7-f622787aeb6c-bound-sa-token\") pod \"image-registry-6d9fd7cf6c-pk2t8\" (UID: \"886aaf44-5bd7-4262-bda7-f622787aeb6c\") " pod="openshift-image-registry/image-registry-6d9fd7cf6c-pk2t8"
Jul 14 13:55:07.224722 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.224576    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/62788698-f167-42d8-b1e9-cc61cca44012-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q6rx4\" (UID: \"62788698-f167-42d8-b1e9-cc61cca44012\") " pod="openshift-insights/insights-runtime-extractor-q6rx4"
Jul 14 13:55:07.224722 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.224625    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/886aaf44-5bd7-4262-bda7-f622787aeb6c-trusted-ca\") pod \"image-registry-6d9fd7cf6c-pk2t8\" (UID: \"886aaf44-5bd7-4262-bda7-f622787aeb6c\") " pod="openshift-image-registry/image-registry-6d9fd7cf6c-pk2t8"
Jul 14 13:55:07.224722 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.224647    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wfrq\" (UniqueName: \"kubernetes.io/projected/62788698-f167-42d8-b1e9-cc61cca44012-kube-api-access-9wfrq\") pod \"insights-runtime-extractor-q6rx4\" (UID: \"62788698-f167-42d8-b1e9-cc61cca44012\") " pod="openshift-insights/insights-runtime-extractor-q6rx4"
Jul 14 13:55:07.224722 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.224679    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/886aaf44-5bd7-4262-bda7-f622787aeb6c-registry-certificates\") pod \"image-registry-6d9fd7cf6c-pk2t8\" (UID: \"886aaf44-5bd7-4262-bda7-f622787aeb6c\") " pod="openshift-image-registry/image-registry-6d9fd7cf6c-pk2t8"
Jul 14 13:55:07.225083 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.224793    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/886aaf44-5bd7-4262-bda7-f622787aeb6c-installation-pull-secrets\") pod \"image-registry-6d9fd7cf6c-pk2t8\" (UID: \"886aaf44-5bd7-4262-bda7-f622787aeb6c\") " pod="openshift-image-registry/image-registry-6d9fd7cf6c-pk2t8"
Jul 14 13:55:07.225083 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.224821    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4bbb\" (UniqueName: \"kubernetes.io/projected/886aaf44-5bd7-4262-bda7-f622787aeb6c-kube-api-access-b4bbb\") pod \"image-registry-6d9fd7cf6c-pk2t8\" (UID: \"886aaf44-5bd7-4262-bda7-f622787aeb6c\") " pod="openshift-image-registry/image-registry-6d9fd7cf6c-pk2t8"
Jul 14 13:55:07.225083 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.224845    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/62788698-f167-42d8-b1e9-cc61cca44012-crio-socket\") pod \"insights-runtime-extractor-q6rx4\" (UID: \"62788698-f167-42d8-b1e9-cc61cca44012\") " pod="openshift-insights/insights-runtime-extractor-q6rx4"
Jul 14 13:55:07.225083 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.224876    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/886aaf44-5bd7-4262-bda7-f622787aeb6c-image-registry-private-configuration\") pod \"image-registry-6d9fd7cf6c-pk2t8\" (UID: \"886aaf44-5bd7-4262-bda7-f622787aeb6c\") " pod="openshift-image-registry/image-registry-6d9fd7cf6c-pk2t8"
Jul 14 13:55:07.225083 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.224902    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/62788698-f167-42d8-b1e9-cc61cca44012-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q6rx4\" (UID: \"62788698-f167-42d8-b1e9-cc61cca44012\") " pod="openshift-insights/insights-runtime-extractor-q6rx4"
Jul 14 13:55:07.225083 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.224926    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5vt8\" (UniqueName: \"kubernetes.io/projected/f2371b29-9e56-469a-ac86-7ead875c7127-kube-api-access-c5vt8\") pod \"ingress-canary-vvtxj\" (UID: \"f2371b29-9e56-469a-ac86-7ead875c7127\") " pod="openshift-ingress-canary/ingress-canary-vvtxj"
Jul 14 13:55:07.225372 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.225325    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/886aaf44-5bd7-4262-bda7-f622787aeb6c-ca-trust-extracted\") pod \"image-registry-6d9fd7cf6c-pk2t8\" (UID: \"886aaf44-5bd7-4262-bda7-f622787aeb6c\") " pod="openshift-image-registry/image-registry-6d9fd7cf6c-pk2t8"
Jul 14 13:55:07.225941 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.225915    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/886aaf44-5bd7-4262-bda7-f622787aeb6c-registry-certificates\") pod \"image-registry-6d9fd7cf6c-pk2t8\" (UID: \"886aaf44-5bd7-4262-bda7-f622787aeb6c\") " pod="openshift-image-registry/image-registry-6d9fd7cf6c-pk2t8"
Jul 14 13:55:07.226067 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.226043    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/886aaf44-5bd7-4262-bda7-f622787aeb6c-trusted-ca\") pod \"image-registry-6d9fd7cf6c-pk2t8\" (UID: \"886aaf44-5bd7-4262-bda7-f622787aeb6c\") " pod="openshift-image-registry/image-registry-6d9fd7cf6c-pk2t8"
Jul 14 13:55:07.229211 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.229180    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/886aaf44-5bd7-4262-bda7-f622787aeb6c-image-registry-private-configuration\") pod \"image-registry-6d9fd7cf6c-pk2t8\" (UID: \"886aaf44-5bd7-4262-bda7-f622787aeb6c\") " pod="openshift-image-registry/image-registry-6d9fd7cf6c-pk2t8"
Jul 14 13:55:07.229211 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.229206    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/886aaf44-5bd7-4262-bda7-f622787aeb6c-registry-tls\") pod \"image-registry-6d9fd7cf6c-pk2t8\" (UID: \"886aaf44-5bd7-4262-bda7-f622787aeb6c\") " pod="openshift-image-registry/image-registry-6d9fd7cf6c-pk2t8"
Jul 14 13:55:07.229374 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.229353    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/886aaf44-5bd7-4262-bda7-f622787aeb6c-installation-pull-secrets\") pod \"image-registry-6d9fd7cf6c-pk2t8\" (UID: \"886aaf44-5bd7-4262-bda7-f622787aeb6c\") " pod="openshift-image-registry/image-registry-6d9fd7cf6c-pk2t8"
Jul 14 13:55:07.238291 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.238256    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/886aaf44-5bd7-4262-bda7-f622787aeb6c-bound-sa-token\") pod \"image-registry-6d9fd7cf6c-pk2t8\" (UID: \"886aaf44-5bd7-4262-bda7-f622787aeb6c\") " pod="openshift-image-registry/image-registry-6d9fd7cf6c-pk2t8"
Jul 14 13:55:07.238790 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.238772    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4bbb\" (UniqueName: \"kubernetes.io/projected/886aaf44-5bd7-4262-bda7-f622787aeb6c-kube-api-access-b4bbb\") pod \"image-registry-6d9fd7cf6c-pk2t8\" (UID: \"886aaf44-5bd7-4262-bda7-f622787aeb6c\") " pod="openshift-image-registry/image-registry-6d9fd7cf6c-pk2t8"
Jul 14 13:55:07.240332 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.240304    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:07.315979 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.315940    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d9fd7cf6c-pk2t8"
Jul 14 13:55:07.325504 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.325460    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wfrq\" (UniqueName: \"kubernetes.io/projected/62788698-f167-42d8-b1e9-cc61cca44012-kube-api-access-9wfrq\") pod \"insights-runtime-extractor-q6rx4\" (UID: \"62788698-f167-42d8-b1e9-cc61cca44012\") " pod="openshift-insights/insights-runtime-extractor-q6rx4"
Jul 14 13:55:07.325606 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.325518    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f6561da1-26ed-4edb-b397-4ce7a197fc07-metrics-tls\") pod \"dns-default-hhmmt\" (UID: \"f6561da1-26ed-4edb-b397-4ce7a197fc07\") " pod="openshift-dns/dns-default-hhmmt"
Jul 14 13:55:07.325606 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.325548    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f6561da1-26ed-4edb-b397-4ce7a197fc07-config-volume\") pod \"dns-default-hhmmt\" (UID: \"f6561da1-26ed-4edb-b397-4ce7a197fc07\") " pod="openshift-dns/dns-default-hhmmt"
Jul 14 13:55:07.325606 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.325574    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6561da1-26ed-4edb-b397-4ce7a197fc07-tmp-dir\") pod \"dns-default-hhmmt\" (UID: \"f6561da1-26ed-4edb-b397-4ce7a197fc07\") " pod="openshift-dns/dns-default-hhmmt"
Jul 14 13:55:07.325759 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.325665    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2bws\" (UniqueName: \"kubernetes.io/projected/f6561da1-26ed-4edb-b397-4ce7a197fc07-kube-api-access-v2bws\") pod \"dns-default-hhmmt\" (UID: \"f6561da1-26ed-4edb-b397-4ce7a197fc07\") " pod="openshift-dns/dns-default-hhmmt"
Jul 14 13:55:07.325759 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.325745    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/62788698-f167-42d8-b1e9-cc61cca44012-crio-socket\") pod \"insights-runtime-extractor-q6rx4\" (UID: \"62788698-f167-42d8-b1e9-cc61cca44012\") " pod="openshift-insights/insights-runtime-extractor-q6rx4"
Jul 14 13:55:07.325862 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.325772    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/62788698-f167-42d8-b1e9-cc61cca44012-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q6rx4\" (UID: \"62788698-f167-42d8-b1e9-cc61cca44012\") " pod="openshift-insights/insights-runtime-extractor-q6rx4"
Jul 14 13:55:07.325862 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.325794    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5vt8\" (UniqueName: \"kubernetes.io/projected/f2371b29-9e56-469a-ac86-7ead875c7127-kube-api-access-c5vt8\") pod \"ingress-canary-vvtxj\" (UID: \"f2371b29-9e56-469a-ac86-7ead875c7127\") " pod="openshift-ingress-canary/ingress-canary-vvtxj"
Jul 14 13:55:07.325862 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.325826    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f2371b29-9e56-469a-ac86-7ead875c7127-cert\") pod \"ingress-canary-vvtxj\" (UID: \"f2371b29-9e56-469a-ac86-7ead875c7127\") " pod="openshift-ingress-canary/ingress-canary-vvtxj"
Jul 14 13:55:07.326005 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.325863    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/62788698-f167-42d8-b1e9-cc61cca44012-data-volume\") pod \"insights-runtime-extractor-q6rx4\" (UID: \"62788698-f167-42d8-b1e9-cc61cca44012\") " pod="openshift-insights/insights-runtime-extractor-q6rx4"
Jul 14 13:55:07.326005 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.325900    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/62788698-f167-42d8-b1e9-cc61cca44012-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q6rx4\" (UID: \"62788698-f167-42d8-b1e9-cc61cca44012\") " pod="openshift-insights/insights-runtime-extractor-q6rx4"
Jul 14 13:55:07.326005 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.325974    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/62788698-f167-42d8-b1e9-cc61cca44012-crio-socket\") pod \"insights-runtime-extractor-q6rx4\" (UID: \"62788698-f167-42d8-b1e9-cc61cca44012\") " pod="openshift-insights/insights-runtime-extractor-q6rx4"
Jul 14 13:55:07.326689 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.326651    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/62788698-f167-42d8-b1e9-cc61cca44012-data-volume\") pod \"insights-runtime-extractor-q6rx4\" (UID: \"62788698-f167-42d8-b1e9-cc61cca44012\") " pod="openshift-insights/insights-runtime-extractor-q6rx4"
Jul 14 13:55:07.329798 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.329773    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/62788698-f167-42d8-b1e9-cc61cca44012-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q6rx4\" (UID: \"62788698-f167-42d8-b1e9-cc61cca44012\") " pod="openshift-insights/insights-runtime-extractor-q6rx4"
Jul 14 13:55:07.333560 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.333535    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f2371b29-9e56-469a-ac86-7ead875c7127-cert\") pod \"ingress-canary-vvtxj\" (UID: \"f2371b29-9e56-469a-ac86-7ead875c7127\") " pod="openshift-ingress-canary/ingress-canary-vvtxj"
Jul 14 13:55:07.334084 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.334060    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/62788698-f167-42d8-b1e9-cc61cca44012-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q6rx4\" (UID: \"62788698-f167-42d8-b1e9-cc61cca44012\") " pod="openshift-insights/insights-runtime-extractor-q6rx4"
Jul 14 13:55:07.335742 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.335724    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wfrq\" (UniqueName: \"kubernetes.io/projected/62788698-f167-42d8-b1e9-cc61cca44012-kube-api-access-9wfrq\") pod \"insights-runtime-extractor-q6rx4\" (UID: \"62788698-f167-42d8-b1e9-cc61cca44012\") " pod="openshift-insights/insights-runtime-extractor-q6rx4"
Jul 14 13:55:07.335807 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.335775    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5vt8\" (UniqueName: \"kubernetes.io/projected/f2371b29-9e56-469a-ac86-7ead875c7127-kube-api-access-c5vt8\") pod \"ingress-canary-vvtxj\" (UID: \"f2371b29-9e56-469a-ac86-7ead875c7127\") " pod="openshift-ingress-canary/ingress-canary-vvtxj"
Jul 14 13:55:07.354736 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.354689    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:07.397067 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.396830    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vvtxj"
Jul 14 13:55:07.403283 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.403242    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-q6rx4"
Jul 14 13:55:07.426982 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.426947    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f6561da1-26ed-4edb-b397-4ce7a197fc07-metrics-tls\") pod \"dns-default-hhmmt\" (UID: \"f6561da1-26ed-4edb-b397-4ce7a197fc07\") " pod="openshift-dns/dns-default-hhmmt"
Jul 14 13:55:07.427165 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.426990    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f6561da1-26ed-4edb-b397-4ce7a197fc07-config-volume\") pod \"dns-default-hhmmt\" (UID: \"f6561da1-26ed-4edb-b397-4ce7a197fc07\") " pod="openshift-dns/dns-default-hhmmt"
Jul 14 13:55:07.427165 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.427054    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6561da1-26ed-4edb-b397-4ce7a197fc07-tmp-dir\") pod \"dns-default-hhmmt\" (UID: \"f6561da1-26ed-4edb-b397-4ce7a197fc07\") " pod="openshift-dns/dns-default-hhmmt"
Jul 14 13:55:07.427165 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.427089    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2bws\" (UniqueName: \"kubernetes.io/projected/f6561da1-26ed-4edb-b397-4ce7a197fc07-kube-api-access-v2bws\") pod \"dns-default-hhmmt\" (UID: \"f6561da1-26ed-4edb-b397-4ce7a197fc07\") " pod="openshift-dns/dns-default-hhmmt"
Jul 14 13:55:07.427814 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.427779    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f6561da1-26ed-4edb-b397-4ce7a197fc07-config-volume\") pod \"dns-default-hhmmt\" (UID: \"f6561da1-26ed-4edb-b397-4ce7a197fc07\") " pod="openshift-dns/dns-default-hhmmt"
Jul 14 13:55:07.428068 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.428043    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6561da1-26ed-4edb-b397-4ce7a197fc07-tmp-dir\") pod \"dns-default-hhmmt\" (UID: \"f6561da1-26ed-4edb-b397-4ce7a197fc07\") " pod="openshift-dns/dns-default-hhmmt"
Jul 14 13:55:07.430111 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.430090    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f6561da1-26ed-4edb-b397-4ce7a197fc07-metrics-tls\") pod \"dns-default-hhmmt\" (UID: \"f6561da1-26ed-4edb-b397-4ce7a197fc07\") " pod="openshift-dns/dns-default-hhmmt"
Jul 14 13:55:07.438897 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.438810    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2bws\" (UniqueName: \"kubernetes.io/projected/f6561da1-26ed-4edb-b397-4ce7a197fc07-kube-api-access-v2bws\") pod \"dns-default-hhmmt\" (UID: \"f6561da1-26ed-4edb-b397-4ce7a197fc07\") " pod="openshift-dns/dns-default-hhmmt"
Jul 14 13:55:07.477749 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.477717    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d9fd7cf6c-pk2t8"]
Jul 14 13:55:07.484650 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:55:07.484615    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod886aaf44_5bd7_4262_bda7_f622787aeb6c.slice/crio-e08cc37b896ed18f0ae1c934144f99ea2a989b45d01be8b52d9f32a272f1521e WatchSource:0}: Error finding container e08cc37b896ed18f0ae1c934144f99ea2a989b45d01be8b52d9f32a272f1521e: Status 404 returned error can't find the container with id e08cc37b896ed18f0ae1c934144f99ea2a989b45d01be8b52d9f32a272f1521e
Jul 14 13:55:07.500108 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.500074    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hhmmt"
Jul 14 13:55:07.546816 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.546777    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vvtxj"]
Jul 14 13:55:07.551539 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:55:07.551505    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2371b29_9e56_469a_ac86_7ead875c7127.slice/crio-5bbb921e777e53baeee8418e035a554735a52d8a0581ab9e3410d66ab50624c0 WatchSource:0}: Error finding container 5bbb921e777e53baeee8418e035a554735a52d8a0581ab9e3410d66ab50624c0: Status 404 returned error can't find the container with id 5bbb921e777e53baeee8418e035a554735a52d8a0581ab9e3410d66ab50624c0
Jul 14 13:55:07.558203 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.558117    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-q6rx4"]
Jul 14 13:55:07.565851 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:55:07.565799    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62788698_f167_42d8_b1e9_cc61cca44012.slice/crio-3895b396f59996ac2d15df0def9fa4ddba08071adaf381b7c8396b07fb6e1608 WatchSource:0}: Error finding container 3895b396f59996ac2d15df0def9fa4ddba08071adaf381b7c8396b07fb6e1608: Status 404 returned error can't find the container with id 3895b396f59996ac2d15df0def9fa4ddba08071adaf381b7c8396b07fb6e1608
Jul 14 13:55:07.649822 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.649791    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hhmmt"]
Jul 14 13:55:07.652850 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:55:07.652820    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6561da1_26ed_4edb_b397_4ce7a197fc07.slice/crio-f2ef2f417e622d2648531d74f2caf471a9473592d8b1eb17305101883f69adcf WatchSource:0}: Error finding container f2ef2f417e622d2648531d74f2caf471a9473592d8b1eb17305101883f69adcf: Status 404 returned error can't find the container with id f2ef2f417e622d2648531d74f2caf471a9473592d8b1eb17305101883f69adcf
Jul 14 13:55:07.926281 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.926245    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vvtxj" event={"ID":"f2371b29-9e56-469a-ac86-7ead875c7127","Type":"ContainerStarted","Data":"5bbb921e777e53baeee8418e035a554735a52d8a0581ab9e3410d66ab50624c0"}
Jul 14 13:55:07.927482 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.927455    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hhmmt" event={"ID":"f6561da1-26ed-4edb-b397-4ce7a197fc07","Type":"ContainerStarted","Data":"f2ef2f417e622d2648531d74f2caf471a9473592d8b1eb17305101883f69adcf"}
Jul 14 13:55:07.929175 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.929142    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d9fd7cf6c-pk2t8" event={"ID":"886aaf44-5bd7-4262-bda7-f622787aeb6c","Type":"ContainerStarted","Data":"e85c2acb549ebc036f028f5535eb22c3d004cb4d9cc2af9ffe8190438310b8a0"}
Jul 14 13:55:07.929347 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.929191    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d9fd7cf6c-pk2t8" event={"ID":"886aaf44-5bd7-4262-bda7-f622787aeb6c","Type":"ContainerStarted","Data":"e08cc37b896ed18f0ae1c934144f99ea2a989b45d01be8b52d9f32a272f1521e"}
Jul 14 13:55:07.929347 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.929338    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6d9fd7cf6c-pk2t8"
Jul 14 13:55:07.930287 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.930261    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q6rx4" event={"ID":"62788698-f167-42d8-b1e9-cc61cca44012","Type":"ContainerStarted","Data":"3895b396f59996ac2d15df0def9fa4ddba08071adaf381b7c8396b07fb6e1608"}
Jul 14 13:55:07.958505 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:07.958438    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6d9fd7cf6c-pk2t8" podStartSLOduration=5.958418649 podStartE2EDuration="5.958418649s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:55:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:55:07.957570948 +0000 UTC m=+55.808008450" watchObservedRunningTime="2026-07-14 13:55:07.958418649 +0000 UTC m=+55.808856151"
Jul 14 13:55:08.462804 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:08.462769    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:08.570152 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:08.570114    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:08.686177 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:08.684711    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:55:08.686177 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:08.684743    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:55:08.686177 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:08.684743    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:55:08.688473 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:08.688316    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 14 13:55:08.689687 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:08.689306    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 14 13:55:08.689687 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:08.689547    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 14 13:55:08.689898 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:08.689800    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-v5knd\""
Jul 14 13:55:08.689898 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:08.689882    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7c4p6\""
Jul 14 13:55:08.697967 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:08.697941    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 14 13:55:08.799523 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:08.799453    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:08.891848 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:08.891813    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:10.051147 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:10.051108    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:10.283232 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:10.283204    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:10.383181 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:10.383091    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:13.951083 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:13.951043    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hhmmt" event={"ID":"f6561da1-26ed-4edb-b397-4ce7a197fc07","Type":"ContainerStarted","Data":"9a96e1a17e5f3379df8c2ff0bc0a0d5cddb773d134999eb2bea533c5f71ff085"}
Jul 14 13:55:13.951534 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:13.951155    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-hhmmt"
Jul 14 13:55:13.951534 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:13.951184    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hhmmt" event={"ID":"f6561da1-26ed-4edb-b397-4ce7a197fc07","Type":"ContainerStarted","Data":"1be876002c2d002dd68073c2df9e45596f850a01a0f87f988733e7eff7f7e0d3"}
Jul 14 13:55:13.952710 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:13.952680    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q6rx4" event={"ID":"62788698-f167-42d8-b1e9-cc61cca44012","Type":"ContainerStarted","Data":"7a07f1912cd67169ed7b77902649e3d518feeced6802dd8baf6a62e5b6ea3795"}
Jul 14 13:55:13.954066 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:13.954043    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vvtxj" event={"ID":"f2371b29-9e56-469a-ac86-7ead875c7127","Type":"ContainerStarted","Data":"350d5a490c81c1ce90f44d84f86dc108ebe4a589a06d917e8f4c4cb279deffbc"}
Jul 14 13:55:13.970409 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:13.970353    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-hhmmt" podStartSLOduration=1.743004022 podStartE2EDuration="6.970336796s" totalImagesPullingTime="5.227332774s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:55:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:55:13.969420518 +0000 UTC m=+61.819858021" watchObservedRunningTime="2026-07-14 13:55:13.970336796 +0000 UTC m=+61.820774346"
Jul 14 13:55:13.985414 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:13.985362    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-vvtxj" podStartSLOduration=1.657614975 podStartE2EDuration="6.985345263s" totalImagesPullingTime="5.327730288s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:55:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:55:13.984631351 +0000 UTC m=+61.835068857" watchObservedRunningTime="2026-07-14 13:55:13.985345263 +0000 UTC m=+61.835782766"
Jul 14 13:55:14.027547 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:14.027504    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:14.128051 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:14.128016    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:14.864167 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:14.864140    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sh6rt"
Jul 14 13:55:14.960123 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:14.960034    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q6rx4" event={"ID":"62788698-f167-42d8-b1e9-cc61cca44012","Type":"ContainerStarted","Data":"7ffaa4d8590e770940b001445e9bba881f2457c00ba378bcb836d97164690813"}
Jul 14 13:55:15.069298 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:15.069259    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:15.172010 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:15.171917    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:15.820280 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:15.820224    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:15.919065 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:15.919022    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:16.971528 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:16.971428    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q6rx4" event={"ID":"62788698-f167-42d8-b1e9-cc61cca44012","Type":"ContainerStarted","Data":"da1adc97a5bf6ba31a690c2b000cf753531af478e08d8aa23af07b4f32124f59"}
Jul 14 13:55:16.994191 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:16.994137    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-q6rx4" podStartSLOduration=1.185043245 podStartE2EDuration="9.994123661s" totalImagesPullingTime="8.809080416s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:55:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:55:16.992621348 +0000 UTC m=+64.843058843" watchObservedRunningTime="2026-07-14 13:55:16.994123661 +0000 UTC m=+64.844561164"
Jul 14 13:55:17.080949 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:17.080913    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:17.178657 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:17.178619    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:17.280170 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:17.280138    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:17.305515 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:17.305472    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/02b53875-f2af-42e5-a311-74b8301e8b62-original-pull-secret\") pod \"global-pull-secret-syncer-5m6xn\" (UID: \"02b53875-f2af-42e5-a311-74b8301e8b62\") " pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:55:17.308454 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:17.308428    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 14 13:55:17.318972 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:17.318938    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/02b53875-f2af-42e5-a311-74b8301e8b62-original-pull-secret\") pod \"global-pull-secret-syncer-5m6xn\" (UID: \"02b53875-f2af-42e5-a311-74b8301e8b62\") " pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:55:17.342708 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:17.342666    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:17.405107 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:17.405073    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5m6xn"
Jul 14 13:55:17.406412 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:17.406390    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/188bee4a-9fbe-4f95-8829-a91ab584ce4e-metrics-certs\") pod \"network-metrics-daemon-486sw\" (UID: \"188bee4a-9fbe-4f95-8829-a91ab584ce4e\") " pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:55:17.409172 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:17.409151    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 14 13:55:17.419644 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:17.419616    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/188bee4a-9fbe-4f95-8829-a91ab584ce4e-metrics-certs\") pod \"network-metrics-daemon-486sw\" (UID: \"188bee4a-9fbe-4f95-8829-a91ab584ce4e\") " pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:55:17.436214 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:17.436181    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:17.507321 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:17.507288    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f274l\" (UniqueName: \"kubernetes.io/projected/421bfc78-19a7-4327-8fde-0b7a9927bb4e-kube-api-access-f274l\") pod \"network-check-target-22z2l\" (UID: \"421bfc78-19a7-4327-8fde-0b7a9927bb4e\") " pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:55:17.510257 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:17.510234    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 14 13:55:17.520898 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:17.520867    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 14 13:55:17.531796 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:17.531727    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f274l\" (UniqueName: \"kubernetes.io/projected/421bfc78-19a7-4327-8fde-0b7a9927bb4e-kube-api-access-f274l\") pod \"network-check-target-22z2l\" (UID: \"421bfc78-19a7-4327-8fde-0b7a9927bb4e\") " pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:55:17.536056 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:17.535971    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5m6xn"]
Jul 14 13:55:17.540197 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:55:17.540160    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02b53875_f2af_42e5_a311_74b8301e8b62.slice/crio-8bf1ab2c70ffc0d6fedaf9146950e3580607e7d57a66069fe5a11eb04b82e49c WatchSource:0}: Error finding container 8bf1ab2c70ffc0d6fedaf9146950e3580607e7d57a66069fe5a11eb04b82e49c: Status 404 returned error can't find the container with id 8bf1ab2c70ffc0d6fedaf9146950e3580607e7d57a66069fe5a11eb04b82e49c
Jul 14 13:55:17.718749 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:17.718714    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-v5knd\""
Jul 14 13:55:17.726204 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:17.726170    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-486sw"
Jul 14 13:55:17.728919 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:17.728900    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7c4p6\""
Jul 14 13:55:17.735722 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:17.735694    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:55:17.862220 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:17.862180    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-486sw"]
Jul 14 13:55:17.865856 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:55:17.865831    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod188bee4a_9fbe_4f95_8829_a91ab584ce4e.slice/crio-64d90bfc1a8e2fb998b1d1c9413409c319177438d3125fa4576ebbc5c502693a WatchSource:0}: Error finding container 64d90bfc1a8e2fb998b1d1c9413409c319177438d3125fa4576ebbc5c502693a: Status 404 returned error can't find the container with id 64d90bfc1a8e2fb998b1d1c9413409c319177438d3125fa4576ebbc5c502693a
Jul 14 13:55:17.880577 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:17.880547    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-22z2l"]
Jul 14 13:55:17.883719 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:55:17.883695    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod421bfc78_19a7_4327_8fde_0b7a9927bb4e.slice/crio-929397966f7c6e7d45144779fd4966dc68e480aca4465c39512ad050ec4fbb02 WatchSource:0}: Error finding container 929397966f7c6e7d45144779fd4966dc68e480aca4465c39512ad050ec4fbb02: Status 404 returned error can't find the container with id 929397966f7c6e7d45144779fd4966dc68e480aca4465c39512ad050ec4fbb02
Jul 14 13:55:17.977223 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:17.977192    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-486sw" event={"ID":"188bee4a-9fbe-4f95-8829-a91ab584ce4e","Type":"ContainerStarted","Data":"64d90bfc1a8e2fb998b1d1c9413409c319177438d3125fa4576ebbc5c502693a"}
Jul 14 13:55:17.978420 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:17.978390    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5m6xn" event={"ID":"02b53875-f2af-42e5-a311-74b8301e8b62","Type":"ContainerStarted","Data":"8bf1ab2c70ffc0d6fedaf9146950e3580607e7d57a66069fe5a11eb04b82e49c"}
Jul 14 13:55:17.979553 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:17.979521    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-22z2l" event={"ID":"421bfc78-19a7-4327-8fde-0b7a9927bb4e","Type":"ContainerStarted","Data":"929397966f7c6e7d45144779fd4966dc68e480aca4465c39512ad050ec4fbb02"}
Jul 14 13:55:18.023315 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:18.023277    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:18.139164 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:18.139071    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:18.261185 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:18.261146    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:19.989572 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:19.989529    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-486sw" event={"ID":"188bee4a-9fbe-4f95-8829-a91ab584ce4e","Type":"ContainerStarted","Data":"337cf9ce11f54f4417bc8f90bd73ba92c866e13e73855b0b79d08728397fba71"}
Jul 14 13:55:19.990057 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:19.989634    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-486sw" event={"ID":"188bee4a-9fbe-4f95-8829-a91ab584ce4e","Type":"ContainerStarted","Data":"69807b9feb8b96a288d145d78ffaadde66b01b94aefb50cb3e382f78ba2ad32e"}
Jul 14 13:55:20.008193 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:20.008133    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-486sw" podStartSLOduration=66.458676344 podStartE2EDuration="1m8.008113842s" totalImagesPullingTime="1.549437498s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:54:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:55:20.007784072 +0000 UTC m=+67.858221577" watchObservedRunningTime="2026-07-14 13:55:20.008113842 +0000 UTC m=+67.858551344"
Jul 14 13:55:20.151587 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:20.151525    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:20.252069 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:20.251882    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:20.502020 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:20.501973    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:21.014688 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:21.014636    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:21.108351 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:21.108306    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:21.222124 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:21.222081    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:22.183977 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.183941    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:22.384697 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.384659    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:22.443362 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.443242    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-6jlxv"]
Jul 14 13:55:22.445966 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.445934    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-59gw6"]
Jul 14 13:55:22.446127 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.446045    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-6jlxv"
Jul 14 13:55:22.448363 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.448331    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 14 13:55:22.448506 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.448371    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 14 13:55:22.449067 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.448783    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 14 13:55:22.449067 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.448793    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-59gw6"
Jul 14 13:55:22.449067 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.448801    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-jjx7q\""
Jul 14 13:55:22.449067 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.448927    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 14 13:55:22.449641 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.449622    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 14 13:55:22.449641 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.449634    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 14 13:55:22.451020 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.451001    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 14 13:55:22.451247 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.451227    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-krlv4\""
Jul 14 13:55:22.451367 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.451285    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 14 13:55:22.451367 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.451356    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 14 13:55:22.459721 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.459695    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-6jlxv"]
Jul 14 13:55:22.470301 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.470263    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:22.539306 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.539264    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:22.548185 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.548152    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/470ac83d-59aa-470a-8107-d15cf5e77a92-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-6jlxv\" (UID: \"470ac83d-59aa-470a-8107-d15cf5e77a92\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6jlxv"
Jul 14 13:55:22.548392 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.548198    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/470ac83d-59aa-470a-8107-d15cf5e77a92-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-6jlxv\" (UID: \"470ac83d-59aa-470a-8107-d15cf5e77a92\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6jlxv"
Jul 14 13:55:22.548392 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.548264    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e04c79d0-6500-4b8a-925e-d9aed17efbf6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-59gw6\" (UID: \"e04c79d0-6500-4b8a-925e-d9aed17efbf6\") " pod="openshift-monitoring/node-exporter-59gw6"
Jul 14 13:55:22.548392 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.548344    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/470ac83d-59aa-470a-8107-d15cf5e77a92-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-6jlxv\" (UID: \"470ac83d-59aa-470a-8107-d15cf5e77a92\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6jlxv"
Jul 14 13:55:22.548537 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.548392    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e04c79d0-6500-4b8a-925e-d9aed17efbf6-sys\") pod \"node-exporter-59gw6\" (UID: \"e04c79d0-6500-4b8a-925e-d9aed17efbf6\") " pod="openshift-monitoring/node-exporter-59gw6"
Jul 14 13:55:22.548537 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.548435    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e04c79d0-6500-4b8a-925e-d9aed17efbf6-node-exporter-wtmp\") pod \"node-exporter-59gw6\" (UID: \"e04c79d0-6500-4b8a-925e-d9aed17efbf6\") " pod="openshift-monitoring/node-exporter-59gw6"
Jul 14 13:55:22.548537 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.548459    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jclk\" (UniqueName: \"kubernetes.io/projected/e04c79d0-6500-4b8a-925e-d9aed17efbf6-kube-api-access-9jclk\") pod \"node-exporter-59gw6\" (UID: \"e04c79d0-6500-4b8a-925e-d9aed17efbf6\") " pod="openshift-monitoring/node-exporter-59gw6"
Jul 14 13:55:22.548537 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.548489    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wf9zx\" (UniqueName: \"kubernetes.io/projected/470ac83d-59aa-470a-8107-d15cf5e77a92-kube-api-access-wf9zx\") pod \"kube-state-metrics-785778ff74-6jlxv\" (UID: \"470ac83d-59aa-470a-8107-d15cf5e77a92\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6jlxv"
Jul 14 13:55:22.548537 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.548509    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e04c79d0-6500-4b8a-925e-d9aed17efbf6-root\") pod \"node-exporter-59gw6\" (UID: \"e04c79d0-6500-4b8a-925e-d9aed17efbf6\") " pod="openshift-monitoring/node-exporter-59gw6"
Jul 14 13:55:22.548796 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.548546    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e04c79d0-6500-4b8a-925e-d9aed17efbf6-node-exporter-textfile\") pod \"node-exporter-59gw6\" (UID: \"e04c79d0-6500-4b8a-925e-d9aed17efbf6\") " pod="openshift-monitoring/node-exporter-59gw6"
Jul 14 13:55:22.548796 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.548571    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e04c79d0-6500-4b8a-925e-d9aed17efbf6-node-exporter-accelerators-collector-config\") pod \"node-exporter-59gw6\" (UID: \"e04c79d0-6500-4b8a-925e-d9aed17efbf6\") " pod="openshift-monitoring/node-exporter-59gw6"
Jul 14 13:55:22.548796 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.548648    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/470ac83d-59aa-470a-8107-d15cf5e77a92-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-6jlxv\" (UID: \"470ac83d-59aa-470a-8107-d15cf5e77a92\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6jlxv"
Jul 14 13:55:22.548796 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.548683    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/470ac83d-59aa-470a-8107-d15cf5e77a92-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-6jlxv\" (UID: \"470ac83d-59aa-470a-8107-d15cf5e77a92\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6jlxv"
Jul 14 13:55:22.548796 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.548725    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e04c79d0-6500-4b8a-925e-d9aed17efbf6-node-exporter-tls\") pod \"node-exporter-59gw6\" (UID: \"e04c79d0-6500-4b8a-925e-d9aed17efbf6\") " pod="openshift-monitoring/node-exporter-59gw6"
Jul 14 13:55:22.548796 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.548750    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e04c79d0-6500-4b8a-925e-d9aed17efbf6-metrics-client-ca\") pod \"node-exporter-59gw6\" (UID: \"e04c79d0-6500-4b8a-925e-d9aed17efbf6\") " pod="openshift-monitoring/node-exporter-59gw6"
Jul 14 13:55:22.642127 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.642087    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:22.650118 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.650086    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/470ac83d-59aa-470a-8107-d15cf5e77a92-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-6jlxv\" (UID: \"470ac83d-59aa-470a-8107-d15cf5e77a92\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6jlxv"
Jul 14 13:55:22.650118 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.650122    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e04c79d0-6500-4b8a-925e-d9aed17efbf6-sys\") pod \"node-exporter-59gw6\" (UID: \"e04c79d0-6500-4b8a-925e-d9aed17efbf6\") " pod="openshift-monitoring/node-exporter-59gw6"
Jul 14 13:55:22.650300 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.650143    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e04c79d0-6500-4b8a-925e-d9aed17efbf6-node-exporter-wtmp\") pod \"node-exporter-59gw6\" (UID: \"e04c79d0-6500-4b8a-925e-d9aed17efbf6\") " pod="openshift-monitoring/node-exporter-59gw6"
Jul 14 13:55:22.650300 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.650192    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jclk\" (UniqueName: \"kubernetes.io/projected/e04c79d0-6500-4b8a-925e-d9aed17efbf6-kube-api-access-9jclk\") pod \"node-exporter-59gw6\" (UID: \"e04c79d0-6500-4b8a-925e-d9aed17efbf6\") " pod="openshift-monitoring/node-exporter-59gw6"
Jul 14 13:55:22.650300 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.650207    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e04c79d0-6500-4b8a-925e-d9aed17efbf6-sys\") pod \"node-exporter-59gw6\" (UID: \"e04c79d0-6500-4b8a-925e-d9aed17efbf6\") " pod="openshift-monitoring/node-exporter-59gw6"
Jul 14 13:55:22.650300 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.650235    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wf9zx\" (UniqueName: \"kubernetes.io/projected/470ac83d-59aa-470a-8107-d15cf5e77a92-kube-api-access-wf9zx\") pod \"kube-state-metrics-785778ff74-6jlxv\" (UID: \"470ac83d-59aa-470a-8107-d15cf5e77a92\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6jlxv"
Jul 14 13:55:22.650300 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.650263    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e04c79d0-6500-4b8a-925e-d9aed17efbf6-root\") pod \"node-exporter-59gw6\" (UID: \"e04c79d0-6500-4b8a-925e-d9aed17efbf6\") " pod="openshift-monitoring/node-exporter-59gw6"
Jul 14 13:55:22.650300 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.650271    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e04c79d0-6500-4b8a-925e-d9aed17efbf6-node-exporter-wtmp\") pod \"node-exporter-59gw6\" (UID: \"e04c79d0-6500-4b8a-925e-d9aed17efbf6\") " pod="openshift-monitoring/node-exporter-59gw6"
Jul 14 13:55:22.650626 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.650315    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e04c79d0-6500-4b8a-925e-d9aed17efbf6-node-exporter-textfile\") pod \"node-exporter-59gw6\" (UID: \"e04c79d0-6500-4b8a-925e-d9aed17efbf6\") " pod="openshift-monitoring/node-exporter-59gw6"
Jul 14 13:55:22.650626 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.650326    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e04c79d0-6500-4b8a-925e-d9aed17efbf6-root\") pod \"node-exporter-59gw6\" (UID: \"e04c79d0-6500-4b8a-925e-d9aed17efbf6\") " pod="openshift-monitoring/node-exporter-59gw6"
Jul 14 13:55:22.650626 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.650338    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e04c79d0-6500-4b8a-925e-d9aed17efbf6-node-exporter-accelerators-collector-config\") pod \"node-exporter-59gw6\" (UID: \"e04c79d0-6500-4b8a-925e-d9aed17efbf6\") " pod="openshift-monitoring/node-exporter-59gw6"
Jul 14 13:55:22.650626 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.650427    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/470ac83d-59aa-470a-8107-d15cf5e77a92-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-6jlxv\" (UID: \"470ac83d-59aa-470a-8107-d15cf5e77a92\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6jlxv"
Jul 14 13:55:22.650626 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.650469    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/470ac83d-59aa-470a-8107-d15cf5e77a92-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-6jlxv\" (UID: \"470ac83d-59aa-470a-8107-d15cf5e77a92\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6jlxv"
Jul 14 13:55:22.650626 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.650519    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e04c79d0-6500-4b8a-925e-d9aed17efbf6-node-exporter-tls\") pod \"node-exporter-59gw6\" (UID: \"e04c79d0-6500-4b8a-925e-d9aed17efbf6\") " pod="openshift-monitoring/node-exporter-59gw6"
Jul 14 13:55:22.650626 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.650543    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e04c79d0-6500-4b8a-925e-d9aed17efbf6-metrics-client-ca\") pod \"node-exporter-59gw6\" (UID: \"e04c79d0-6500-4b8a-925e-d9aed17efbf6\") " pod="openshift-monitoring/node-exporter-59gw6"
Jul 14 13:55:22.650626 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.650577    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/470ac83d-59aa-470a-8107-d15cf5e77a92-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-6jlxv\" (UID: \"470ac83d-59aa-470a-8107-d15cf5e77a92\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6jlxv"
Jul 14 13:55:22.650626 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.650606    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/470ac83d-59aa-470a-8107-d15cf5e77a92-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-6jlxv\" (UID: \"470ac83d-59aa-470a-8107-d15cf5e77a92\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6jlxv"
Jul 14 13:55:22.651080 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.650638    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/470ac83d-59aa-470a-8107-d15cf5e77a92-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-6jlxv\" (UID: \"470ac83d-59aa-470a-8107-d15cf5e77a92\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6jlxv"
Jul 14 13:55:22.651080 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.650691    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e04c79d0-6500-4b8a-925e-d9aed17efbf6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-59gw6\" (UID: \"e04c79d0-6500-4b8a-925e-d9aed17efbf6\") " pod="openshift-monitoring/node-exporter-59gw6"
Jul 14 13:55:22.651080 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.650706    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e04c79d0-6500-4b8a-925e-d9aed17efbf6-node-exporter-textfile\") pod \"node-exporter-59gw6\" (UID: \"e04c79d0-6500-4b8a-925e-d9aed17efbf6\") " pod="openshift-monitoring/node-exporter-59gw6"
Jul 14 13:55:22.651080 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:55:22.650813    2576 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 14 13:55:22.651080 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:55:22.650874    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/470ac83d-59aa-470a-8107-d15cf5e77a92-kube-state-metrics-tls podName:470ac83d-59aa-470a-8107-d15cf5e77a92 nodeName:}" failed. No retries permitted until 2026-07-14 13:55:23.150854232 +0000 UTC m=+71.001291733 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/470ac83d-59aa-470a-8107-d15cf5e77a92-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-6jlxv" (UID: "470ac83d-59aa-470a-8107-d15cf5e77a92") : secret "kube-state-metrics-tls" not found
Jul 14 13:55:22.651351 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.651176    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e04c79d0-6500-4b8a-925e-d9aed17efbf6-node-exporter-accelerators-collector-config\") pod \"node-exporter-59gw6\" (UID: \"e04c79d0-6500-4b8a-925e-d9aed17efbf6\") " pod="openshift-monitoring/node-exporter-59gw6"
Jul 14 13:55:22.651351 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.651285    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e04c79d0-6500-4b8a-925e-d9aed17efbf6-metrics-client-ca\") pod \"node-exporter-59gw6\" (UID: \"e04c79d0-6500-4b8a-925e-d9aed17efbf6\") " pod="openshift-monitoring/node-exporter-59gw6"
Jul 14 13:55:22.651456 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.651355    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/470ac83d-59aa-470a-8107-d15cf5e77a92-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-6jlxv\" (UID: \"470ac83d-59aa-470a-8107-d15cf5e77a92\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6jlxv"
Jul 14 13:55:22.651618 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.651577    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/470ac83d-59aa-470a-8107-d15cf5e77a92-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-6jlxv\" (UID: \"470ac83d-59aa-470a-8107-d15cf5e77a92\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6jlxv"
Jul 14 13:55:22.652979 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.652956    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e04c79d0-6500-4b8a-925e-d9aed17efbf6-node-exporter-tls\") pod \"node-exporter-59gw6\" (UID: \"e04c79d0-6500-4b8a-925e-d9aed17efbf6\") " pod="openshift-monitoring/node-exporter-59gw6"
Jul 14 13:55:22.653100 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.653080    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/470ac83d-59aa-470a-8107-d15cf5e77a92-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-6jlxv\" (UID: \"470ac83d-59aa-470a-8107-d15cf5e77a92\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6jlxv"
Jul 14 13:55:22.653199 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.653178    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e04c79d0-6500-4b8a-925e-d9aed17efbf6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-59gw6\" (UID: \"e04c79d0-6500-4b8a-925e-d9aed17efbf6\") " pod="openshift-monitoring/node-exporter-59gw6"
Jul 14 13:55:22.659370 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.659348    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jclk\" (UniqueName: \"kubernetes.io/projected/e04c79d0-6500-4b8a-925e-d9aed17efbf6-kube-api-access-9jclk\") pod \"node-exporter-59gw6\" (UID: \"e04c79d0-6500-4b8a-925e-d9aed17efbf6\") " pod="openshift-monitoring/node-exporter-59gw6"
Jul 14 13:55:22.659584 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.659553    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wf9zx\" (UniqueName: \"kubernetes.io/projected/470ac83d-59aa-470a-8107-d15cf5e77a92-kube-api-access-wf9zx\") pod \"kube-state-metrics-785778ff74-6jlxv\" (UID: \"470ac83d-59aa-470a-8107-d15cf5e77a92\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6jlxv"
Jul 14 13:55:22.766397 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.766368    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-59gw6"
Jul 14 13:55:22.778429 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.778396    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:22.779760 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:55:22.779731    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode04c79d0_6500_4b8a_925e_d9aed17efbf6.slice/crio-395a2ceab16dd88ddee1c15c26fa6a58d54026e61c941d1459f4257dd363f356 WatchSource:0}: Error finding container 395a2ceab16dd88ddee1c15c26fa6a58d54026e61c941d1459f4257dd363f356: Status 404 returned error can't find the container with id 395a2ceab16dd88ddee1c15c26fa6a58d54026e61c941d1459f4257dd363f356
Jul 14 13:55:22.876827 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.876792    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:22.978607 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:22.978478    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:23.002481 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:23.002433    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5m6xn" event={"ID":"02b53875-f2af-42e5-a311-74b8301e8b62","Type":"ContainerStarted","Data":"80c575c4b6dabc6829ccf1732f7b8bfb1177b82db10600adcc3a97a45090e663"}
Jul 14 13:55:23.004060 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:23.004031    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-22z2l" event={"ID":"421bfc78-19a7-4327-8fde-0b7a9927bb4e","Type":"ContainerStarted","Data":"a305edf175442103be283115088d83f5dc727cef76bbbdfa6ee940d12c8fba6e"}
Jul 14 13:55:23.004197 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:23.004146    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:55:23.005184 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:23.005166    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-59gw6" event={"ID":"e04c79d0-6500-4b8a-925e-d9aed17efbf6","Type":"ContainerStarted","Data":"395a2ceab16dd88ddee1c15c26fa6a58d54026e61c941d1459f4257dd363f356"}
Jul 14 13:55:23.019511 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:23.019346    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-5m6xn" podStartSLOduration=65.846192792 podStartE2EDuration="1m11.019327577s" totalImagesPullingTime="5.173134785s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:54:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:55:23.018988489 +0000 UTC m=+70.869425991" watchObservedRunningTime="2026-07-14 13:55:23.019327577 +0000 UTC m=+70.869765080"
Jul 14 13:55:23.038956 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:23.038893    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-22z2l" podStartSLOduration=66.214319347 podStartE2EDuration="1m11.038877286s" totalImagesPullingTime="4.824557939s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:54:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:55:23.037312062 +0000 UTC m=+70.887749566" watchObservedRunningTime="2026-07-14 13:55:23.038877286 +0000 UTC m=+70.889314787"
Jul 14 13:55:23.155545 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:23.155506    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/470ac83d-59aa-470a-8107-d15cf5e77a92-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-6jlxv\" (UID: \"470ac83d-59aa-470a-8107-d15cf5e77a92\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6jlxv"
Jul 14 13:55:23.155738 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:55:23.155678    2576 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 14 13:55:23.155780 ip-10-0-128-127 kubenswrapper[2576]: E0714 13:55:23.155750    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/470ac83d-59aa-470a-8107-d15cf5e77a92-kube-state-metrics-tls podName:470ac83d-59aa-470a-8107-d15cf5e77a92 nodeName:}" failed. No retries permitted until 2026-07-14 13:55:24.155734718 +0000 UTC m=+72.006172200 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/470ac83d-59aa-470a-8107-d15cf5e77a92-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-6jlxv" (UID: "470ac83d-59aa-470a-8107-d15cf5e77a92") : secret "kube-state-metrics-tls" not found
Jul 14 13:55:23.168743 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:23.168704    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:23.391601 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:23.391546    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:23.488347 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:23.488308    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:23.710747 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:23.710699    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:23.798203 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:23.798166    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:23.899355 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:23.899262    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:23.962954 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:23.962928    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-hhmmt"
Jul 14 13:55:24.013847 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:24.013809    2576 generic.go:356] "Generic (PLEG): container finished" podID="e04c79d0-6500-4b8a-925e-d9aed17efbf6" containerID="a7464cfb6fa70c1b55b9c5acdc08658fe349f2de175128e3bae6aac7d0983b97" exitCode=0
Jul 14 13:55:24.014000 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:24.013858    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-59gw6" event={"ID":"e04c79d0-6500-4b8a-925e-d9aed17efbf6","Type":"ContainerDied","Data":"a7464cfb6fa70c1b55b9c5acdc08658fe349f2de175128e3bae6aac7d0983b97"}
Jul 14 13:55:24.163365 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:24.163266    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/470ac83d-59aa-470a-8107-d15cf5e77a92-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-6jlxv\" (UID: \"470ac83d-59aa-470a-8107-d15cf5e77a92\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6jlxv"
Jul 14 13:55:24.166013 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:24.165980    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/470ac83d-59aa-470a-8107-d15cf5e77a92-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-6jlxv\" (UID: \"470ac83d-59aa-470a-8107-d15cf5e77a92\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6jlxv"
Jul 14 13:55:24.259587 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:24.259542    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-6jlxv"
Jul 14 13:55:24.396114 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:24.396078    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-6jlxv"]
Jul 14 13:55:24.399548 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:55:24.399516    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod470ac83d_59aa_470a_8107_d15cf5e77a92.slice/crio-df3b716757d46f76588afb772d9dfd36fdb52ff9fdf1570d6130f7591099f11d WatchSource:0}: Error finding container df3b716757d46f76588afb772d9dfd36fdb52ff9fdf1570d6130f7591099f11d: Status 404 returned error can't find the container with id df3b716757d46f76588afb772d9dfd36fdb52ff9fdf1570d6130f7591099f11d
Jul 14 13:55:24.463520 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:24.463431    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:24.566906 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:24.566872    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:25.020485 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.020443    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-59gw6" event={"ID":"e04c79d0-6500-4b8a-925e-d9aed17efbf6","Type":"ContainerStarted","Data":"94aa2862ed815fbeed6e6151dea80e59fb4fe81b27e5e15d18b19dc01de8fe94"}
Jul 14 13:55:25.020687 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.020511    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-59gw6" event={"ID":"e04c79d0-6500-4b8a-925e-d9aed17efbf6","Type":"ContainerStarted","Data":"5538a396860d32c3ecc3b8e344b686c9ec24cc30d07d640a695d8b43f6257731"}
Jul 14 13:55:25.021903 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.021861    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-6jlxv" event={"ID":"470ac83d-59aa-470a-8107-d15cf5e77a92","Type":"ContainerStarted","Data":"df3b716757d46f76588afb772d9dfd36fdb52ff9fdf1570d6130f7591099f11d"}
Jul 14 13:55:25.043506 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.043439    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-59gw6" podStartSLOduration=2.144698965 podStartE2EDuration="3.043416814s" totalImagesPullingTime="833.155448ms" totalInitContainerRuntime="65.562401ms" isStatefulPod=true podCreationTimestamp="2026-07-14 13:55:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:55:25.04310012 +0000 UTC m=+72.893537623" watchObservedRunningTime="2026-07-14 13:55:25.043416814 +0000 UTC m=+72.893854297"
Jul 14 13:55:25.256014 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.255973    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:25.355155 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.355042    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:25.453254 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.453212    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:25.506083 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.506048    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-676c66bd44-g5zp9"]
Jul 14 13:55:25.510046 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.510014    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-676c66bd44-g5zp9"
Jul 14 13:55:25.516374 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.516085    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 14 13:55:25.516374 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.516102    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 14 13:55:25.516374 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.516150    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-fam1kqjp0f9en\""
Jul 14 13:55:25.516374 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.516258    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 14 13:55:25.516783 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.516431    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 14 13:55:25.516989 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.516969    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 14 13:55:25.517088 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.516996    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-mfqcz\""
Jul 14 13:55:25.536166 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.536127    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-676c66bd44-g5zp9"]
Jul 14 13:55:25.575476 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.575430    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f170ae29-26b7-4bbe-8de9-787f8de9617f-metrics-client-ca\") pod \"thanos-querier-676c66bd44-g5zp9\" (UID: \"f170ae29-26b7-4bbe-8de9-787f8de9617f\") " pod="openshift-monitoring/thanos-querier-676c66bd44-g5zp9"
Jul 14 13:55:25.575764 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.575497    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f170ae29-26b7-4bbe-8de9-787f8de9617f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-676c66bd44-g5zp9\" (UID: \"f170ae29-26b7-4bbe-8de9-787f8de9617f\") " pod="openshift-monitoring/thanos-querier-676c66bd44-g5zp9"
Jul 14 13:55:25.575764 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.575548    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f170ae29-26b7-4bbe-8de9-787f8de9617f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-676c66bd44-g5zp9\" (UID: \"f170ae29-26b7-4bbe-8de9-787f8de9617f\") " pod="openshift-monitoring/thanos-querier-676c66bd44-g5zp9"
Jul 14 13:55:25.575764 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.575607    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f170ae29-26b7-4bbe-8de9-787f8de9617f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-676c66bd44-g5zp9\" (UID: \"f170ae29-26b7-4bbe-8de9-787f8de9617f\") " pod="openshift-monitoring/thanos-querier-676c66bd44-g5zp9"
Jul 14 13:55:25.575764 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.575645    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9ctd\" (UniqueName: \"kubernetes.io/projected/f170ae29-26b7-4bbe-8de9-787f8de9617f-kube-api-access-s9ctd\") pod \"thanos-querier-676c66bd44-g5zp9\" (UID: \"f170ae29-26b7-4bbe-8de9-787f8de9617f\") " pod="openshift-monitoring/thanos-querier-676c66bd44-g5zp9"
Jul 14 13:55:25.575764 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.575680    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f170ae29-26b7-4bbe-8de9-787f8de9617f-secret-grpc-tls\") pod \"thanos-querier-676c66bd44-g5zp9\" (UID: \"f170ae29-26b7-4bbe-8de9-787f8de9617f\") " pod="openshift-monitoring/thanos-querier-676c66bd44-g5zp9"
Jul 14 13:55:25.575764 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.575703    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f170ae29-26b7-4bbe-8de9-787f8de9617f-secret-thanos-querier-tls\") pod \"thanos-querier-676c66bd44-g5zp9\" (UID: \"f170ae29-26b7-4bbe-8de9-787f8de9617f\") " pod="openshift-monitoring/thanos-querier-676c66bd44-g5zp9"
Jul 14 13:55:25.575764 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.575759    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f170ae29-26b7-4bbe-8de9-787f8de9617f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-676c66bd44-g5zp9\" (UID: \"f170ae29-26b7-4bbe-8de9-787f8de9617f\") " pod="openshift-monitoring/thanos-querier-676c66bd44-g5zp9"
Jul 14 13:55:25.676311 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.676210    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f170ae29-26b7-4bbe-8de9-787f8de9617f-metrics-client-ca\") pod \"thanos-querier-676c66bd44-g5zp9\" (UID: \"f170ae29-26b7-4bbe-8de9-787f8de9617f\") " pod="openshift-monitoring/thanos-querier-676c66bd44-g5zp9"
Jul 14 13:55:25.676311 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.676274    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f170ae29-26b7-4bbe-8de9-787f8de9617f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-676c66bd44-g5zp9\" (UID: \"f170ae29-26b7-4bbe-8de9-787f8de9617f\") " pod="openshift-monitoring/thanos-querier-676c66bd44-g5zp9"
Jul 14 13:55:25.676311 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.676307    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f170ae29-26b7-4bbe-8de9-787f8de9617f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-676c66bd44-g5zp9\" (UID: \"f170ae29-26b7-4bbe-8de9-787f8de9617f\") " pod="openshift-monitoring/thanos-querier-676c66bd44-g5zp9"
Jul 14 13:55:25.676677 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.676341    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:25.676677 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.676472    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f170ae29-26b7-4bbe-8de9-787f8de9617f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-676c66bd44-g5zp9\" (UID: \"f170ae29-26b7-4bbe-8de9-787f8de9617f\") " pod="openshift-monitoring/thanos-querier-676c66bd44-g5zp9"
Jul 14 13:55:25.676677 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.676561    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9ctd\" (UniqueName: \"kubernetes.io/projected/f170ae29-26b7-4bbe-8de9-787f8de9617f-kube-api-access-s9ctd\") pod \"thanos-querier-676c66bd44-g5zp9\" (UID: \"f170ae29-26b7-4bbe-8de9-787f8de9617f\") " pod="openshift-monitoring/thanos-querier-676c66bd44-g5zp9"
Jul 14 13:55:25.676677 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.676603    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f170ae29-26b7-4bbe-8de9-787f8de9617f-secret-grpc-tls\") pod \"thanos-querier-676c66bd44-g5zp9\" (UID: \"f170ae29-26b7-4bbe-8de9-787f8de9617f\") " pod="openshift-monitoring/thanos-querier-676c66bd44-g5zp9"
Jul 14 13:55:25.676677 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.676624    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f170ae29-26b7-4bbe-8de9-787f8de9617f-secret-thanos-querier-tls\") pod \"thanos-querier-676c66bd44-g5zp9\" (UID: \"f170ae29-26b7-4bbe-8de9-787f8de9617f\") " pod="openshift-monitoring/thanos-querier-676c66bd44-g5zp9"
Jul 14 13:55:25.676677 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.676671    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f170ae29-26b7-4bbe-8de9-787f8de9617f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-676c66bd44-g5zp9\" (UID: \"f170ae29-26b7-4bbe-8de9-787f8de9617f\") " pod="openshift-monitoring/thanos-querier-676c66bd44-g5zp9"
Jul 14 13:55:25.677390 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.677360    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f170ae29-26b7-4bbe-8de9-787f8de9617f-metrics-client-ca\") pod \"thanos-querier-676c66bd44-g5zp9\" (UID: \"f170ae29-26b7-4bbe-8de9-787f8de9617f\") " pod="openshift-monitoring/thanos-querier-676c66bd44-g5zp9"
Jul 14 13:55:25.679494 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.679463    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f170ae29-26b7-4bbe-8de9-787f8de9617f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-676c66bd44-g5zp9\" (UID: \"f170ae29-26b7-4bbe-8de9-787f8de9617f\") " pod="openshift-monitoring/thanos-querier-676c66bd44-g5zp9"
Jul 14 13:55:25.679616 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.679547    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f170ae29-26b7-4bbe-8de9-787f8de9617f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-676c66bd44-g5zp9\" (UID: \"f170ae29-26b7-4bbe-8de9-787f8de9617f\") " pod="openshift-monitoring/thanos-querier-676c66bd44-g5zp9"
Jul 14 13:55:25.679693 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.679653    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f170ae29-26b7-4bbe-8de9-787f8de9617f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-676c66bd44-g5zp9\" (UID: \"f170ae29-26b7-4bbe-8de9-787f8de9617f\") " pod="openshift-monitoring/thanos-querier-676c66bd44-g5zp9"
Jul 14 13:55:25.679693 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.679673    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f170ae29-26b7-4bbe-8de9-787f8de9617f-secret-grpc-tls\") pod \"thanos-querier-676c66bd44-g5zp9\" (UID: \"f170ae29-26b7-4bbe-8de9-787f8de9617f\") " pod="openshift-monitoring/thanos-querier-676c66bd44-g5zp9"
Jul 14 13:55:25.680371 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.680332    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f170ae29-26b7-4bbe-8de9-787f8de9617f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-676c66bd44-g5zp9\" (UID: \"f170ae29-26b7-4bbe-8de9-787f8de9617f\") " pod="openshift-monitoring/thanos-querier-676c66bd44-g5zp9"
Jul 14 13:55:25.680688 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.680665    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f170ae29-26b7-4bbe-8de9-787f8de9617f-secret-thanos-querier-tls\") pod \"thanos-querier-676c66bd44-g5zp9\" (UID: \"f170ae29-26b7-4bbe-8de9-787f8de9617f\") " pod="openshift-monitoring/thanos-querier-676c66bd44-g5zp9"
Jul 14 13:55:25.687042 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.687022    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9ctd\" (UniqueName: \"kubernetes.io/projected/f170ae29-26b7-4bbe-8de9-787f8de9617f-kube-api-access-s9ctd\") pod \"thanos-querier-676c66bd44-g5zp9\" (UID: \"f170ae29-26b7-4bbe-8de9-787f8de9617f\") " pod="openshift-monitoring/thanos-querier-676c66bd44-g5zp9"
Jul 14 13:55:25.822276 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.822224    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-676c66bd44-g5zp9"
Jul 14 13:55:25.906205 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.906097    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:25.992084 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:25.992050    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-676c66bd44-g5zp9"]
Jul 14 13:55:25.996702 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:55:25.996671    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf170ae29_26b7_4bbe_8de9_787f8de9617f.slice/crio-5afd498bef496516500c933d7ff21e121978c6be1bcd86dec3624e03d62397fd WatchSource:0}: Error finding container 5afd498bef496516500c933d7ff21e121978c6be1bcd86dec3624e03d62397fd: Status 404 returned error can't find the container with id 5afd498bef496516500c933d7ff21e121978c6be1bcd86dec3624e03d62397fd
Jul 14 13:55:26.028058 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:26.028024    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-6jlxv" event={"ID":"470ac83d-59aa-470a-8107-d15cf5e77a92","Type":"ContainerStarted","Data":"f53162c932620698d45b7ee637157fa962091d8ccd1fdcd621278e49bc37d4a7"}
Jul 14 13:55:26.028309 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:26.028093    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-6jlxv" event={"ID":"470ac83d-59aa-470a-8107-d15cf5e77a92","Type":"ContainerStarted","Data":"9e5898e96d11d0b275ff56b3210813c3939edb438fbc7aa37b17e473860bfd5d"}
Jul 14 13:55:26.028309 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:26.028125    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-6jlxv" event={"ID":"470ac83d-59aa-470a-8107-d15cf5e77a92","Type":"ContainerStarted","Data":"f46832139bc080dc7c10c0fad037c1baf9347a34e9d2e7767f0050187aa56da1"}
Jul 14 13:55:26.029305 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:26.029262    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-676c66bd44-g5zp9" event={"ID":"f170ae29-26b7-4bbe-8de9-787f8de9617f","Type":"ContainerStarted","Data":"5afd498bef496516500c933d7ff21e121978c6be1bcd86dec3624e03d62397fd"}
Jul 14 13:55:26.050310 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:26.050247    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-6jlxv" podStartSLOduration=2.712192331 podStartE2EDuration="4.050229288s" totalImagesPullingTime="1.338036957s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:55:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:55:26.049931817 +0000 UTC m=+73.900369321" watchObservedRunningTime="2026-07-14 13:55:26.050229288 +0000 UTC m=+73.900666791"
Jul 14 13:55:26.118927 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:26.118883    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:26.188490 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:26.188459    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:27.208175 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:27.208135    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:27.273658 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:27.273620    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:27.416794 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:27.416710    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:27.590111 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:27.590074    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:28.740873 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.740839    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:28.748953 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.748915    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 13:55:28.754921 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.754892    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.758796 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.758766    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 14 13:55:28.758796 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.758781    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 14 13:55:28.758796 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.758795    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 14 13:55:28.759077 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.758790    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 14 13:55:28.759077 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.758891    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 14 13:55:28.759077 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.758928    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 14 13:55:28.759077 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.758971    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 14 13:55:28.759634 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.759615    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 14 13:55:28.759751 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.759733    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 14 13:55:28.759861 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.759839    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-7gmhb\""
Jul 14 13:55:28.759984 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.759960    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 14 13:55:28.759984 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.759973    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 14 13:55:28.760172 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.760072    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 14 13:55:28.760172 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.760138    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 14 13:55:28.760309 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.760239    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-738sl6rbm2jv9\""
Jul 14 13:55:28.760375 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.760313    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 14 13:55:28.762004 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.761985    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 14 13:55:28.768793 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.768766    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 13:55:28.803433 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.803341    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r6sm\" (UniqueName: \"kubernetes.io/projected/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-kube-api-access-9r6sm\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.803433 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.803378    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.803685 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.803437    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.803685 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.803471    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.803685 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.803492    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.803685 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.803528    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.803685 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.803567    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.803685 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.803616    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-config\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.803685 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.803643    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.803685 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.803673    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-web-config\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.804063 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.803701    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-config-out\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.804063 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.803728    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.804063 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.803771    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.804063 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.803800    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.804063 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.803828    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.804063 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.803856    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.804063 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.803878    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.804063 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.803910    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.804063 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.803964    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.804063 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.803994    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.904679 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.904641    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r6sm\" (UniqueName: \"kubernetes.io/projected/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-kube-api-access-9r6sm\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.904880 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.904692    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.904880 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.904726    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.904880 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.904781    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.904880 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.904807    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.904880 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.904862    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.905208 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.904914    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.905268 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.905205    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-config\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.905322 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.905264    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.905370 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.905305    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-web-config\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.905370 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.905357    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-config-out\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.905463 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.905382    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.905463 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.905451    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.905561 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.905499    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.905561 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.905534    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.905677 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.905579    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.905677 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.905625    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.905677 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.905624    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.905677 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.905664    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.905885 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.905719    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.905885 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.905721    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.905885 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.905762    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.906245 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.906214    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.906341 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.905669    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.907671 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.907648    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.907807 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.907719    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.908071 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.908043    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.908195 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.908165    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.908283 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.908248    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.910656 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.910623    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.910844 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.910821    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.911338 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.911298    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-config-out\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.911338 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.911310    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.911498 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.911354    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-config\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.911903 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.911884    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.912037 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.911909    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.912511 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.912488    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.912918 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.912897    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-web-config\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.914216 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.914191    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.917496 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.917471    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r6sm\" (UniqueName: \"kubernetes.io/projected/1bf6c916-ce8f-4617-8e78-6b6fce3b4657-kube-api-access-9r6sm\") pod \"prometheus-k8s-0\" (UID: \"1bf6c916-ce8f-4617-8e78-6b6fce3b4657\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:28.938910 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:28.938883    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6d9fd7cf6c-pk2t8"
Jul 14 13:55:29.041607 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:29.041548    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-676c66bd44-g5zp9" event={"ID":"f170ae29-26b7-4bbe-8de9-787f8de9617f","Type":"ContainerStarted","Data":"ce9fd54db8fd0c21bf7d2d429ea21ad691606aefc288364722a734f476dade9d"}
Jul 14 13:55:29.041774 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:29.041641    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-676c66bd44-g5zp9" event={"ID":"f170ae29-26b7-4bbe-8de9-787f8de9617f","Type":"ContainerStarted","Data":"542a3a31697ec8e43d39c345a1d344a9e3bc18ddd98b88fb5c1d4af2fc8bb8b1"}
Jul 14 13:55:29.041774 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:29.041668    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-676c66bd44-g5zp9" event={"ID":"f170ae29-26b7-4bbe-8de9-787f8de9617f","Type":"ContainerStarted","Data":"b3fa7dd90d0db522230def6e898b753805cc7162994e48a5dd8cc1394cf321a2"}
Jul 14 13:55:29.067051 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:29.066942    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:29.120640 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:29.120022    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:29.209615 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:29.209571    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:29.220371 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:29.220343    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 13:55:29.223499 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:55:29.223471    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bf6c916_ce8f_4617_8e78_6b6fce3b4657.slice/crio-ec6a02706715eb4c982a73f52271d4c6f716658bb0ca6fb4bd2e0a879b93f798 WatchSource:0}: Error finding container ec6a02706715eb4c982a73f52271d4c6f716658bb0ca6fb4bd2e0a879b93f798: Status 404 returned error can't find the container with id ec6a02706715eb4c982a73f52271d4c6f716658bb0ca6fb4bd2e0a879b93f798
Jul 14 13:55:29.750567 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:29.750522    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:29.809639 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:29.809568    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:30.047976 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:30.047932    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1bf6c916-ce8f-4617-8e78-6b6fce3b4657","Type":"ContainerStarted","Data":"ec6a02706715eb4c982a73f52271d4c6f716658bb0ca6fb4bd2e0a879b93f798"}
Jul 14 13:55:31.053056 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:31.053018    2576 generic.go:356] "Generic (PLEG): container finished" podID="1bf6c916-ce8f-4617-8e78-6b6fce3b4657" containerID="db21fc902ab56821f7b5571da9af9c1c5bdbfca67532d7173de90082269ff4f1" exitCode=0
Jul 14 13:55:31.053537 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:31.053097    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1bf6c916-ce8f-4617-8e78-6b6fce3b4657","Type":"ContainerDied","Data":"db21fc902ab56821f7b5571da9af9c1c5bdbfca67532d7173de90082269ff4f1"}
Jul 14 13:55:31.056727 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:31.056701    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-676c66bd44-g5zp9" event={"ID":"f170ae29-26b7-4bbe-8de9-787f8de9617f","Type":"ContainerStarted","Data":"722d9fa1f63359291b77654208593978565fc01f09d14a5940892e99eba0fd8d"}
Jul 14 13:55:31.056836 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:31.056766    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-676c66bd44-g5zp9" event={"ID":"f170ae29-26b7-4bbe-8de9-787f8de9617f","Type":"ContainerStarted","Data":"b2a46e107f1cd1dcf97fca0e90805ed1dd5ed3fe2fd706919ce2e9a8af549a47"}
Jul 14 13:55:31.056836 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:31.056802    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-676c66bd44-g5zp9" event={"ID":"f170ae29-26b7-4bbe-8de9-787f8de9617f","Type":"ContainerStarted","Data":"e209b4533453979f8bdccc4e6f08e57680920a2f888b3e4eb2ef4a94dca69b55"}
Jul 14 13:55:31.056836 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:31.056833    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-676c66bd44-g5zp9"
Jul 14 13:55:31.114992 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:31.114932    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-676c66bd44-g5zp9" podStartSLOduration=1.934353142 podStartE2EDuration="6.114914934s" totalImagesPullingTime="4.180561792s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:55:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:55:31.113647876 +0000 UTC m=+78.964085384" watchObservedRunningTime="2026-07-14 13:55:31.114914934 +0000 UTC m=+78.965352436"
Jul 14 13:55:32.161545 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:32.161508    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:32.265210 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:32.265159    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:32.363285 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:32.363244    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:34.048820 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:34.048393    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:34.158367 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:34.158329    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:35.084081 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:35.083823    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1bf6c916-ce8f-4617-8e78-6b6fce3b4657","Type":"ContainerStarted","Data":"86cd9ba5378c7aad1b1d1e4e47efd0f041adcd85423b603fc3770da652a2468b"}
Jul 14 13:55:35.084081 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:35.083943    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1bf6c916-ce8f-4617-8e78-6b6fce3b4657","Type":"ContainerStarted","Data":"f995664f42e8455c021fac3587111e275cdea49baa667f71a13b0e0cdc9e283a"}
Jul 14 13:55:35.084081 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:35.083989    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1bf6c916-ce8f-4617-8e78-6b6fce3b4657","Type":"ContainerStarted","Data":"c0365498afe2811dc54d8ded39e896d77ef49de6d4f3715dfe591c6381bd3a6d"}
Jul 14 13:55:35.132244 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:35.132176    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:35.232733 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:35.232702    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:36.091070 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:36.091025    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1bf6c916-ce8f-4617-8e78-6b6fce3b4657","Type":"ContainerStarted","Data":"0cade1eac7075141286d278994a00b5b7834fa7f5eb74b837e8a0abee537929e"}
Jul 14 13:55:36.091445 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:36.091141    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1bf6c916-ce8f-4617-8e78-6b6fce3b4657","Type":"ContainerStarted","Data":"4f6461a7aedb450d8827201867fde4222b9dadd04e99a154c54de483ea0dba2c"}
Jul 14 13:55:36.091445 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:36.091184    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1bf6c916-ce8f-4617-8e78-6b6fce3b4657","Type":"ContainerStarted","Data":"248a9c90d066f952b6988add8010850a45ce9eac2542a58845044169f0971c04"}
Jul 14 13:55:36.123666 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:36.123580    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.173592661 podStartE2EDuration="8.123555383s" totalImagesPullingTime="4.932248201s" totalInitContainerRuntime="17.714521ms" isStatefulPod=false podCreationTimestamp="2026-07-14 13:55:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:55:36.121286084 +0000 UTC m=+83.971723587" watchObservedRunningTime="2026-07-14 13:55:36.123555383 +0000 UTC m=+83.973992888"
Jul 14 13:55:37.070737 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:37.070708    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-676c66bd44-g5zp9"
Jul 14 13:55:38.143997 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:38.143960    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:38.266264 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:38.266220    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:38.416954 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:38.416863    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:38.921406 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:38.921364    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:38.983397 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:38.983356    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:39.067528 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:39.067489    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:55:40.300443 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:40.300402    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:40.457775 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:40.457734    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:43.774605 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:43.774550    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:54.016971 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:54.016936    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-22z2l"
Jul 14 13:55:55.319758 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:55.319723    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:55:55.407427 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:55:55.407395    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:56:01.552265 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:56:01.552221    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:56:01.618907 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:56:01.618870    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:56:22.368400 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:56:22.368356    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:56:29.068142 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:56:29.068091    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:56:29.087548 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:56:29.087520    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:56:29.295832 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:56:29.295801    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 13:56:41.674861 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:56:41.674811    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:56:41.894243 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:56:41.894196    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:56:51.088953 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:56:51.088917    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:56:51.185370 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:56:51.185321    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:56:51.243443 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:56:51.243398    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:56:51.820126 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:56:51.820081    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:56:51.883376 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:56:51.883342    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:56:56.924115 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:56:56.924075    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:56:56.987113 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:56:56.987061    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:57:08.881029 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:57:08.880991    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:57:08.941402 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:57:08.941363    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:57:13.429242 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:57:13.429190    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:57:13.481522 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:57:13.481484    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:57:13.578015 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:57:13.577987    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:57:20.525483 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:57:20.525440    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:57:25.258419 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:57:25.258382    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:57:25.357670 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:57:25.357624    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:57:29.500042 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:57:29.500011    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:57:41.866273 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:57:41.866232    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:58:19.935358 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:58:19.935308    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:58:20.036436 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:58:20.036382    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:58:46.776864 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:58:46.776773    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:58:47.541962 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:58:47.541925    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:58:50.387003 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:58:50.386968    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:58:50.487382 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:58:50.487354    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:58:52.042345 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:58:52.042303    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:58:52.146188 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:58:52.146140    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:58:52.725348 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:58:52.725309    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:58:52.824721 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:58:52.824691    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:58:54.737997 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:58:54.737962    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:58:57.484012 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:58:57.483970    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:59:03.358876 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:03.358842    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:59:09.345994 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:09.345957    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:59:09.402393 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:09.402353    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:59:10.819906 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:10.819868    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-88lc4"]
Jul 14 13:59:10.824307 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:10.824288    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-88lc4"
Jul 14 13:59:10.826740 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:10.826720    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 14 13:59:10.827569 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:10.827551    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 14 13:59:10.827707 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:10.827553    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-llf9n\""
Jul 14 13:59:10.831790 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:10.831667    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-88lc4"]
Jul 14 13:59:10.956357 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:10.956317    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e76c961-6669-4694-8bd1-9e1d85dc0027-tmp\") pod \"openshift-lws-operator-fd8ccff4c-88lc4\" (UID: \"5e76c961-6669-4694-8bd1-9e1d85dc0027\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-88lc4"
Jul 14 13:59:10.956550 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:10.956377    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmmn4\" (UniqueName: \"kubernetes.io/projected/5e76c961-6669-4694-8bd1-9e1d85dc0027-kube-api-access-jmmn4\") pod \"openshift-lws-operator-fd8ccff4c-88lc4\" (UID: \"5e76c961-6669-4694-8bd1-9e1d85dc0027\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-88lc4"
Jul 14 13:59:11.057047 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:11.056983    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e76c961-6669-4694-8bd1-9e1d85dc0027-tmp\") pod \"openshift-lws-operator-fd8ccff4c-88lc4\" (UID: \"5e76c961-6669-4694-8bd1-9e1d85dc0027\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-88lc4"
Jul 14 13:59:11.057215 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:11.057108    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmmn4\" (UniqueName: \"kubernetes.io/projected/5e76c961-6669-4694-8bd1-9e1d85dc0027-kube-api-access-jmmn4\") pod \"openshift-lws-operator-fd8ccff4c-88lc4\" (UID: \"5e76c961-6669-4694-8bd1-9e1d85dc0027\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-88lc4"
Jul 14 13:59:11.057449 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:11.057425    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e76c961-6669-4694-8bd1-9e1d85dc0027-tmp\") pod \"openshift-lws-operator-fd8ccff4c-88lc4\" (UID: \"5e76c961-6669-4694-8bd1-9e1d85dc0027\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-88lc4"
Jul 14 13:59:11.066040 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:11.066008    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmmn4\" (UniqueName: \"kubernetes.io/projected/5e76c961-6669-4694-8bd1-9e1d85dc0027-kube-api-access-jmmn4\") pod \"openshift-lws-operator-fd8ccff4c-88lc4\" (UID: \"5e76c961-6669-4694-8bd1-9e1d85dc0027\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-88lc4"
Jul 14 13:59:11.142567 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:11.142482    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-88lc4"
Jul 14 13:59:11.271526 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:11.271490    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-88lc4"]
Jul 14 13:59:11.274691 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:59:11.274657    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5e76c961_6669_4694_8bd1_9e1d85dc0027.slice/crio-cae2f3be06fe79ea5d937f08234baeb0128a420e2f6a1c92bd8ab415442e31f6 WatchSource:0}: Error finding container cae2f3be06fe79ea5d937f08234baeb0128a420e2f6a1c92bd8ab415442e31f6: Status 404 returned error can't find the container with id cae2f3be06fe79ea5d937f08234baeb0128a420e2f6a1c92bd8ab415442e31f6
Jul 14 13:59:11.840416 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:11.840381    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-88lc4" event={"ID":"5e76c961-6669-4694-8bd1-9e1d85dc0027","Type":"ContainerStarted","Data":"cae2f3be06fe79ea5d937f08234baeb0128a420e2f6a1c92bd8ab415442e31f6"}
Jul 14 13:59:12.532221 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:12.532117    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sh6rt_5600e3e1-e775-482e-bc72-76d732552705/ovn-acl-logging/0.log"
Jul 14 13:59:12.533903 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:12.533874    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sh6rt_5600e3e1-e775-482e-bc72-76d732552705/ovn-acl-logging/0.log"
Jul 14 13:59:12.536105 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:12.536085    2576 kubelet.go:1648] "Image garbage collection succeeded"
Jul 14 13:59:14.854154 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:14.854115    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-88lc4" event={"ID":"5e76c961-6669-4694-8bd1-9e1d85dc0027","Type":"ContainerStarted","Data":"b3d25f271915fbe6761e5cd436fb1f95770caa6bdb7577eaf89e614ed3b094bf"}
Jul 14 13:59:14.871571 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:14.871509    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-88lc4" podStartSLOduration=2.207069572 podStartE2EDuration="4.871489822s" totalImagesPullingTime="2.66442025s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:59:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:59:14.869936651 +0000 UTC m=+302.720374155" watchObservedRunningTime="2026-07-14 13:59:14.871489822 +0000 UTC m=+302.721927325"
Jul 14 13:59:15.569403 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:15.569364    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:59:15.669694 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:15.669660    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:59:17.229737 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:17.229702    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:59:19.403264 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:19.403227    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:59:19.459085 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:19.459038    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:59:25.438990 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:25.438946    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:59:25.510852 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:25.510816    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-d76b86dd9-7gp6n"]
Jul 14 13:59:25.514240 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:25.514215    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-d76b86dd9-7gp6n"
Jul 14 13:59:25.516644 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:25.516620    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 14 13:59:25.517574 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:25.517560    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-b4ztj\""
Jul 14 13:59:25.517653 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:25.517562    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 14 13:59:25.517699 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:25.517663    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 14 13:59:25.528116 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:25.528092    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-d76b86dd9-7gp6n"]
Jul 14 13:59:25.537927 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:25.537891    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:59:25.683192 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:25.683147    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/201cbcaf-0b08-445d-a706-2b5f7da85021-manager-config\") pod \"lws-controller-manager-d76b86dd9-7gp6n\" (UID: \"201cbcaf-0b08-445d-a706-2b5f7da85021\") " pod="openshift-lws-operator/lws-controller-manager-d76b86dd9-7gp6n"
Jul 14 13:59:25.683192 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:25.683196    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/201cbcaf-0b08-445d-a706-2b5f7da85021-metrics-cert\") pod \"lws-controller-manager-d76b86dd9-7gp6n\" (UID: \"201cbcaf-0b08-445d-a706-2b5f7da85021\") " pod="openshift-lws-operator/lws-controller-manager-d76b86dd9-7gp6n"
Jul 14 13:59:25.683415 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:25.683301    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjnrd\" (UniqueName: \"kubernetes.io/projected/201cbcaf-0b08-445d-a706-2b5f7da85021-kube-api-access-wjnrd\") pod \"lws-controller-manager-d76b86dd9-7gp6n\" (UID: \"201cbcaf-0b08-445d-a706-2b5f7da85021\") " pod="openshift-lws-operator/lws-controller-manager-d76b86dd9-7gp6n"
Jul 14 13:59:25.683415 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:25.683360    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/201cbcaf-0b08-445d-a706-2b5f7da85021-cert\") pod \"lws-controller-manager-d76b86dd9-7gp6n\" (UID: \"201cbcaf-0b08-445d-a706-2b5f7da85021\") " pod="openshift-lws-operator/lws-controller-manager-d76b86dd9-7gp6n"
Jul 14 13:59:25.784396 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:25.784358    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/201cbcaf-0b08-445d-a706-2b5f7da85021-cert\") pod \"lws-controller-manager-d76b86dd9-7gp6n\" (UID: \"201cbcaf-0b08-445d-a706-2b5f7da85021\") " pod="openshift-lws-operator/lws-controller-manager-d76b86dd9-7gp6n"
Jul 14 13:59:25.784686 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:25.784490    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/201cbcaf-0b08-445d-a706-2b5f7da85021-manager-config\") pod \"lws-controller-manager-d76b86dd9-7gp6n\" (UID: \"201cbcaf-0b08-445d-a706-2b5f7da85021\") " pod="openshift-lws-operator/lws-controller-manager-d76b86dd9-7gp6n"
Jul 14 13:59:25.784686 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:25.784523    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/201cbcaf-0b08-445d-a706-2b5f7da85021-metrics-cert\") pod \"lws-controller-manager-d76b86dd9-7gp6n\" (UID: \"201cbcaf-0b08-445d-a706-2b5f7da85021\") " pod="openshift-lws-operator/lws-controller-manager-d76b86dd9-7gp6n"
Jul 14 13:59:25.784686 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:25.784627    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjnrd\" (UniqueName: \"kubernetes.io/projected/201cbcaf-0b08-445d-a706-2b5f7da85021-kube-api-access-wjnrd\") pod \"lws-controller-manager-d76b86dd9-7gp6n\" (UID: \"201cbcaf-0b08-445d-a706-2b5f7da85021\") " pod="openshift-lws-operator/lws-controller-manager-d76b86dd9-7gp6n"
Jul 14 13:59:25.785330 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:25.785303    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/201cbcaf-0b08-445d-a706-2b5f7da85021-manager-config\") pod \"lws-controller-manager-d76b86dd9-7gp6n\" (UID: \"201cbcaf-0b08-445d-a706-2b5f7da85021\") " pod="openshift-lws-operator/lws-controller-manager-d76b86dd9-7gp6n"
Jul 14 13:59:25.786928 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:25.786908    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/201cbcaf-0b08-445d-a706-2b5f7da85021-cert\") pod \"lws-controller-manager-d76b86dd9-7gp6n\" (UID: \"201cbcaf-0b08-445d-a706-2b5f7da85021\") " pod="openshift-lws-operator/lws-controller-manager-d76b86dd9-7gp6n"
Jul 14 13:59:25.787083 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:25.787065    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/201cbcaf-0b08-445d-a706-2b5f7da85021-metrics-cert\") pod \"lws-controller-manager-d76b86dd9-7gp6n\" (UID: \"201cbcaf-0b08-445d-a706-2b5f7da85021\") " pod="openshift-lws-operator/lws-controller-manager-d76b86dd9-7gp6n"
Jul 14 13:59:25.794468 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:25.794436    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjnrd\" (UniqueName: \"kubernetes.io/projected/201cbcaf-0b08-445d-a706-2b5f7da85021-kube-api-access-wjnrd\") pod \"lws-controller-manager-d76b86dd9-7gp6n\" (UID: \"201cbcaf-0b08-445d-a706-2b5f7da85021\") " pod="openshift-lws-operator/lws-controller-manager-d76b86dd9-7gp6n"
Jul 14 13:59:25.823233 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:25.823187    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-d76b86dd9-7gp6n"
Jul 14 13:59:25.972667 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:25.972631    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-d76b86dd9-7gp6n"]
Jul 14 13:59:25.977136 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:59:25.977106    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod201cbcaf_0b08_445d_a706_2b5f7da85021.slice/crio-696215dae80d8a9c76a965966858a22b2b256efda9de2a8bf4f65e1c2cd224a6 WatchSource:0}: Error finding container 696215dae80d8a9c76a965966858a22b2b256efda9de2a8bf4f65e1c2cd224a6: Status 404 returned error can't find the container with id 696215dae80d8a9c76a965966858a22b2b256efda9de2a8bf4f65e1c2cd224a6
Jul 14 13:59:25.981395 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:25.981364    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 13:59:26.900070 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:26.900035    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-d76b86dd9-7gp6n" event={"ID":"201cbcaf-0b08-445d-a706-2b5f7da85021","Type":"ContainerStarted","Data":"696215dae80d8a9c76a965966858a22b2b256efda9de2a8bf4f65e1c2cd224a6"}
Jul 14 13:59:28.909942 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:28.909905    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-d76b86dd9-7gp6n" event={"ID":"201cbcaf-0b08-445d-a706-2b5f7da85021","Type":"ContainerStarted","Data":"e3a248efec9c2387473cdec7a410e00d8a0bf6da256e6e59c055bf670e87284a"}
Jul 14 13:59:28.910429 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:28.910037    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-d76b86dd9-7gp6n"
Jul 14 13:59:29.001103 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:29.001048    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-d76b86dd9-7gp6n" podStartSLOduration=1.861515466 podStartE2EDuration="4.001030887s" totalImagesPullingTime="2.139515421s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:59:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:59:28.999554572 +0000 UTC m=+316.849992075" watchObservedRunningTime="2026-07-14 13:59:29.001030887 +0000 UTC m=+316.851468390"
Jul 14 13:59:29.590435 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:29.590387    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:59:29.697832 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:29.697794    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:59:30.201539 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:30.201496    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:59:30.446853 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:30.446818    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:59:30.546927 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:30.546882    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:59:30.610016 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:30.609963    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f858c8968-gfxlg"]
Jul 14 13:59:30.613526 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:30.613501    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-gfxlg"
Jul 14 13:59:30.616009 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:30.615984    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 14 13:59:30.616171 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:30.616037    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 14 13:59:30.616582 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:30.616564    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 14 13:59:30.616832 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:30.616817    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 14 13:59:30.616886 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:30.616838    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-654n7\""
Jul 14 13:59:30.624647 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:30.624624    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0e733524-eb39-472d-9f61-739555243347-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f858c8968-gfxlg\" (UID: \"0e733524-eb39-472d-9f61-739555243347\") " pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-gfxlg"
Jul 14 13:59:30.624747 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:30.624669    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0e733524-eb39-472d-9f61-739555243347-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f858c8968-gfxlg\" (UID: \"0e733524-eb39-472d-9f61-739555243347\") " pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-gfxlg"
Jul 14 13:59:30.624747 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:30.624687    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0e733524-eb39-472d-9f61-739555243347-tmp\") pod \"opendatahub-operator-controller-manager-6f858c8968-gfxlg\" (UID: \"0e733524-eb39-472d-9f61-739555243347\") " pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-gfxlg"
Jul 14 13:59:30.624821 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:30.624766    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0e733524-eb39-472d-9f61-739555243347-manifests\") pod \"opendatahub-operator-controller-manager-6f858c8968-gfxlg\" (UID: \"0e733524-eb39-472d-9f61-739555243347\") " pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-gfxlg"
Jul 14 13:59:30.624821 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:30.624800    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwdmr\" (UniqueName: \"kubernetes.io/projected/0e733524-eb39-472d-9f61-739555243347-kube-api-access-hwdmr\") pod \"opendatahub-operator-controller-manager-6f858c8968-gfxlg\" (UID: \"0e733524-eb39-472d-9f61-739555243347\") " pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-gfxlg"
Jul 14 13:59:30.635853 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:30.635802    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f858c8968-gfxlg"]
Jul 14 13:59:30.726178 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:30.726133    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0e733524-eb39-472d-9f61-739555243347-manifests\") pod \"opendatahub-operator-controller-manager-6f858c8968-gfxlg\" (UID: \"0e733524-eb39-472d-9f61-739555243347\") " pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-gfxlg"
Jul 14 13:59:30.726367 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:30.726182    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwdmr\" (UniqueName: \"kubernetes.io/projected/0e733524-eb39-472d-9f61-739555243347-kube-api-access-hwdmr\") pod \"opendatahub-operator-controller-manager-6f858c8968-gfxlg\" (UID: \"0e733524-eb39-472d-9f61-739555243347\") " pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-gfxlg"
Jul 14 13:59:30.726367 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:30.726225    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0e733524-eb39-472d-9f61-739555243347-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f858c8968-gfxlg\" (UID: \"0e733524-eb39-472d-9f61-739555243347\") " pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-gfxlg"
Jul 14 13:59:30.726367 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:30.726271    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0e733524-eb39-472d-9f61-739555243347-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f858c8968-gfxlg\" (UID: \"0e733524-eb39-472d-9f61-739555243347\") " pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-gfxlg"
Jul 14 13:59:30.726367 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:30.726297    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0e733524-eb39-472d-9f61-739555243347-tmp\") pod \"opendatahub-operator-controller-manager-6f858c8968-gfxlg\" (UID: \"0e733524-eb39-472d-9f61-739555243347\") " pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-gfxlg"
Jul 14 13:59:30.726726 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:30.726698    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0e733524-eb39-472d-9f61-739555243347-manifests\") pod \"opendatahub-operator-controller-manager-6f858c8968-gfxlg\" (UID: \"0e733524-eb39-472d-9f61-739555243347\") " pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-gfxlg"
Jul 14 13:59:30.729045 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:30.729012    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0e733524-eb39-472d-9f61-739555243347-tmp\") pod \"opendatahub-operator-controller-manager-6f858c8968-gfxlg\" (UID: \"0e733524-eb39-472d-9f61-739555243347\") " pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-gfxlg"
Jul 14 13:59:30.729400 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:30.729375    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0e733524-eb39-472d-9f61-739555243347-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f858c8968-gfxlg\" (UID: \"0e733524-eb39-472d-9f61-739555243347\") " pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-gfxlg"
Jul 14 13:59:30.729438 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:30.729375    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0e733524-eb39-472d-9f61-739555243347-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f858c8968-gfxlg\" (UID: \"0e733524-eb39-472d-9f61-739555243347\") " pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-gfxlg"
Jul 14 13:59:30.739406 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:30.739370    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwdmr\" (UniqueName: \"kubernetes.io/projected/0e733524-eb39-472d-9f61-739555243347-kube-api-access-hwdmr\") pod \"opendatahub-operator-controller-manager-6f858c8968-gfxlg\" (UID: \"0e733524-eb39-472d-9f61-739555243347\") " pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-gfxlg"
Jul 14 13:59:30.770421 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:30.770384    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:59:30.869801 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:30.869700    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:59:30.923747 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:30.923715    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-gfxlg"
Jul 14 13:59:30.973034 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:30.972999    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:59:31.043561 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:31.043526    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:59:31.065403 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:31.065357    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f858c8968-gfxlg"]
Jul 14 13:59:31.070147 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:59:31.070113    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e733524_eb39_472d_9f61_739555243347.slice/crio-f358e6d16d6f11c563ea2377191850ee9d700a5396ff5dca8b97c3e740333e27 WatchSource:0}: Error finding container f358e6d16d6f11c563ea2377191850ee9d700a5396ff5dca8b97c3e740333e27: Status 404 returned error can't find the container with id f358e6d16d6f11c563ea2377191850ee9d700a5396ff5dca8b97c3e740333e27
Jul 14 13:59:31.155870 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:31.155765    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:59:31.926166 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:31.926123    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-gfxlg" event={"ID":"0e733524-eb39-472d-9f61-739555243347","Type":"ContainerStarted","Data":"f358e6d16d6f11c563ea2377191850ee9d700a5396ff5dca8b97c3e740333e27"}
Jul 14 13:59:34.941348 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:34.941305    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-gfxlg" event={"ID":"0e733524-eb39-472d-9f61-739555243347","Type":"ContainerStarted","Data":"e882c048563e8c61225a91cc695e0b4efc35fe7ceca5124471bb9cb10525ae98"}
Jul 14 13:59:35.946930 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:35.946888    2576 generic.go:356] "Generic (PLEG): container finished" podID="0e733524-eb39-472d-9f61-739555243347" containerID="e882c048563e8c61225a91cc695e0b4efc35fe7ceca5124471bb9cb10525ae98" exitCode=0
Jul 14 13:59:35.947336 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:35.946981    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-gfxlg" event={"ID":"0e733524-eb39-472d-9f61-739555243347","Type":"ContainerDied","Data":"e882c048563e8c61225a91cc695e0b4efc35fe7ceca5124471bb9cb10525ae98"}
Jul 14 13:59:36.953456 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:36.953414    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-gfxlg" event={"ID":"0e733524-eb39-472d-9f61-739555243347","Type":"ContainerStarted","Data":"e8383f519e3deab964ee9017e2b3447a1648078e8885846a270deef4dd2350a3"}
Jul 14 13:59:36.953873 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:36.953549    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-gfxlg"
Jul 14 13:59:36.979898 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:36.979846    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-gfxlg" podStartSLOduration=2.395983059 podStartE2EDuration="6.979827888s" totalImagesPullingTime="3.529115129s" totalInitContainerRuntime="1.0547297s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:59:30 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:59:36.977992192 +0000 UTC m=+324.828429738" watchObservedRunningTime="2026-07-14 13:59:36.979827888 +0000 UTC m=+324.830265723"
Jul 14 13:59:39.917194 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:39.917158    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-d76b86dd9-7gp6n"
Jul 14 13:59:47.960417 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:47.960387    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-gfxlg"
Jul 14 13:59:49.789033 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:49.788999    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:59:49.918046 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:49.917991    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:59:50.150073 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:50.149972    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:59:50.285498 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:50.285462    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-64945c8566-vbbg6"]
Jul 14 13:59:50.293536 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:50.293511    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-64945c8566-vbbg6"
Jul 14 13:59:50.295963 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:50.295934    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 14 13:59:50.296109 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:50.295977    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 14 13:59:50.296109 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:50.295994    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-4kcfp\""
Jul 14 13:59:50.296109 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:50.295935    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 14 13:59:50.296109 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:50.295943    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 14 13:59:50.299695 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:50.299667    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-64945c8566-vbbg6"]
Jul 14 13:59:50.392819 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:50.392782    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/13fadc50-918b-4e8d-a4d4-b0381ee96030-tmp\") pod \"kube-auth-proxy-64945c8566-vbbg6\" (UID: \"13fadc50-918b-4e8d-a4d4-b0381ee96030\") " pod="openshift-ingress/kube-auth-proxy-64945c8566-vbbg6"
Jul 14 13:59:50.392996 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:50.392904    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13fadc50-918b-4e8d-a4d4-b0381ee96030-tls-certs\") pod \"kube-auth-proxy-64945c8566-vbbg6\" (UID: \"13fadc50-918b-4e8d-a4d4-b0381ee96030\") " pod="openshift-ingress/kube-auth-proxy-64945c8566-vbbg6"
Jul 14 13:59:50.392996 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:50.392976    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrxs6\" (UniqueName: \"kubernetes.io/projected/13fadc50-918b-4e8d-a4d4-b0381ee96030-kube-api-access-wrxs6\") pod \"kube-auth-proxy-64945c8566-vbbg6\" (UID: \"13fadc50-918b-4e8d-a4d4-b0381ee96030\") " pod="openshift-ingress/kube-auth-proxy-64945c8566-vbbg6"
Jul 14 13:59:50.439922 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:50.439827    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:59:50.493929 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:50.493887    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrxs6\" (UniqueName: \"kubernetes.io/projected/13fadc50-918b-4e8d-a4d4-b0381ee96030-kube-api-access-wrxs6\") pod \"kube-auth-proxy-64945c8566-vbbg6\" (UID: \"13fadc50-918b-4e8d-a4d4-b0381ee96030\") " pod="openshift-ingress/kube-auth-proxy-64945c8566-vbbg6"
Jul 14 13:59:50.493929 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:50.493926    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/13fadc50-918b-4e8d-a4d4-b0381ee96030-tmp\") pod \"kube-auth-proxy-64945c8566-vbbg6\" (UID: \"13fadc50-918b-4e8d-a4d4-b0381ee96030\") " pod="openshift-ingress/kube-auth-proxy-64945c8566-vbbg6"
Jul 14 13:59:50.494187 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:50.494008    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13fadc50-918b-4e8d-a4d4-b0381ee96030-tls-certs\") pod \"kube-auth-proxy-64945c8566-vbbg6\" (UID: \"13fadc50-918b-4e8d-a4d4-b0381ee96030\") " pod="openshift-ingress/kube-auth-proxy-64945c8566-vbbg6"
Jul 14 13:59:50.496323 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:50.496296    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/13fadc50-918b-4e8d-a4d4-b0381ee96030-tmp\") pod \"kube-auth-proxy-64945c8566-vbbg6\" (UID: \"13fadc50-918b-4e8d-a4d4-b0381ee96030\") " pod="openshift-ingress/kube-auth-proxy-64945c8566-vbbg6"
Jul 14 13:59:50.496544 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:50.496525    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13fadc50-918b-4e8d-a4d4-b0381ee96030-tls-certs\") pod \"kube-auth-proxy-64945c8566-vbbg6\" (UID: \"13fadc50-918b-4e8d-a4d4-b0381ee96030\") " pod="openshift-ingress/kube-auth-proxy-64945c8566-vbbg6"
Jul 14 13:59:50.509194 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:50.509156    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrxs6\" (UniqueName: \"kubernetes.io/projected/13fadc50-918b-4e8d-a4d4-b0381ee96030-kube-api-access-wrxs6\") pod \"kube-auth-proxy-64945c8566-vbbg6\" (UID: \"13fadc50-918b-4e8d-a4d4-b0381ee96030\") " pod="openshift-ingress/kube-auth-proxy-64945c8566-vbbg6"
Jul 14 13:59:50.571223 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:50.571187    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:59:50.604826 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:50.604783    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-64945c8566-vbbg6"
Jul 14 13:59:50.682378 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:50.682340    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:59:50.757709 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:50.757679    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-64945c8566-vbbg6"]
Jul 14 13:59:50.760408 ip-10-0-128-127 kubenswrapper[2576]: W0714 13:59:50.760377    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13fadc50_918b_4e8d_a4d4_b0381ee96030.slice/crio-0a57151baad5a8adca46823b9691e283de5701111d694d51b1c5e01488d68faa WatchSource:0}: Error finding container 0a57151baad5a8adca46823b9691e283de5701111d694d51b1c5e01488d68faa: Status 404 returned error can't find the container with id 0a57151baad5a8adca46823b9691e283de5701111d694d51b1c5e01488d68faa
Jul 14 13:59:51.008460 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:51.008368    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-64945c8566-vbbg6" event={"ID":"13fadc50-918b-4e8d-a4d4-b0381ee96030","Type":"ContainerStarted","Data":"0a57151baad5a8adca46823b9691e283de5701111d694d51b1c5e01488d68faa"}
Jul 14 13:59:51.022746 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:51.022694    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:59:51.862875 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:51.862829    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:59:51.920365 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:51.920325    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:59:52.483763 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:52.483728    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:59:55.025663 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:55.025626    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-64945c8566-vbbg6" event={"ID":"13fadc50-918b-4e8d-a4d4-b0381ee96030","Type":"ContainerStarted","Data":"d0063c3f0461c9fb9ae03d8e3d351ade53218064378593bf88340e78556a5d21"}
Jul 14 13:59:55.043166 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:55.043079    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-64945c8566-vbbg6" podStartSLOduration=1.71348097 podStartE2EDuration="5.043057445s" totalImagesPullingTime="3.329576475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:59:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:59:55.041491385 +0000 UTC m=+342.891928890" watchObservedRunningTime="2026-07-14 13:59:55.043057445 +0000 UTC m=+342.893494948"
Jul 14 13:59:57.276173 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:57.276131    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:59:57.380950 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:57.380912    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:59:58.053722 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:58.053680    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:59:58.146231 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:58.146194    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 13:59:58.312407 ip-10-0-128-127 kubenswrapper[2576]: I0714 13:59:58.312303    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:00:01.476845 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:00:01.476807    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-kg9mv"]
Jul 14 14:00:01.481233 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:00:01.481203    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kg9mv"
Jul 14 14:00:01.483553 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:00:01.483529    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 14 14:00:01.483739 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:00:01.483706    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-kwjzg\""
Jul 14 14:00:01.489884 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:00:01.489859    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-kg9mv"]
Jul 14 14:00:01.522914 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:00:01.522864    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:00:01.596130 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:00:01.596090    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glccb\" (UniqueName: \"kubernetes.io/projected/cd810af6-a109-428b-acfb-ecbcdf9634b5-kube-api-access-glccb\") pod \"llmisvc-controller-manager-777c6bd7d5-kg9mv\" (UID: \"cd810af6-a109-428b-acfb-ecbcdf9634b5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kg9mv"
Jul 14 14:00:01.596319 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:00:01.596145    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cd810af6-a109-428b-acfb-ecbcdf9634b5-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-kg9mv\" (UID: \"cd810af6-a109-428b-acfb-ecbcdf9634b5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kg9mv"
Jul 14 14:00:01.634531 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:00:01.634484    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:00:01.696790 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:00:01.696752    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cd810af6-a109-428b-acfb-ecbcdf9634b5-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-kg9mv\" (UID: \"cd810af6-a109-428b-acfb-ecbcdf9634b5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kg9mv"
Jul 14 14:00:01.696949 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:00:01.696843    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glccb\" (UniqueName: \"kubernetes.io/projected/cd810af6-a109-428b-acfb-ecbcdf9634b5-kube-api-access-glccb\") pod \"llmisvc-controller-manager-777c6bd7d5-kg9mv\" (UID: \"cd810af6-a109-428b-acfb-ecbcdf9634b5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kg9mv"
Jul 14 14:00:01.696949 ip-10-0-128-127 kubenswrapper[2576]: E0714 14:00:01.696925    2576 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 14 14:00:01.697026 ip-10-0-128-127 kubenswrapper[2576]: E0714 14:00:01.697014    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd810af6-a109-428b-acfb-ecbcdf9634b5-cert podName:cd810af6-a109-428b-acfb-ecbcdf9634b5 nodeName:}" failed. No retries permitted until 2026-07-14 14:00:02.196995914 +0000 UTC m=+350.047433395 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/cd810af6-a109-428b-acfb-ecbcdf9634b5-cert") pod "llmisvc-controller-manager-777c6bd7d5-kg9mv" (UID: "cd810af6-a109-428b-acfb-ecbcdf9634b5") : secret "llmisvc-webhook-server-cert" not found
Jul 14 14:00:01.706061 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:00:01.706026    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glccb\" (UniqueName: \"kubernetes.io/projected/cd810af6-a109-428b-acfb-ecbcdf9634b5-kube-api-access-glccb\") pod \"llmisvc-controller-manager-777c6bd7d5-kg9mv\" (UID: \"cd810af6-a109-428b-acfb-ecbcdf9634b5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kg9mv"
Jul 14 14:00:01.811011 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:00:01.810968    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:00:02.202195 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:00:02.202091    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cd810af6-a109-428b-acfb-ecbcdf9634b5-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-kg9mv\" (UID: \"cd810af6-a109-428b-acfb-ecbcdf9634b5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kg9mv"
Jul 14 14:00:02.204611 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:00:02.204562    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cd810af6-a109-428b-acfb-ecbcdf9634b5-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-kg9mv\" (UID: \"cd810af6-a109-428b-acfb-ecbcdf9634b5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kg9mv"
Jul 14 14:00:02.232370 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:00:02.232335    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:00:02.392791 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:00:02.392751    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kg9mv"
Jul 14 14:00:02.540418 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:00:02.540385    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-kg9mv"]
Jul 14 14:00:02.545972 ip-10-0-128-127 kubenswrapper[2576]: W0714 14:00:02.545940    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podcd810af6_a109_428b_acfb_ecbcdf9634b5.slice/crio-73cc413798e47f2ab50625de8cb8b4d650e481df1b035b1809d6fd937be881a1 WatchSource:0}: Error finding container 73cc413798e47f2ab50625de8cb8b4d650e481df1b035b1809d6fd937be881a1: Status 404 returned error can't find the container with id 73cc413798e47f2ab50625de8cb8b4d650e481df1b035b1809d6fd937be881a1
Jul 14 14:00:02.703178 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:00:02.703142    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:00:02.841719 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:00:02.841684    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:00:02.992466 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:00:02.992427    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:00:03.062383 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:00:03.062347    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kg9mv" event={"ID":"cd810af6-a109-428b-acfb-ecbcdf9634b5","Type":"ContainerStarted","Data":"73cc413798e47f2ab50625de8cb8b4d650e481df1b035b1809d6fd937be881a1"}
Jul 14 14:00:07.082918 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:00:07.082882    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kg9mv" event={"ID":"cd810af6-a109-428b-acfb-ecbcdf9634b5","Type":"ContainerStarted","Data":"bef5f6bdff9997e93bef9d830d70eaeb479d31146847ea32ec1b22297b2a46ae"}
Jul 14 14:00:07.083316 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:00:07.083003    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kg9mv"
Jul 14 14:00:17.905813 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:00:17.905770    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:00:18.005655 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:00:18.005584    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:00:38.093763 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:00:38.093709    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kg9mv"
Jul 14 14:00:38.113232 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:00:38.113179    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kg9mv" podStartSLOduration=33.279992766 podStartE2EDuration="37.113163845s" totalImagesPullingTime="3.833171079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:00:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:00:07.102374333 +0000 UTC m=+354.952811836" watchObservedRunningTime="2026-07-14 14:00:38.113163845 +0000 UTC m=+385.963601364"
Jul 14 14:00:51.477894 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:00:51.477855    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:00:51.571318 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:00:51.571274    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:00:51.686402 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:00:51.686363    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:01:20.551129 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:20.551086    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:01:21.030439 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:21.030407    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:01:25.448261 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:25.448179    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:01:27.047086 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:27.047039    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:01:27.148295 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:27.148264    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:01:28.290818 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:28.290786    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:01:30.546028 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:30.545995    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:01:31.266038 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:31.266004    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:01:33.084922 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:33.084882    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:01:33.700156 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:33.700120    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:01:36.040228 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:36.040186    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:01:36.387562 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:36.387470    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:01:36.490644 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:36.490609    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:01:36.595722 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:36.595682    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:01:40.293032 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:40.292993    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:01:40.430125 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:40.430087    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:01:42.919621 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:42.919560    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-7lcrd"]
Jul 14 14:01:42.920989 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:42.920953    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:01:42.923043 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:42.923026    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7lcrd"
Jul 14 14:01:42.931703 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:42.931676    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 14 14:01:42.931703 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:42.931692    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 14 14:01:42.931899 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:42.931799    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 14 14:01:42.931979 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:42.931965    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 14 14:01:42.932516 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:42.932502    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-ncn5b\""
Jul 14 14:01:42.948173 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:42.948142    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-7lcrd"]
Jul 14 14:01:43.094293 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:43.094263    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ada14027-8b6d-4738-b0fa-cfd2bb01480e-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-7lcrd\" (UID: \"ada14027-8b6d-4738-b0fa-cfd2bb01480e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7lcrd"
Jul 14 14:01:43.094484 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:43.094324    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bw9xm\" (UniqueName: \"kubernetes.io/projected/ada14027-8b6d-4738-b0fa-cfd2bb01480e-kube-api-access-bw9xm\") pod \"kuadrant-console-plugin-66b5774985-7lcrd\" (UID: \"ada14027-8b6d-4738-b0fa-cfd2bb01480e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7lcrd"
Jul 14 14:01:43.094484 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:43.094367    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/ada14027-8b6d-4738-b0fa-cfd2bb01480e-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-7lcrd\" (UID: \"ada14027-8b6d-4738-b0fa-cfd2bb01480e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7lcrd"
Jul 14 14:01:43.118280 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:43.118247    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:01:43.195397 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:43.195301    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bw9xm\" (UniqueName: \"kubernetes.io/projected/ada14027-8b6d-4738-b0fa-cfd2bb01480e-kube-api-access-bw9xm\") pod \"kuadrant-console-plugin-66b5774985-7lcrd\" (UID: \"ada14027-8b6d-4738-b0fa-cfd2bb01480e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7lcrd"
Jul 14 14:01:43.195397 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:43.195355    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/ada14027-8b6d-4738-b0fa-cfd2bb01480e-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-7lcrd\" (UID: \"ada14027-8b6d-4738-b0fa-cfd2bb01480e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7lcrd"
Jul 14 14:01:43.195617 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:43.195404    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ada14027-8b6d-4738-b0fa-cfd2bb01480e-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-7lcrd\" (UID: \"ada14027-8b6d-4738-b0fa-cfd2bb01480e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7lcrd"
Jul 14 14:01:43.196176 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:43.196158    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ada14027-8b6d-4738-b0fa-cfd2bb01480e-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-7lcrd\" (UID: \"ada14027-8b6d-4738-b0fa-cfd2bb01480e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7lcrd"
Jul 14 14:01:43.197926 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:43.197901    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/ada14027-8b6d-4738-b0fa-cfd2bb01480e-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-7lcrd\" (UID: \"ada14027-8b6d-4738-b0fa-cfd2bb01480e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7lcrd"
Jul 14 14:01:43.209767 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:43.209745    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bw9xm\" (UniqueName: \"kubernetes.io/projected/ada14027-8b6d-4738-b0fa-cfd2bb01480e-kube-api-access-bw9xm\") pod \"kuadrant-console-plugin-66b5774985-7lcrd\" (UID: \"ada14027-8b6d-4738-b0fa-cfd2bb01480e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7lcrd"
Jul 14 14:01:43.232197 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:43.232152    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7lcrd"
Jul 14 14:01:43.369956 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:43.369920    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-7lcrd"]
Jul 14 14:01:43.373527 ip-10-0-128-127 kubenswrapper[2576]: W0714 14:01:43.373492    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podada14027_8b6d_4738_b0fa_cfd2bb01480e.slice/crio-9399fa638d1525f71a3e7b0d8cd90705f2f76517740afd473cb73eb3d6237327 WatchSource:0}: Error finding container 9399fa638d1525f71a3e7b0d8cd90705f2f76517740afd473cb73eb3d6237327: Status 404 returned error can't find the container with id 9399fa638d1525f71a3e7b0d8cd90705f2f76517740afd473cb73eb3d6237327
Jul 14 14:01:43.466046 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:43.465959    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7lcrd" event={"ID":"ada14027-8b6d-4738-b0fa-cfd2bb01480e","Type":"ContainerStarted","Data":"9399fa638d1525f71a3e7b0d8cd90705f2f76517740afd473cb73eb3d6237327"}
Jul 14 14:01:43.483997 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:43.483964    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:01:43.590732 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:43.590696    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:01:49.976283 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:49.976245    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:01:50.074262 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:50.074208    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:01:50.691169 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:50.691130    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:01:50.783350 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:50.783285    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:01:52.374334 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:52.374296    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:01:52.522310 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:52.522273    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:01:55.788058 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:55.788009    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:01:55.882150 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:55.882105    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:01:56.323170 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:56.323123    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:01:56.425730 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:56.425677    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:01:59.701674 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:59.701633    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:01:59.804868 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:01:59.804822    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:02:03.102156 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:03.102104    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:02:03.202829 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:03.202790    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:02:03.284909 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:03.284873    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:02:03.539878 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:03.539840    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:02:03.613828 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:03.613766    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:02:07.584028 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:07.583932    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7lcrd" event={"ID":"ada14027-8b6d-4738-b0fa-cfd2bb01480e","Type":"ContainerStarted","Data":"cabd52d5ad4bd17eb503e8b3b7386d7a9aa078e6b0ad25bbbd089e42f59b9bd3"}
Jul 14 14:02:07.610048 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:07.609989    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7lcrd" podStartSLOduration=1.698794847 podStartE2EDuration="25.609970708s" totalImagesPullingTime="23.911175861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:01:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:02:07.609835908 +0000 UTC m=+475.460273412" watchObservedRunningTime="2026-07-14 14:02:07.609970708 +0000 UTC m=+475.460408230"
Jul 14 14:02:10.012425 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:10.012367    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:02:10.113731 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:10.113669    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:02:22.663746 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:22.663708    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:02:22.971698 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:22.971577    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:02:29.961443 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:29.961402    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:02:33.616867 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:33.616825    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:02:33.721210 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:33.721170    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:02:33.789185 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:33.789154    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:02:33.980627 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:33.980522    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:02:34.104668 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:34.104628    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:02:34.104668 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:34.104671    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:02:34.104934 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:34.104788    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"
Jul 14 14:02:34.108692 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:34.108662    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 14 14:02:34.164874 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:34.164829    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/41e392e0-c389-4901-9eae-78919fadc2a5-config-file\") pod \"limitador-limitador-54d6d988d5-dbjgd\" (UID: \"41e392e0-c389-4901-9eae-78919fadc2a5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"
Jul 14 14:02:34.165056 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:34.164892    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzm6k\" (UniqueName: \"kubernetes.io/projected/41e392e0-c389-4901-9eae-78919fadc2a5-kube-api-access-qzm6k\") pod \"limitador-limitador-54d6d988d5-dbjgd\" (UID: \"41e392e0-c389-4901-9eae-78919fadc2a5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"
Jul 14 14:02:34.266348 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:34.266249    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/41e392e0-c389-4901-9eae-78919fadc2a5-config-file\") pod \"limitador-limitador-54d6d988d5-dbjgd\" (UID: \"41e392e0-c389-4901-9eae-78919fadc2a5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"
Jul 14 14:02:34.266348 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:34.266310    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzm6k\" (UniqueName: \"kubernetes.io/projected/41e392e0-c389-4901-9eae-78919fadc2a5-kube-api-access-qzm6k\") pod \"limitador-limitador-54d6d988d5-dbjgd\" (UID: \"41e392e0-c389-4901-9eae-78919fadc2a5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"
Jul 14 14:02:34.266935 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:34.266911    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/41e392e0-c389-4901-9eae-78919fadc2a5-config-file\") pod \"limitador-limitador-54d6d988d5-dbjgd\" (UID: \"41e392e0-c389-4901-9eae-78919fadc2a5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"
Jul 14 14:02:34.275475 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:34.275447    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzm6k\" (UniqueName: \"kubernetes.io/projected/41e392e0-c389-4901-9eae-78919fadc2a5-kube-api-access-qzm6k\") pod \"limitador-limitador-54d6d988d5-dbjgd\" (UID: \"41e392e0-c389-4901-9eae-78919fadc2a5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"
Jul 14 14:02:34.415552 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:34.415509    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"
Jul 14 14:02:34.462400 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:34.462363    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-gvbj4"]
Jul 14 14:02:34.467471 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:34.467442    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-gvbj4"
Jul 14 14:02:34.469813 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:34.469790    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-79trx\""
Jul 14 14:02:34.475210 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:34.475178    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-gvbj4"]
Jul 14 14:02:34.509162 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:34.509121    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:02:34.568459 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:34.568415    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkh7g\" (UniqueName: \"kubernetes.io/projected/c6ad5215-eba0-47a9-901d-cf33dba4ed06-kube-api-access-bkh7g\") pod \"authorino-58cd955675-gvbj4\" (UID: \"c6ad5215-eba0-47a9-901d-cf33dba4ed06\") " pod="kuadrant-system/authorino-58cd955675-gvbj4"
Jul 14 14:02:34.581915 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:34.580880    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:02:34.585367 ip-10-0-128-127 kubenswrapper[2576]: W0714 14:02:34.585324    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41e392e0_c389_4901_9eae_78919fadc2a5.slice/crio-72f664902b08530e605e77a7dd7fb79ba62d40515f8a50072ea3825190b56e5d WatchSource:0}: Error finding container 72f664902b08530e605e77a7dd7fb79ba62d40515f8a50072ea3825190b56e5d: Status 404 returned error can't find the container with id 72f664902b08530e605e77a7dd7fb79ba62d40515f8a50072ea3825190b56e5d
Jul 14 14:02:34.597360 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:34.597330    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:02:34.669339 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:34.669295    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkh7g\" (UniqueName: \"kubernetes.io/projected/c6ad5215-eba0-47a9-901d-cf33dba4ed06-kube-api-access-bkh7g\") pod \"authorino-58cd955675-gvbj4\" (UID: \"c6ad5215-eba0-47a9-901d-cf33dba4ed06\") " pod="kuadrant-system/authorino-58cd955675-gvbj4"
Jul 14 14:02:34.683106 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:34.683070    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkh7g\" (UniqueName: \"kubernetes.io/projected/c6ad5215-eba0-47a9-901d-cf33dba4ed06-kube-api-access-bkh7g\") pod \"authorino-58cd955675-gvbj4\" (UID: \"c6ad5215-eba0-47a9-901d-cf33dba4ed06\") " pod="kuadrant-system/authorino-58cd955675-gvbj4"
Jul 14 14:02:34.695238 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:34.695203    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-dbjgd" event={"ID":"41e392e0-c389-4901-9eae-78919fadc2a5","Type":"ContainerStarted","Data":"72f664902b08530e605e77a7dd7fb79ba62d40515f8a50072ea3825190b56e5d"}
Jul 14 14:02:34.780841 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:34.780741    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-gvbj4"
Jul 14 14:02:35.127241 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:35.127203    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-gvbj4"]
Jul 14 14:02:35.130152 ip-10-0-128-127 kubenswrapper[2576]: W0714 14:02:35.130111    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc6ad5215_eba0_47a9_901d_cf33dba4ed06.slice/crio-21dae4ab67f82a6824ee90d01b71f8444ae7889dcec7456f5217fc26854618d6 WatchSource:0}: Error finding container 21dae4ab67f82a6824ee90d01b71f8444ae7889dcec7456f5217fc26854618d6: Status 404 returned error can't find the container with id 21dae4ab67f82a6824ee90d01b71f8444ae7889dcec7456f5217fc26854618d6
Jul 14 14:02:35.701271 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:35.701235    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-gvbj4" event={"ID":"c6ad5215-eba0-47a9-901d-cf33dba4ed06","Type":"ContainerStarted","Data":"21dae4ab67f82a6824ee90d01b71f8444ae7889dcec7456f5217fc26854618d6"}
Jul 14 14:02:40.218315 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:40.218208    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-gvbj4"]
Jul 14 14:02:40.731744 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:40.731696    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-dbjgd" event={"ID":"41e392e0-c389-4901-9eae-78919fadc2a5","Type":"ContainerStarted","Data":"f1ad07dab7c6352f2e0f13c86bcf8f71f16655bc79d410836fab738be9cf4704"}
Jul 14 14:02:40.731948 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:40.731816    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"
Jul 14 14:02:40.733191 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:40.733168    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-gvbj4" event={"ID":"c6ad5215-eba0-47a9-901d-cf33dba4ed06","Type":"ContainerStarted","Data":"bbf175f23f3319e6189197d0a2c1ae7ec51f8043f52f639f0c224bb9ceb48c3d"}
Jul 14 14:02:40.763884 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:40.763829    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-dbjgd" podStartSLOduration=2.556481049 podStartE2EDuration="7.763812453s" totalImagesPullingTime="5.207331404s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:02:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:02:40.754011482 +0000 UTC m=+508.604448984" watchObservedRunningTime="2026-07-14 14:02:40.763812453 +0000 UTC m=+508.614249995"
Jul 14 14:02:40.779266 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:40.779201    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-gvbj4" podStartSLOduration=2.114014201 podStartE2EDuration="6.779184313s" totalImagesPullingTime="4.665170112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:02:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:02:40.778916832 +0000 UTC m=+508.629354336" watchObservedRunningTime="2026-07-14 14:02:40.779184313 +0000 UTC m=+508.629621815"
Jul 14 14:02:41.737970 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:41.737920    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-gvbj4" podUID="c6ad5215-eba0-47a9-901d-cf33dba4ed06" containerName="authorino" containerID="cri-o://bbf175f23f3319e6189197d0a2c1ae7ec51f8043f52f639f0c224bb9ceb48c3d" gracePeriod=30
Jul 14 14:02:41.981200 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:41.981167    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-gvbj4"
Jul 14 14:02:42.041988 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:42.041947    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bkh7g\" (UniqueName: \"kubernetes.io/projected/c6ad5215-eba0-47a9-901d-cf33dba4ed06-kube-api-access-bkh7g\") pod \"c6ad5215-eba0-47a9-901d-cf33dba4ed06\" (UID: \"c6ad5215-eba0-47a9-901d-cf33dba4ed06\") "
Jul 14 14:02:42.044236 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:42.044200    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6ad5215-eba0-47a9-901d-cf33dba4ed06-kube-api-access-bkh7g" (OuterVolumeSpecName: "kube-api-access-bkh7g") pod "c6ad5215-eba0-47a9-901d-cf33dba4ed06" (UID: "c6ad5215-eba0-47a9-901d-cf33dba4ed06"). InnerVolumeSpecName "kube-api-access-bkh7g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 14:02:42.142783 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:42.142734    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bkh7g\" (UniqueName: \"kubernetes.io/projected/c6ad5215-eba0-47a9-901d-cf33dba4ed06-kube-api-access-bkh7g\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:02:42.743061 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:42.743026    2576 generic.go:356] "Generic (PLEG): container finished" podID="c6ad5215-eba0-47a9-901d-cf33dba4ed06" containerID="bbf175f23f3319e6189197d0a2c1ae7ec51f8043f52f639f0c224bb9ceb48c3d" exitCode=0
Jul 14 14:02:42.743550 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:42.743079    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-gvbj4"
Jul 14 14:02:42.743550 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:42.743083    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-gvbj4" event={"ID":"c6ad5215-eba0-47a9-901d-cf33dba4ed06","Type":"ContainerDied","Data":"bbf175f23f3319e6189197d0a2c1ae7ec51f8043f52f639f0c224bb9ceb48c3d"}
Jul 14 14:02:42.743550 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:42.743176    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-gvbj4" event={"ID":"c6ad5215-eba0-47a9-901d-cf33dba4ed06","Type":"ContainerDied","Data":"21dae4ab67f82a6824ee90d01b71f8444ae7889dcec7456f5217fc26854618d6"}
Jul 14 14:02:42.743550 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:42.743194    2576 scope.go:117] "RemoveContainer" containerID="bbf175f23f3319e6189197d0a2c1ae7ec51f8043f52f639f0c224bb9ceb48c3d"
Jul 14 14:02:42.751716 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:42.751695    2576 scope.go:117] "RemoveContainer" containerID="bbf175f23f3319e6189197d0a2c1ae7ec51f8043f52f639f0c224bb9ceb48c3d"
Jul 14 14:02:42.752041 ip-10-0-128-127 kubenswrapper[2576]: E0714 14:02:42.752013    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbf175f23f3319e6189197d0a2c1ae7ec51f8043f52f639f0c224bb9ceb48c3d\": container with ID starting with bbf175f23f3319e6189197d0a2c1ae7ec51f8043f52f639f0c224bb9ceb48c3d not found: ID does not exist" containerID="bbf175f23f3319e6189197d0a2c1ae7ec51f8043f52f639f0c224bb9ceb48c3d"
Jul 14 14:02:42.752104 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:42.752048    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbf175f23f3319e6189197d0a2c1ae7ec51f8043f52f639f0c224bb9ceb48c3d"} err="failed to get container status \"bbf175f23f3319e6189197d0a2c1ae7ec51f8043f52f639f0c224bb9ceb48c3d\": rpc error: code = NotFound desc = could not find container \"bbf175f23f3319e6189197d0a2c1ae7ec51f8043f52f639f0c224bb9ceb48c3d\": container with ID starting with bbf175f23f3319e6189197d0a2c1ae7ec51f8043f52f639f0c224bb9ceb48c3d not found: ID does not exist"
Jul 14 14:02:42.760514 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:42.760480    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-gvbj4"]
Jul 14 14:02:42.763084 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:42.763058    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-gvbj4"]
Jul 14 14:02:44.688301 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:44.688270    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6ad5215-eba0-47a9-901d-cf33dba4ed06" path="/var/lib/kubelet/pods/c6ad5215-eba0-47a9-901d-cf33dba4ed06/volumes"
Jul 14 14:02:51.739281 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:51.739248    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"
Jul 14 14:02:56.592181 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:56.592134    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:02:56.696224 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:56.696181    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:02:57.145608 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:02:57.145546    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:03:09.579187 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:09.579131    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:03:10.112722 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:10.112687    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-64b6dfd976-sgmqw"]
Jul 14 14:03:10.113062 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:10.113035    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6ad5215-eba0-47a9-901d-cf33dba4ed06" containerName="authorino"
Jul 14 14:03:10.113154 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:10.113065    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6ad5215-eba0-47a9-901d-cf33dba4ed06" containerName="authorino"
Jul 14 14:03:10.113154 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:10.113122    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6ad5215-eba0-47a9-901d-cf33dba4ed06" containerName="authorino"
Jul 14 14:03:10.174070 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:10.174036    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-64b6dfd976-sgmqw"]
Jul 14 14:03:10.174238 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:10.174139    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-64b6dfd976-sgmqw"
Jul 14 14:03:10.176643 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:10.176616    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-79trx\""
Jul 14 14:03:10.181692 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:10.181670    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-64b6dfd976-sgmqw"]
Jul 14 14:03:10.183087 ip-10-0-128-127 kubenswrapper[2576]: E0714 14:03:10.183066    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-8vpl7], unattached volumes=[], failed to process volumes=[kube-api-access-8vpl7]: context canceled" pod="kuadrant-system/authorino-64b6dfd976-sgmqw" podUID="f8be9c09-e071-4973-af29-ac3d7dd57596"
Jul 14 14:03:10.186056 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:10.186037    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vpl7\" (UniqueName: \"kubernetes.io/projected/f8be9c09-e071-4973-af29-ac3d7dd57596-kube-api-access-8vpl7\") pod \"authorino-64b6dfd976-sgmqw\" (UID: \"f8be9c09-e071-4973-af29-ac3d7dd57596\") " pod="kuadrant-system/authorino-64b6dfd976-sgmqw"
Jul 14 14:03:10.208587 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:10.208551    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c6bb74b88-szn66"]
Jul 14 14:03:10.216737 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:10.216700    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c6bb74b88-szn66"
Jul 14 14:03:10.219186 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:10.219161    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 14 14:03:10.219950 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:10.219927    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c6bb74b88-szn66"]
Jul 14 14:03:10.286907 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:10.286870    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tm8bf\" (UniqueName: \"kubernetes.io/projected/788b35c6-6202-4ace-8631-44651e2fbe7a-kube-api-access-tm8bf\") pod \"authorino-c6bb74b88-szn66\" (UID: \"788b35c6-6202-4ace-8631-44651e2fbe7a\") " pod="kuadrant-system/authorino-c6bb74b88-szn66"
Jul 14 14:03:10.287099 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:10.286950    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/788b35c6-6202-4ace-8631-44651e2fbe7a-tls-cert\") pod \"authorino-c6bb74b88-szn66\" (UID: \"788b35c6-6202-4ace-8631-44651e2fbe7a\") " pod="kuadrant-system/authorino-c6bb74b88-szn66"
Jul 14 14:03:10.287099 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:10.287043    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vpl7\" (UniqueName: \"kubernetes.io/projected/f8be9c09-e071-4973-af29-ac3d7dd57596-kube-api-access-8vpl7\") pod \"authorino-64b6dfd976-sgmqw\" (UID: \"f8be9c09-e071-4973-af29-ac3d7dd57596\") " pod="kuadrant-system/authorino-64b6dfd976-sgmqw"
Jul 14 14:03:10.295931 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:10.295905    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vpl7\" (UniqueName: \"kubernetes.io/projected/f8be9c09-e071-4973-af29-ac3d7dd57596-kube-api-access-8vpl7\") pod \"authorino-64b6dfd976-sgmqw\" (UID: \"f8be9c09-e071-4973-af29-ac3d7dd57596\") " pod="kuadrant-system/authorino-64b6dfd976-sgmqw"
Jul 14 14:03:10.387757 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:10.387651    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tm8bf\" (UniqueName: \"kubernetes.io/projected/788b35c6-6202-4ace-8631-44651e2fbe7a-kube-api-access-tm8bf\") pod \"authorino-c6bb74b88-szn66\" (UID: \"788b35c6-6202-4ace-8631-44651e2fbe7a\") " pod="kuadrant-system/authorino-c6bb74b88-szn66"
Jul 14 14:03:10.387757 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:10.387708    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/788b35c6-6202-4ace-8631-44651e2fbe7a-tls-cert\") pod \"authorino-c6bb74b88-szn66\" (UID: \"788b35c6-6202-4ace-8631-44651e2fbe7a\") " pod="kuadrant-system/authorino-c6bb74b88-szn66"
Jul 14 14:03:10.390118 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:10.390098    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/788b35c6-6202-4ace-8631-44651e2fbe7a-tls-cert\") pod \"authorino-c6bb74b88-szn66\" (UID: \"788b35c6-6202-4ace-8631-44651e2fbe7a\") " pod="kuadrant-system/authorino-c6bb74b88-szn66"
Jul 14 14:03:10.397056 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:10.397030    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tm8bf\" (UniqueName: \"kubernetes.io/projected/788b35c6-6202-4ace-8631-44651e2fbe7a-kube-api-access-tm8bf\") pod \"authorino-c6bb74b88-szn66\" (UID: \"788b35c6-6202-4ace-8631-44651e2fbe7a\") " pod="kuadrant-system/authorino-c6bb74b88-szn66"
Jul 14 14:03:10.527607 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:10.527552    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c6bb74b88-szn66"
Jul 14 14:03:10.672335 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:10.672161    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c6bb74b88-szn66"]
Jul 14 14:03:10.674946 ip-10-0-128-127 kubenswrapper[2576]: W0714 14:03:10.674912    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod788b35c6_6202_4ace_8631_44651e2fbe7a.slice/crio-b76cf95ba59e20926c2d6bc2484e071110f4f9f456c13eeb480709b389504074 WatchSource:0}: Error finding container b76cf95ba59e20926c2d6bc2484e071110f4f9f456c13eeb480709b389504074: Status 404 returned error can't find the container with id b76cf95ba59e20926c2d6bc2484e071110f4f9f456c13eeb480709b389504074
Jul 14 14:03:10.869309 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:10.869279    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-64b6dfd976-sgmqw"
Jul 14 14:03:10.869547 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:10.869282    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c6bb74b88-szn66" event={"ID":"788b35c6-6202-4ace-8631-44651e2fbe7a","Type":"ContainerStarted","Data":"b76cf95ba59e20926c2d6bc2484e071110f4f9f456c13eeb480709b389504074"}
Jul 14 14:03:10.889045 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:10.889017    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-64b6dfd976-sgmqw"
Jul 14 14:03:10.891533 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:10.891510    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8vpl7\" (UniqueName: \"kubernetes.io/projected/f8be9c09-e071-4973-af29-ac3d7dd57596-kube-api-access-8vpl7\") pod \"f8be9c09-e071-4973-af29-ac3d7dd57596\" (UID: \"f8be9c09-e071-4973-af29-ac3d7dd57596\") "
Jul 14 14:03:10.893708 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:10.893686    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8be9c09-e071-4973-af29-ac3d7dd57596-kube-api-access-8vpl7" (OuterVolumeSpecName: "kube-api-access-8vpl7") pod "f8be9c09-e071-4973-af29-ac3d7dd57596" (UID: "f8be9c09-e071-4973-af29-ac3d7dd57596"). InnerVolumeSpecName "kube-api-access-8vpl7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 14:03:10.992228 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:10.992159    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vpl7\" (UniqueName: \"kubernetes.io/projected/f8be9c09-e071-4973-af29-ac3d7dd57596-kube-api-access-8vpl7\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:03:11.496771 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:11.496739    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:03:11.875394 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:11.875364    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-64b6dfd976-sgmqw"
Jul 14 14:03:11.875394 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:11.875368    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c6bb74b88-szn66" event={"ID":"788b35c6-6202-4ace-8631-44651e2fbe7a","Type":"ContainerStarted","Data":"b51064fc102f4f59458c33f8902f83671bda24110b256d952320ea38720fed21"}
Jul 14 14:03:11.895963 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:11.895908    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-c6bb74b88-szn66" podStartSLOduration=1.409214874 podStartE2EDuration="1.89589133s" totalImagesPullingTime="486.676456ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:03:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:03:11.894636262 +0000 UTC m=+539.745073765" watchObservedRunningTime="2026-07-14 14:03:11.89589133 +0000 UTC m=+539.746328832"
Jul 14 14:03:11.919549 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:11.919511    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-64b6dfd976-sgmqw"]
Jul 14 14:03:11.924429 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:11.924393    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-64b6dfd976-sgmqw"]
Jul 14 14:03:12.689646 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:12.689618    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8be9c09-e071-4973-af29-ac3d7dd57596" path="/var/lib/kubelet/pods/f8be9c09-e071-4973-af29-ac3d7dd57596/volumes"
Jul 14 14:03:16.934801 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:16.934763    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:03:17.122760 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:17.122720    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:03:17.267884 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:17.267787    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:03:20.827067 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:20.827030    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:03:20.929473 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:20.929407    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:03:52.851889 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:52.851825    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:03:52.972377 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:52.972344    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:03:53.540117 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:53.540075    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:03:53.760384 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:53.760346    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:03:54.475770 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:54.475731    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:03:54.600854 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:54.600814    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:03:54.857336 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:54.857295    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:03:57.037439 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:57.037402    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:03:57.099067 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:03:57.098997    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:04:04.598617 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:04:04.598538    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:04:06.320908 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:04:06.320870    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:04:12.558662 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:04:12.558632    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sh6rt_5600e3e1-e775-482e-bc72-76d732552705/ovn-acl-logging/0.log"
Jul 14 14:04:12.559090 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:04:12.558830    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sh6rt_5600e3e1-e775-482e-bc72-76d732552705/ovn-acl-logging/0.log"
Jul 14 14:04:21.841532 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:04:21.841467    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:04:22.114017 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:04:22.113913    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:04:42.628443 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:04:42.628348    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:04:53.391399 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:04:53.391353    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:04:54.504342 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:04:54.504302    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:05:00.159723 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:00.159675    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:05:09.115965 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:09.115926    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:05:10.202011 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:10.201973    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:05:10.214033 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:10.213995    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:05:21.077440 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:21.077399    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:05:22.414230 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:22.414186    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:05:27.103205 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:27.103169    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:05:29.509399 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:29.509351    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:05:32.708518 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:32.708472    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:05:32.809548 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:32.809505    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:05:54.888568 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:54.888532    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k"]
Jul 14 14:05:54.891954 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:54.891939    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k"
Jul 14 14:05:54.895655 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:54.895628    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 14 14:05:54.895813 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:54.895721    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 14 14:05:54.895938 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:54.895915    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 14 14:05:54.896013 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:54.895956    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6dw7s\""
Jul 14 14:05:54.902544 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:54.902517    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k"]
Jul 14 14:05:54.933098 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:54.933059    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:05:54.958287 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:54.958251    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8934bfa2-4d59-4a28-ab17-b0feeaad4cda-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k\" (UID: \"8934bfa2-4d59-4a28-ab17-b0feeaad4cda\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k"
Jul 14 14:05:54.958421 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:54.958303    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8934bfa2-4d59-4a28-ab17-b0feeaad4cda-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k\" (UID: \"8934bfa2-4d59-4a28-ab17-b0feeaad4cda\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k"
Jul 14 14:05:54.958421 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:54.958338    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sclmf\" (UniqueName: \"kubernetes.io/projected/8934bfa2-4d59-4a28-ab17-b0feeaad4cda-kube-api-access-sclmf\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k\" (UID: \"8934bfa2-4d59-4a28-ab17-b0feeaad4cda\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k"
Jul 14 14:05:54.958421 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:54.958371    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8934bfa2-4d59-4a28-ab17-b0feeaad4cda-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k\" (UID: \"8934bfa2-4d59-4a28-ab17-b0feeaad4cda\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k"
Jul 14 14:05:54.958553 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:54.958445    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8934bfa2-4d59-4a28-ab17-b0feeaad4cda-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k\" (UID: \"8934bfa2-4d59-4a28-ab17-b0feeaad4cda\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k"
Jul 14 14:05:54.958553 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:54.958470    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8934bfa2-4d59-4a28-ab17-b0feeaad4cda-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k\" (UID: \"8934bfa2-4d59-4a28-ab17-b0feeaad4cda\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k"
Jul 14 14:05:55.059081 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:55.059042    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8934bfa2-4d59-4a28-ab17-b0feeaad4cda-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k\" (UID: \"8934bfa2-4d59-4a28-ab17-b0feeaad4cda\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k"
Jul 14 14:05:55.059275 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:55.059094    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sclmf\" (UniqueName: \"kubernetes.io/projected/8934bfa2-4d59-4a28-ab17-b0feeaad4cda-kube-api-access-sclmf\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k\" (UID: \"8934bfa2-4d59-4a28-ab17-b0feeaad4cda\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k"
Jul 14 14:05:55.059275 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:55.059139    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8934bfa2-4d59-4a28-ab17-b0feeaad4cda-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k\" (UID: \"8934bfa2-4d59-4a28-ab17-b0feeaad4cda\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k"
Jul 14 14:05:55.059275 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:55.059186    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8934bfa2-4d59-4a28-ab17-b0feeaad4cda-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k\" (UID: \"8934bfa2-4d59-4a28-ab17-b0feeaad4cda\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k"
Jul 14 14:05:55.059275 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:55.059211    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8934bfa2-4d59-4a28-ab17-b0feeaad4cda-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k\" (UID: \"8934bfa2-4d59-4a28-ab17-b0feeaad4cda\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k"
Jul 14 14:05:55.059275 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:55.059260    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8934bfa2-4d59-4a28-ab17-b0feeaad4cda-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k\" (UID: \"8934bfa2-4d59-4a28-ab17-b0feeaad4cda\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k"
Jul 14 14:05:55.059650 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:55.059630    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8934bfa2-4d59-4a28-ab17-b0feeaad4cda-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k\" (UID: \"8934bfa2-4d59-4a28-ab17-b0feeaad4cda\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k"
Jul 14 14:05:55.059732 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:55.059629    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8934bfa2-4d59-4a28-ab17-b0feeaad4cda-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k\" (UID: \"8934bfa2-4d59-4a28-ab17-b0feeaad4cda\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k"
Jul 14 14:05:55.059732 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:55.059664    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8934bfa2-4d59-4a28-ab17-b0feeaad4cda-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k\" (UID: \"8934bfa2-4d59-4a28-ab17-b0feeaad4cda\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k"
Jul 14 14:05:55.061436 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:55.061403    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8934bfa2-4d59-4a28-ab17-b0feeaad4cda-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k\" (UID: \"8934bfa2-4d59-4a28-ab17-b0feeaad4cda\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k"
Jul 14 14:05:55.061685 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:55.061668    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8934bfa2-4d59-4a28-ab17-b0feeaad4cda-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k\" (UID: \"8934bfa2-4d59-4a28-ab17-b0feeaad4cda\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k"
Jul 14 14:05:55.067101 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:55.067076    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sclmf\" (UniqueName: \"kubernetes.io/projected/8934bfa2-4d59-4a28-ab17-b0feeaad4cda-kube-api-access-sclmf\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k\" (UID: \"8934bfa2-4d59-4a28-ab17-b0feeaad4cda\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k"
Jul 14 14:05:55.202551 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:55.202450    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k"
Jul 14 14:05:55.347298 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:55.347136    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k"]
Jul 14 14:05:55.350038 ip-10-0-128-127 kubenswrapper[2576]: W0714 14:05:55.350008    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8934bfa2_4d59_4a28_ab17_b0feeaad4cda.slice/crio-2de21decca54fc05fed130ce28c0f8e55c51a24470d443be155d5af4fac7eb7c WatchSource:0}: Error finding container 2de21decca54fc05fed130ce28c0f8e55c51a24470d443be155d5af4fac7eb7c: Status 404 returned error can't find the container with id 2de21decca54fc05fed130ce28c0f8e55c51a24470d443be155d5af4fac7eb7c
Jul 14 14:05:55.351370 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:55.351350    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 14:05:55.590872 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:55.590838    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k" event={"ID":"8934bfa2-4d59-4a28-ab17-b0feeaad4cda","Type":"ContainerStarted","Data":"2de21decca54fc05fed130ce28c0f8e55c51a24470d443be155d5af4fac7eb7c"}
Jul 14 14:05:57.338081 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:57.338042    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:05:57.603148 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:57.603057    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k" event={"ID":"8934bfa2-4d59-4a28-ab17-b0feeaad4cda","Type":"ContainerStarted","Data":"7df23a841bd509acb4ab314bea71ccb97a68fba183495c132ee46bf8c4be4af9"}
Jul 14 14:05:57.634018 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:05:57.633958    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k" podStartSLOduration=2.17023396 podStartE2EDuration="3.633940023s" totalImagesPullingTime="1.463706063s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:05:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:05:57.632179144 +0000 UTC m=+705.482616644" watchObservedRunningTime="2026-07-14 14:05:57.633940023 +0000 UTC m=+705.484377526"
Jul 14 14:06:05.202693 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:05.202583    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k"
Jul 14 14:06:05.202693 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:05.202672    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k"
Jul 14 14:06:05.215787 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:05.215750    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k"
Jul 14 14:06:05.648543 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:05.648511    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k"
Jul 14 14:06:14.265040 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:14.265001    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:06:48.337413 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:48.337378    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7b78d8d56f-z795f"]
Jul 14 14:06:48.340981 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:48.340959    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b78d8d56f-z795f"
Jul 14 14:06:48.346853 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:48.346822    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7b78d8d56f-z795f"]
Jul 14 14:06:48.355539 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:48.355506    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67424\" (UniqueName: \"kubernetes.io/projected/1bc2547d-3d25-4140-b48b-5176097ca87a-kube-api-access-67424\") pod \"authorino-7b78d8d56f-z795f\" (UID: \"1bc2547d-3d25-4140-b48b-5176097ca87a\") " pod="kuadrant-system/authorino-7b78d8d56f-z795f"
Jul 14 14:06:48.355748 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:48.355556    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1bc2547d-3d25-4140-b48b-5176097ca87a-tls-cert\") pod \"authorino-7b78d8d56f-z795f\" (UID: \"1bc2547d-3d25-4140-b48b-5176097ca87a\") " pod="kuadrant-system/authorino-7b78d8d56f-z795f"
Jul 14 14:06:48.456628 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:48.456557    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67424\" (UniqueName: \"kubernetes.io/projected/1bc2547d-3d25-4140-b48b-5176097ca87a-kube-api-access-67424\") pod \"authorino-7b78d8d56f-z795f\" (UID: \"1bc2547d-3d25-4140-b48b-5176097ca87a\") " pod="kuadrant-system/authorino-7b78d8d56f-z795f"
Jul 14 14:06:48.456833 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:48.456661    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1bc2547d-3d25-4140-b48b-5176097ca87a-tls-cert\") pod \"authorino-7b78d8d56f-z795f\" (UID: \"1bc2547d-3d25-4140-b48b-5176097ca87a\") " pod="kuadrant-system/authorino-7b78d8d56f-z795f"
Jul 14 14:06:48.459163 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:48.459125    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1bc2547d-3d25-4140-b48b-5176097ca87a-tls-cert\") pod \"authorino-7b78d8d56f-z795f\" (UID: \"1bc2547d-3d25-4140-b48b-5176097ca87a\") " pod="kuadrant-system/authorino-7b78d8d56f-z795f"
Jul 14 14:06:48.465332 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:48.465304    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67424\" (UniqueName: \"kubernetes.io/projected/1bc2547d-3d25-4140-b48b-5176097ca87a-kube-api-access-67424\") pod \"authorino-7b78d8d56f-z795f\" (UID: \"1bc2547d-3d25-4140-b48b-5176097ca87a\") " pod="kuadrant-system/authorino-7b78d8d56f-z795f"
Jul 14 14:06:48.651910 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:48.651819    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b78d8d56f-z795f"
Jul 14 14:06:48.794425 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:48.794386    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:06:48.800142 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:48.800106    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7b78d8d56f-z795f"]
Jul 14 14:06:48.803224 ip-10-0-128-127 kubenswrapper[2576]: W0714 14:06:48.803193    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1bc2547d_3d25_4140_b48b_5176097ca87a.slice/crio-b1e15abce4598d20ca3fc6b67f3c5a1aea11aa01d981d9d69f1c59f1dac83621 WatchSource:0}: Error finding container b1e15abce4598d20ca3fc6b67f3c5a1aea11aa01d981d9d69f1c59f1dac83621: Status 404 returned error can't find the container with id b1e15abce4598d20ca3fc6b67f3c5a1aea11aa01d981d9d69f1c59f1dac83621
Jul 14 14:06:48.821188 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:48.821152    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b78d8d56f-z795f" event={"ID":"1bc2547d-3d25-4140-b48b-5176097ca87a","Type":"ContainerStarted","Data":"b1e15abce4598d20ca3fc6b67f3c5a1aea11aa01d981d9d69f1c59f1dac83621"}
Jul 14 14:06:48.890957 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:48.890922    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:06:49.829310 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:49.829256    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b78d8d56f-z795f" event={"ID":"1bc2547d-3d25-4140-b48b-5176097ca87a","Type":"ContainerStarted","Data":"0d64d9ae3967692f9d9feaeb708469ab29a95c907fc5986453ada51126d83d75"}
Jul 14 14:06:49.852897 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:49.852823    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7b78d8d56f-z795f" podStartSLOduration=1.430644163 podStartE2EDuration="1.85279625s" totalImagesPullingTime="422.152087ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:06:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:06:49.850836636 +0000 UTC m=+757.701274139" watchObservedRunningTime="2026-07-14 14:06:49.85279625 +0000 UTC m=+757.703233753"
Jul 14 14:06:49.884128 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:49.883363    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c6bb74b88-szn66"]
Jul 14 14:06:49.884128 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:49.883567    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-c6bb74b88-szn66" podUID="788b35c6-6202-4ace-8631-44651e2fbe7a" containerName="authorino" containerID="cri-o://b51064fc102f4f59458c33f8902f83671bda24110b256d952320ea38720fed21" gracePeriod=30
Jul 14 14:06:50.076558 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:50.076512    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:06:50.150394 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:50.150360    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c6bb74b88-szn66"
Jul 14 14:06:50.175008 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:50.174976    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/788b35c6-6202-4ace-8631-44651e2fbe7a-tls-cert\") pod \"788b35c6-6202-4ace-8631-44651e2fbe7a\" (UID: \"788b35c6-6202-4ace-8631-44651e2fbe7a\") "
Jul 14 14:06:50.175187 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:50.175104    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tm8bf\" (UniqueName: \"kubernetes.io/projected/788b35c6-6202-4ace-8631-44651e2fbe7a-kube-api-access-tm8bf\") pod \"788b35c6-6202-4ace-8631-44651e2fbe7a\" (UID: \"788b35c6-6202-4ace-8631-44651e2fbe7a\") "
Jul 14 14:06:50.177423 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:50.177325    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/788b35c6-6202-4ace-8631-44651e2fbe7a-kube-api-access-tm8bf" (OuterVolumeSpecName: "kube-api-access-tm8bf") pod "788b35c6-6202-4ace-8631-44651e2fbe7a" (UID: "788b35c6-6202-4ace-8631-44651e2fbe7a"). InnerVolumeSpecName "kube-api-access-tm8bf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 14:06:50.179877 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:50.179844    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:06:50.186556 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:50.186527    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/788b35c6-6202-4ace-8631-44651e2fbe7a-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "788b35c6-6202-4ace-8631-44651e2fbe7a" (UID: "788b35c6-6202-4ace-8631-44651e2fbe7a"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 14:06:50.276243 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:50.276201    2576 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/788b35c6-6202-4ace-8631-44651e2fbe7a-tls-cert\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:06:50.276243 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:50.276239    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tm8bf\" (UniqueName: \"kubernetes.io/projected/788b35c6-6202-4ace-8631-44651e2fbe7a-kube-api-access-tm8bf\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:06:50.834684 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:50.834652    2576 generic.go:356] "Generic (PLEG): container finished" podID="788b35c6-6202-4ace-8631-44651e2fbe7a" containerID="b51064fc102f4f59458c33f8902f83671bda24110b256d952320ea38720fed21" exitCode=0
Jul 14 14:06:50.835192 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:50.834700    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c6bb74b88-szn66"
Jul 14 14:06:50.835192 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:50.834733    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c6bb74b88-szn66" event={"ID":"788b35c6-6202-4ace-8631-44651e2fbe7a","Type":"ContainerDied","Data":"b51064fc102f4f59458c33f8902f83671bda24110b256d952320ea38720fed21"}
Jul 14 14:06:50.835192 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:50.834775    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c6bb74b88-szn66" event={"ID":"788b35c6-6202-4ace-8631-44651e2fbe7a","Type":"ContainerDied","Data":"b76cf95ba59e20926c2d6bc2484e071110f4f9f456c13eeb480709b389504074"}
Jul 14 14:06:50.835192 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:50.834793    2576 scope.go:117] "RemoveContainer" containerID="b51064fc102f4f59458c33f8902f83671bda24110b256d952320ea38720fed21"
Jul 14 14:06:50.846127 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:50.846107    2576 scope.go:117] "RemoveContainer" containerID="b51064fc102f4f59458c33f8902f83671bda24110b256d952320ea38720fed21"
Jul 14 14:06:50.846515 ip-10-0-128-127 kubenswrapper[2576]: E0714 14:06:50.846485    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b51064fc102f4f59458c33f8902f83671bda24110b256d952320ea38720fed21\": container with ID starting with b51064fc102f4f59458c33f8902f83671bda24110b256d952320ea38720fed21 not found: ID does not exist" containerID="b51064fc102f4f59458c33f8902f83671bda24110b256d952320ea38720fed21"
Jul 14 14:06:50.846558 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:50.846524    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b51064fc102f4f59458c33f8902f83671bda24110b256d952320ea38720fed21"} err="failed to get container status \"b51064fc102f4f59458c33f8902f83671bda24110b256d952320ea38720fed21\": rpc error: code = NotFound desc = could not find container \"b51064fc102f4f59458c33f8902f83671bda24110b256d952320ea38720fed21\": container with ID starting with b51064fc102f4f59458c33f8902f83671bda24110b256d952320ea38720fed21 not found: ID does not exist"
Jul 14 14:06:50.851358 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:50.851315    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c6bb74b88-szn66"]
Jul 14 14:06:50.857661 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:50.857630    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c6bb74b88-szn66"]
Jul 14 14:06:52.688380 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:06:52.688336    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="788b35c6-6202-4ace-8631-44651e2fbe7a" path="/var/lib/kubelet/pods/788b35c6-6202-4ace-8631-44651e2fbe7a/volumes"
Jul 14 14:07:18.019965 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:07:18.019927    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:07:28.005655 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:07:28.005619    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:07:32.906436 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:07:32.906400    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:07:38.714208 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:07:38.714127    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:07:41.767650 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:07:41.767620    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:07:41.867904 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:07:41.867853    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:07:49.116564 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:07:49.116505    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:07:58.816021 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:07:58.815986    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:08:01.783428 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:08:01.783373    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:08:03.492026 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:08:03.491990    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:08:27.919581 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:08:27.919545    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:08:38.908098 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:08:38.908055    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:08:44.614412 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:08:44.614366    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:08:44.710681 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:08:44.710643    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:08:44.869398 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:08:44.869319    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:08:51.342173 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:08:51.342130    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:08:51.444445 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:08:51.444402    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:08:51.548316 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:08:51.548279    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:08:51.760370 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:08:51.760279    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:08:57.724196 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:08:57.724158    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:08:57.782351 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:08:57.782306    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:09:12.583553 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:09:12.583522    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sh6rt_5600e3e1-e775-482e-bc72-76d732552705/ovn-acl-logging/0.log"
Jul 14 14:09:12.584121 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:09:12.583840    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sh6rt_5600e3e1-e775-482e-bc72-76d732552705/ovn-acl-logging/0.log"
Jul 14 14:09:14.621698 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:09:14.621656    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:09:24.310554 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:09:24.310515    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:09:33.944713 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:09:33.944671    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:09:44.566788 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:09:44.566754    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:09:54.419501 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:09:54.419459    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:10:10.213202 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:10:10.213161    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:10:16.914639 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:10:16.914577    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:10:24.429247 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:10:24.429208    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:10:47.315931 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:10:47.315846    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:10:56.813823 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:10:56.813784    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:10:59.260668 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:10:59.260635    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:11:21.322128 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:11:21.322090    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:11:31.529309 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:11:31.529270    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:11:48.014436 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:11:48.014392    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:11:58.225650 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:11:58.225615    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:12:02.152668 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:12:02.152630    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:12:09.827166 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:12:09.827122    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:12:18.607515 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:12:18.607467    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:12:27.714832 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:12:27.714796    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:14:12.608276 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:14:12.608247    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sh6rt_5600e3e1-e775-482e-bc72-76d732552705/ovn-acl-logging/0.log"
Jul 14 14:14:12.609444 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:14:12.609422    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sh6rt_5600e3e1-e775-482e-bc72-76d732552705/ovn-acl-logging/0.log"
Jul 14 14:14:15.619626 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:14:15.619572    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:14:27.215031 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:14:27.214991    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:14:33.953065 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:14:33.953024    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:14:35.436638 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:14:35.436604    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:14:48.022024 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:14:48.021974    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:14:59.415801 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:14:59.415764    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:15:10.909573 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:15:10.909534    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:15:18.733535 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:15:18.733486    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:15:29.615683 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:15:29.615641    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:15:41.319166 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:15:41.319129    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:15:49.914698 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:15:49.914656    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:16:04.312352 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:16:04.312315    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:16:20.824285 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:16:20.824248    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:16:30.117672 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:16:30.117623    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:16:40.130194 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:16:40.130103    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:16:51.219811 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:16:51.219770    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:17:08.624477 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:17:08.624435    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:17:18.310459 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:17:18.310416    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:17:27.716647 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:17:27.716607    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:17:38.210156 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:17:38.210108    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:17:47.618725 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:17:47.618687    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:17:56.513673 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:17:56.513634    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:18:10.224287 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:18:10.224245    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:18:18.828315 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:18:18.828269    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:18:28.511180 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:18:28.511119    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:18:40.421808 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:18:40.421769    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:18:52.715944 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:18:52.715908    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:19:01.616129 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:19:01.616081    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:19:12.633150 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:19:12.633118    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sh6rt_5600e3e1-e775-482e-bc72-76d732552705/ovn-acl-logging/0.log"
Jul 14 14:19:12.634545 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:19:12.634526    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sh6rt_5600e3e1-e775-482e-bc72-76d732552705/ovn-acl-logging/0.log"
Jul 14 14:19:13.513665 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:19:13.513630    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:19:25.306421 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:19:25.306382    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:19:38.414768 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:19:38.414683    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:19:44.618374 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:19:44.618332    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:19:46.313714 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:19:46.313674    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:19:47.974128 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:19:47.974083    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:19:50.403511 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:19:50.403474    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:19:50.507412 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:19:50.507376    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:19:50.631399 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:19:50.631342    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:19:50.700919 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:19:50.700811    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:19:51.819489 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:19:51.819436    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:19:51.909247 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:19:51.909209    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:19:52.065613 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:19:52.065561    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:19:52.592931 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:19:52.592902    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:19:52.733914 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:19:52.733879    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:19:54.529550 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:19:54.529506    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:19:54.711075 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:19:54.711036    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:20:27.921657 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:20:27.921585    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:20:57.645870 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:20:57.645830    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:21:00.407658 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:00.407622    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:21:02.192161 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:02.192122    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:21:08.296953 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:08.296860    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:21:08.392974 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:08.392934    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:21:12.633617 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:12.633555    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:21:13.552001 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:13.551961    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:21:13.653944 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:13.653903    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:21:13.914317 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:13.914285    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:21:15.201049 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:15.201011    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:21:15.267358 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:15.267323    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-26a3f6d9-54fc968657-k4t9n"]
Jul 14 14:21:15.267671 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:15.267655    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="788b35c6-6202-4ace-8631-44651e2fbe7a" containerName="authorino"
Jul 14 14:21:15.267671 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:15.267671    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="788b35c6-6202-4ace-8631-44651e2fbe7a" containerName="authorino"
Jul 14 14:21:15.267809 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:15.267733    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="788b35c6-6202-4ace-8631-44651e2fbe7a" containerName="authorino"
Jul 14 14:21:15.270833 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:15.270806    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-26a3f6d9-54fc968657-k4t9n"
Jul 14 14:21:15.273479 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:15.273451    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-26a3f6d9-serving-cert\""
Jul 14 14:21:15.273679 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:15.273619    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 14 14:21:15.273896 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:15.273871    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-znm9j\""
Jul 14 14:21:15.274664 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:15.274643    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 14:21:15.282484 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:15.282454    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-26a3f6d9-54fc968657-k4t9n"]
Jul 14 14:21:15.317325 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:15.317294    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:21:15.331205 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:15.331171    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3abd403-73a1-4a21-a957-918b52f95ab1-openshift-service-ca\") pod \"maas-api-e2e-migrate-26a3f6d9-54fc968657-k4t9n\" (UID: \"d3abd403-73a1-4a21-a957-918b52f95ab1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-26a3f6d9-54fc968657-k4t9n"
Jul 14 14:21:15.331405 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:15.331282    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3abd403-73a1-4a21-a957-918b52f95ab1-maas-api-tls\") pod \"maas-api-e2e-migrate-26a3f6d9-54fc968657-k4t9n\" (UID: \"d3abd403-73a1-4a21-a957-918b52f95ab1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-26a3f6d9-54fc968657-k4t9n"
Jul 14 14:21:15.331405 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:15.331374    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbbxx\" (UniqueName: \"kubernetes.io/projected/d3abd403-73a1-4a21-a957-918b52f95ab1-kube-api-access-tbbxx\") pod \"maas-api-e2e-migrate-26a3f6d9-54fc968657-k4t9n\" (UID: \"d3abd403-73a1-4a21-a957-918b52f95ab1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-26a3f6d9-54fc968657-k4t9n"
Jul 14 14:21:15.432452 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:15.432410    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3abd403-73a1-4a21-a957-918b52f95ab1-maas-api-tls\") pod \"maas-api-e2e-migrate-26a3f6d9-54fc968657-k4t9n\" (UID: \"d3abd403-73a1-4a21-a957-918b52f95ab1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-26a3f6d9-54fc968657-k4t9n"
Jul 14 14:21:15.432681 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:15.432469    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbbxx\" (UniqueName: \"kubernetes.io/projected/d3abd403-73a1-4a21-a957-918b52f95ab1-kube-api-access-tbbxx\") pod \"maas-api-e2e-migrate-26a3f6d9-54fc968657-k4t9n\" (UID: \"d3abd403-73a1-4a21-a957-918b52f95ab1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-26a3f6d9-54fc968657-k4t9n"
Jul 14 14:21:15.432681 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:15.432494    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3abd403-73a1-4a21-a957-918b52f95ab1-openshift-service-ca\") pod \"maas-api-e2e-migrate-26a3f6d9-54fc968657-k4t9n\" (UID: \"d3abd403-73a1-4a21-a957-918b52f95ab1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-26a3f6d9-54fc968657-k4t9n"
Jul 14 14:21:15.432681 ip-10-0-128-127 kubenswrapper[2576]: E0714 14:21:15.432584    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-26a3f6d9-serving-cert: secret "maas-api-e2e-migrate-26a3f6d9-serving-cert" not found
Jul 14 14:21:15.432861 ip-10-0-128-127 kubenswrapper[2576]: E0714 14:21:15.432694    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3abd403-73a1-4a21-a957-918b52f95ab1-maas-api-tls podName:d3abd403-73a1-4a21-a957-918b52f95ab1 nodeName:}" failed. No retries permitted until 2026-07-14 14:21:15.932671398 +0000 UTC m=+1623.783108903 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d3abd403-73a1-4a21-a957-918b52f95ab1-maas-api-tls") pod "maas-api-e2e-migrate-26a3f6d9-54fc968657-k4t9n" (UID: "d3abd403-73a1-4a21-a957-918b52f95ab1") : secret "maas-api-e2e-migrate-26a3f6d9-serving-cert" not found
Jul 14 14:21:15.433172 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:15.433152    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3abd403-73a1-4a21-a957-918b52f95ab1-openshift-service-ca\") pod \"maas-api-e2e-migrate-26a3f6d9-54fc968657-k4t9n\" (UID: \"d3abd403-73a1-4a21-a957-918b52f95ab1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-26a3f6d9-54fc968657-k4t9n"
Jul 14 14:21:15.441516 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:15.441480    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbbxx\" (UniqueName: \"kubernetes.io/projected/d3abd403-73a1-4a21-a957-918b52f95ab1-kube-api-access-tbbxx\") pod \"maas-api-e2e-migrate-26a3f6d9-54fc968657-k4t9n\" (UID: \"d3abd403-73a1-4a21-a957-918b52f95ab1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-26a3f6d9-54fc968657-k4t9n"
Jul 14 14:21:15.651829 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:15.651795    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:21:15.818568 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:15.818535    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:21:15.920458 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:15.920359    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:21:15.936685 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:15.936634    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3abd403-73a1-4a21-a957-918b52f95ab1-maas-api-tls\") pod \"maas-api-e2e-migrate-26a3f6d9-54fc968657-k4t9n\" (UID: \"d3abd403-73a1-4a21-a957-918b52f95ab1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-26a3f6d9-54fc968657-k4t9n"
Jul 14 14:21:15.939124 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:15.939098    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3abd403-73a1-4a21-a957-918b52f95ab1-maas-api-tls\") pod \"maas-api-e2e-migrate-26a3f6d9-54fc968657-k4t9n\" (UID: \"d3abd403-73a1-4a21-a957-918b52f95ab1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-26a3f6d9-54fc968657-k4t9n"
Jul 14 14:21:16.184400 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:16.184290    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-26a3f6d9-54fc968657-k4t9n"
Jul 14 14:21:16.329651 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:16.329618    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-26a3f6d9-54fc968657-k4t9n"]
Jul 14 14:21:16.332317 ip-10-0-128-127 kubenswrapper[2576]: W0714 14:21:16.332288    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3abd403_73a1_4a21_a957_918b52f95ab1.slice/crio-555ac386071bf1fe4f5cec464a700eb5839368d3eeaac2fbf5e98145fe39e475 WatchSource:0}: Error finding container 555ac386071bf1fe4f5cec464a700eb5839368d3eeaac2fbf5e98145fe39e475: Status 404 returned error can't find the container with id 555ac386071bf1fe4f5cec464a700eb5839368d3eeaac2fbf5e98145fe39e475
Jul 14 14:21:16.333697 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:16.333677    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 14:21:16.625419 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:16.625383    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-26a3f6d9-54fc968657-k4t9n" event={"ID":"d3abd403-73a1-4a21-a957-918b52f95ab1","Type":"ContainerStarted","Data":"555ac386071bf1fe4f5cec464a700eb5839368d3eeaac2fbf5e98145fe39e475"}
Jul 14 14:21:18.307907 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:18.307871    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:21:19.396702 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:19.396668    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:21:19.644814 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:19.644717    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-26a3f6d9-54fc968657-k4t9n" event={"ID":"d3abd403-73a1-4a21-a957-918b52f95ab1","Type":"ContainerStarted","Data":"13e7188238609934b2fd74ab8a3560483ef944c0168e8e81f4003b25b27f942f"}
Jul 14 14:21:19.644976 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:19.644828    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-26a3f6d9-54fc968657-k4t9n"
Jul 14 14:21:19.653042 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:19.653000    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:21:19.663915 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:19.663852    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-26a3f6d9-54fc968657-k4t9n" podStartSLOduration=2.102007249 podStartE2EDuration="4.663836327s" totalImagesPullingTime="2.561829078s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:21:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:21:19.661862758 +0000 UTC m=+1627.512300282" watchObservedRunningTime="2026-07-14 14:21:19.663836327 +0000 UTC m=+1627.514273830"
Jul 14 14:21:22.838319 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:22.838277    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:21:25.654795 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:21:25.654766    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-26a3f6d9-54fc968657-k4t9n"
Jul 14 14:22:20.492711 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:20.492669    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:22:20.635734 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:20.635690    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:22:20.835974 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:20.835933    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-26a3f6d9-54fc968657-k4t9n"]
Jul 14 14:22:20.836232 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:20.836186    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-26a3f6d9-54fc968657-k4t9n" podUID="d3abd403-73a1-4a21-a957-918b52f95ab1" containerName="maas-api" containerID="cri-o://13e7188238609934b2fd74ab8a3560483ef944c0168e8e81f4003b25b27f942f" gracePeriod=30
Jul 14 14:22:20.882459 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:20.882415    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:22:21.077568 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:21.077543    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-26a3f6d9-54fc968657-k4t9n"
Jul 14 14:22:21.225146 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:21.225052    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tbbxx\" (UniqueName: \"kubernetes.io/projected/d3abd403-73a1-4a21-a957-918b52f95ab1-kube-api-access-tbbxx\") pod \"d3abd403-73a1-4a21-a957-918b52f95ab1\" (UID: \"d3abd403-73a1-4a21-a957-918b52f95ab1\") "
Jul 14 14:22:21.225295 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:21.225186    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3abd403-73a1-4a21-a957-918b52f95ab1-openshift-service-ca\") pod \"d3abd403-73a1-4a21-a957-918b52f95ab1\" (UID: \"d3abd403-73a1-4a21-a957-918b52f95ab1\") "
Jul 14 14:22:21.225295 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:21.225234    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3abd403-73a1-4a21-a957-918b52f95ab1-maas-api-tls\") pod \"d3abd403-73a1-4a21-a957-918b52f95ab1\" (UID: \"d3abd403-73a1-4a21-a957-918b52f95ab1\") "
Jul 14 14:22:21.225543 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:21.225505    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3abd403-73a1-4a21-a957-918b52f95ab1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d3abd403-73a1-4a21-a957-918b52f95ab1" (UID: "d3abd403-73a1-4a21-a957-918b52f95ab1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 14:22:21.227455 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:21.227425    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3abd403-73a1-4a21-a957-918b52f95ab1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d3abd403-73a1-4a21-a957-918b52f95ab1" (UID: "d3abd403-73a1-4a21-a957-918b52f95ab1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 14:22:21.227550 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:21.227466    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3abd403-73a1-4a21-a957-918b52f95ab1-kube-api-access-tbbxx" (OuterVolumeSpecName: "kube-api-access-tbbxx") pod "d3abd403-73a1-4a21-a957-918b52f95ab1" (UID: "d3abd403-73a1-4a21-a957-918b52f95ab1"). InnerVolumeSpecName "kube-api-access-tbbxx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 14:22:21.326889 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:21.326853    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3abd403-73a1-4a21-a957-918b52f95ab1-maas-api-tls\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:22:21.326889 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:21.326889    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tbbxx\" (UniqueName: \"kubernetes.io/projected/d3abd403-73a1-4a21-a957-918b52f95ab1-kube-api-access-tbbxx\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:22:21.327087 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:21.326899    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3abd403-73a1-4a21-a957-918b52f95ab1-openshift-service-ca\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:22:21.925346 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:21.925306    2576 generic.go:356] "Generic (PLEG): container finished" podID="d3abd403-73a1-4a21-a957-918b52f95ab1" containerID="13e7188238609934b2fd74ab8a3560483ef944c0168e8e81f4003b25b27f942f" exitCode=0
Jul 14 14:22:21.925346 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:21.925348    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-26a3f6d9-54fc968657-k4t9n"
Jul 14 14:22:21.925863 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:21.925388    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-26a3f6d9-54fc968657-k4t9n" event={"ID":"d3abd403-73a1-4a21-a957-918b52f95ab1","Type":"ContainerDied","Data":"13e7188238609934b2fd74ab8a3560483ef944c0168e8e81f4003b25b27f942f"}
Jul 14 14:22:21.925863 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:21.925446    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-26a3f6d9-54fc968657-k4t9n" event={"ID":"d3abd403-73a1-4a21-a957-918b52f95ab1","Type":"ContainerDied","Data":"555ac386071bf1fe4f5cec464a700eb5839368d3eeaac2fbf5e98145fe39e475"}
Jul 14 14:22:21.925863 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:21.925482    2576 scope.go:117] "RemoveContainer" containerID="13e7188238609934b2fd74ab8a3560483ef944c0168e8e81f4003b25b27f942f"
Jul 14 14:22:21.934793 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:21.934771    2576 scope.go:117] "RemoveContainer" containerID="13e7188238609934b2fd74ab8a3560483ef944c0168e8e81f4003b25b27f942f"
Jul 14 14:22:21.935081 ip-10-0-128-127 kubenswrapper[2576]: E0714 14:22:21.935060    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13e7188238609934b2fd74ab8a3560483ef944c0168e8e81f4003b25b27f942f\": container with ID starting with 13e7188238609934b2fd74ab8a3560483ef944c0168e8e81f4003b25b27f942f not found: ID does not exist" containerID="13e7188238609934b2fd74ab8a3560483ef944c0168e8e81f4003b25b27f942f"
Jul 14 14:22:21.935148 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:21.935090    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13e7188238609934b2fd74ab8a3560483ef944c0168e8e81f4003b25b27f942f"} err="failed to get container status \"13e7188238609934b2fd74ab8a3560483ef944c0168e8e81f4003b25b27f942f\": rpc error: code = NotFound desc = could not find container \"13e7188238609934b2fd74ab8a3560483ef944c0168e8e81f4003b25b27f942f\": container with ID starting with 13e7188238609934b2fd74ab8a3560483ef944c0168e8e81f4003b25b27f942f not found: ID does not exist"
Jul 14 14:22:21.946524 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:21.946490    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-26a3f6d9-54fc968657-k4t9n"]
Jul 14 14:22:21.950683 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:21.950655    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-26a3f6d9-54fc968657-k4t9n"]
Jul 14 14:22:22.688315 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:22.688277    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3abd403-73a1-4a21-a957-918b52f95ab1" path="/var/lib/kubelet/pods/d3abd403-73a1-4a21-a957-918b52f95ab1/volumes"
Jul 14 14:22:23.318220 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:23.318181    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:22:27.029782 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:27.029737    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:22:27.830438 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:27.830396    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:22:27.925100 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:27.925062    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:22:28.943508 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:28.943447    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:22:29.020047 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:29.020011    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6f52b160-6656b5c9f-z5q9v"]
Jul 14 14:22:29.020528 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:29.020511    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3abd403-73a1-4a21-a957-918b52f95ab1" containerName="maas-api"
Jul 14 14:22:29.020613 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:29.020530    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3abd403-73a1-4a21-a957-918b52f95ab1" containerName="maas-api"
Jul 14 14:22:29.020659 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:29.020626    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3abd403-73a1-4a21-a957-918b52f95ab1" containerName="maas-api"
Jul 14 14:22:29.025256 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:29.025226    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6f52b160-6656b5c9f-z5q9v"
Jul 14 14:22:29.028302 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:29.028268    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 14 14:22:29.028443 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:29.028334    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-znm9j\""
Jul 14 14:22:29.029175 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:29.029157    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 14:22:29.029292 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:29.029158    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-6f52b160-serving-cert\""
Jul 14 14:22:29.033893 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:29.033862    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6f52b160-6656b5c9f-z5q9v"]
Jul 14 14:22:29.081354 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:29.081304    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:22:29.102820 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:29.102780    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f88b920e-1381-4fed-a482-02f3d2ae7c48-maas-api-tls\") pod \"maas-api-e2e-ait-6f52b160-6656b5c9f-z5q9v\" (UID: \"f88b920e-1381-4fed-a482-02f3d2ae7c48\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6f52b160-6656b5c9f-z5q9v"
Jul 14 14:22:29.103009 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:29.102845    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqgkx\" (UniqueName: \"kubernetes.io/projected/f88b920e-1381-4fed-a482-02f3d2ae7c48-kube-api-access-bqgkx\") pod \"maas-api-e2e-ait-6f52b160-6656b5c9f-z5q9v\" (UID: \"f88b920e-1381-4fed-a482-02f3d2ae7c48\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6f52b160-6656b5c9f-z5q9v"
Jul 14 14:22:29.103009 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:29.102899    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f88b920e-1381-4fed-a482-02f3d2ae7c48-openshift-service-ca\") pod \"maas-api-e2e-ait-6f52b160-6656b5c9f-z5q9v\" (UID: \"f88b920e-1381-4fed-a482-02f3d2ae7c48\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6f52b160-6656b5c9f-z5q9v"
Jul 14 14:22:29.203833 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:29.203736    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f88b920e-1381-4fed-a482-02f3d2ae7c48-maas-api-tls\") pod \"maas-api-e2e-ait-6f52b160-6656b5c9f-z5q9v\" (UID: \"f88b920e-1381-4fed-a482-02f3d2ae7c48\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6f52b160-6656b5c9f-z5q9v"
Jul 14 14:22:29.203833 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:29.203815    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqgkx\" (UniqueName: \"kubernetes.io/projected/f88b920e-1381-4fed-a482-02f3d2ae7c48-kube-api-access-bqgkx\") pod \"maas-api-e2e-ait-6f52b160-6656b5c9f-z5q9v\" (UID: \"f88b920e-1381-4fed-a482-02f3d2ae7c48\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6f52b160-6656b5c9f-z5q9v"
Jul 14 14:22:29.204052 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:29.203870    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f88b920e-1381-4fed-a482-02f3d2ae7c48-openshift-service-ca\") pod \"maas-api-e2e-ait-6f52b160-6656b5c9f-z5q9v\" (UID: \"f88b920e-1381-4fed-a482-02f3d2ae7c48\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6f52b160-6656b5c9f-z5q9v"
Jul 14 14:22:29.204052 ip-10-0-128-127 kubenswrapper[2576]: E0714 14:22:29.203891    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-6f52b160-serving-cert: secret "maas-api-e2e-ait-6f52b160-serving-cert" not found
Jul 14 14:22:29.204052 ip-10-0-128-127 kubenswrapper[2576]: E0714 14:22:29.203967    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f88b920e-1381-4fed-a482-02f3d2ae7c48-maas-api-tls podName:f88b920e-1381-4fed-a482-02f3d2ae7c48 nodeName:}" failed. No retries permitted until 2026-07-14 14:22:29.703943685 +0000 UTC m=+1697.554381176 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f88b920e-1381-4fed-a482-02f3d2ae7c48-maas-api-tls") pod "maas-api-e2e-ait-6f52b160-6656b5c9f-z5q9v" (UID: "f88b920e-1381-4fed-a482-02f3d2ae7c48") : secret "maas-api-e2e-ait-6f52b160-serving-cert" not found
Jul 14 14:22:29.204546 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:29.204524    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f88b920e-1381-4fed-a482-02f3d2ae7c48-openshift-service-ca\") pod \"maas-api-e2e-ait-6f52b160-6656b5c9f-z5q9v\" (UID: \"f88b920e-1381-4fed-a482-02f3d2ae7c48\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6f52b160-6656b5c9f-z5q9v"
Jul 14 14:22:29.209894 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:29.209857    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:22:29.223474 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:29.223440    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqgkx\" (UniqueName: \"kubernetes.io/projected/f88b920e-1381-4fed-a482-02f3d2ae7c48-kube-api-access-bqgkx\") pod \"maas-api-e2e-ait-6f52b160-6656b5c9f-z5q9v\" (UID: \"f88b920e-1381-4fed-a482-02f3d2ae7c48\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6f52b160-6656b5c9f-z5q9v"
Jul 14 14:22:29.709280 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:29.709241    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f88b920e-1381-4fed-a482-02f3d2ae7c48-maas-api-tls\") pod \"maas-api-e2e-ait-6f52b160-6656b5c9f-z5q9v\" (UID: \"f88b920e-1381-4fed-a482-02f3d2ae7c48\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6f52b160-6656b5c9f-z5q9v"
Jul 14 14:22:29.711908 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:29.711875    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f88b920e-1381-4fed-a482-02f3d2ae7c48-maas-api-tls\") pod \"maas-api-e2e-ait-6f52b160-6656b5c9f-z5q9v\" (UID: \"f88b920e-1381-4fed-a482-02f3d2ae7c48\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6f52b160-6656b5c9f-z5q9v"
Jul 14 14:22:29.828479 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:29.828439    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:22:29.918920 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:29.918885    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:22:29.937423 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:29.937384    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6f52b160-6656b5c9f-z5q9v"
Jul 14 14:22:30.101522 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:30.101488    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6f52b160-6656b5c9f-z5q9v"]
Jul 14 14:22:30.105020 ip-10-0-128-127 kubenswrapper[2576]: W0714 14:22:30.104985    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf88b920e_1381_4fed_a482_02f3d2ae7c48.slice/crio-777ef153a87532a378c92c7705f0c9d37ab1ff382e942d551bf0fc111affdce4 WatchSource:0}: Error finding container 777ef153a87532a378c92c7705f0c9d37ab1ff382e942d551bf0fc111affdce4: Status 404 returned error can't find the container with id 777ef153a87532a378c92c7705f0c9d37ab1ff382e942d551bf0fc111affdce4
Jul 14 14:22:30.974530 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:30.974490    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6f52b160-6656b5c9f-z5q9v" event={"ID":"f88b920e-1381-4fed-a482-02f3d2ae7c48","Type":"ContainerStarted","Data":"9ab1ecb295ffb38adce8d883b09e6cef95328f0d03c8d234b8bcd341d1cabab4"}
Jul 14 14:22:30.974740 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:30.974553    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6f52b160-6656b5c9f-z5q9v" event={"ID":"f88b920e-1381-4fed-a482-02f3d2ae7c48","Type":"ContainerStarted","Data":"777ef153a87532a378c92c7705f0c9d37ab1ff382e942d551bf0fc111affdce4"}
Jul 14 14:22:30.974740 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:30.974647    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6f52b160-6656b5c9f-z5q9v"
Jul 14 14:22:30.993083 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:30.993020    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6f52b160-6656b5c9f-z5q9v" podStartSLOduration=1.309973016 podStartE2EDuration="1.992999874s" totalImagesPullingTime="683.026858ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:22:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:22:30.992759072 +0000 UTC m=+1698.843196598" watchObservedRunningTime="2026-07-14 14:22:30.992999874 +0000 UTC m=+1698.843437379"
Jul 14 14:22:31.442481 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:31.442438    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:22:32.815116 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:32.815073    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:22:36.985244 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:36.985169    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6f52b160-6656b5c9f-z5q9v"
Jul 14 14:22:44.790613 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:44.790559    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6f52b160-6656b5c9f-z5q9v"]
Jul 14 14:22:44.791018 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:44.790814    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6f52b160-6656b5c9f-z5q9v" podUID="f88b920e-1381-4fed-a482-02f3d2ae7c48" containerName="maas-api" containerID="cri-o://9ab1ecb295ffb38adce8d883b09e6cef95328f0d03c8d234b8bcd341d1cabab4" gracePeriod=30
Jul 14 14:22:44.887621 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:44.887559    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:22:45.035990 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:45.035956    2576 generic.go:356] "Generic (PLEG): container finished" podID="f88b920e-1381-4fed-a482-02f3d2ae7c48" containerID="9ab1ecb295ffb38adce8d883b09e6cef95328f0d03c8d234b8bcd341d1cabab4" exitCode=0
Jul 14 14:22:45.036155 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:45.036030    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6f52b160-6656b5c9f-z5q9v" event={"ID":"f88b920e-1381-4fed-a482-02f3d2ae7c48","Type":"ContainerDied","Data":"9ab1ecb295ffb38adce8d883b09e6cef95328f0d03c8d234b8bcd341d1cabab4"}
Jul 14 14:22:45.036155 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:45.036093    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6f52b160-6656b5c9f-z5q9v" event={"ID":"f88b920e-1381-4fed-a482-02f3d2ae7c48","Type":"ContainerDied","Data":"777ef153a87532a378c92c7705f0c9d37ab1ff382e942d551bf0fc111affdce4"}
Jul 14 14:22:45.036155 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:45.036111    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="777ef153a87532a378c92c7705f0c9d37ab1ff382e942d551bf0fc111affdce4"
Jul 14 14:22:45.039913 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:45.039884    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6f52b160-6656b5c9f-z5q9v"
Jul 14 14:22:45.137275 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:45.137169    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f88b920e-1381-4fed-a482-02f3d2ae7c48-openshift-service-ca\") pod \"f88b920e-1381-4fed-a482-02f3d2ae7c48\" (UID: \"f88b920e-1381-4fed-a482-02f3d2ae7c48\") "
Jul 14 14:22:45.137275 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:45.137228    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bqgkx\" (UniqueName: \"kubernetes.io/projected/f88b920e-1381-4fed-a482-02f3d2ae7c48-kube-api-access-bqgkx\") pod \"f88b920e-1381-4fed-a482-02f3d2ae7c48\" (UID: \"f88b920e-1381-4fed-a482-02f3d2ae7c48\") "
Jul 14 14:22:45.137275 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:45.137255    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f88b920e-1381-4fed-a482-02f3d2ae7c48-maas-api-tls\") pod \"f88b920e-1381-4fed-a482-02f3d2ae7c48\" (UID: \"f88b920e-1381-4fed-a482-02f3d2ae7c48\") "
Jul 14 14:22:45.137666 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:45.137640    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f88b920e-1381-4fed-a482-02f3d2ae7c48-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f88b920e-1381-4fed-a482-02f3d2ae7c48" (UID: "f88b920e-1381-4fed-a482-02f3d2ae7c48"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 14:22:45.139532 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:45.139503    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f88b920e-1381-4fed-a482-02f3d2ae7c48-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f88b920e-1381-4fed-a482-02f3d2ae7c48" (UID: "f88b920e-1381-4fed-a482-02f3d2ae7c48"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 14:22:45.139644 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:45.139512    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f88b920e-1381-4fed-a482-02f3d2ae7c48-kube-api-access-bqgkx" (OuterVolumeSpecName: "kube-api-access-bqgkx") pod "f88b920e-1381-4fed-a482-02f3d2ae7c48" (UID: "f88b920e-1381-4fed-a482-02f3d2ae7c48"). InnerVolumeSpecName "kube-api-access-bqgkx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 14:22:45.238141 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:45.238107    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f88b920e-1381-4fed-a482-02f3d2ae7c48-maas-api-tls\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:22:45.238141 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:45.238137    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f88b920e-1381-4fed-a482-02f3d2ae7c48-openshift-service-ca\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:22:45.238141 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:45.238148    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqgkx\" (UniqueName: \"kubernetes.io/projected/f88b920e-1381-4fed-a482-02f3d2ae7c48-kube-api-access-bqgkx\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:22:46.041014 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:46.040972    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6f52b160-6656b5c9f-z5q9v"
Jul 14 14:22:46.068631 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:46.068568    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6f52b160-6656b5c9f-z5q9v"]
Jul 14 14:22:46.070898 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:46.070865    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6f52b160-6656b5c9f-z5q9v"]
Jul 14 14:22:46.688372 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:46.688339    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f88b920e-1381-4fed-a482-02f3d2ae7c48" path="/var/lib/kubelet/pods/f88b920e-1381-4fed-a482-02f3d2ae7c48/volumes"
Jul 14 14:22:47.825804 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:47.825773    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:22:49.450087 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:49.450048    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:22:56.792400 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:56.792355    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:22:56.917128 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:22:56.917085    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:23:00.538428 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:00.538388    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:23:00.944267 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:00.944230    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:23:02.044831 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:02.044793    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:23:02.086081 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:02.086050    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9cad56cb-bd5dcf69f-wm4jm"]
Jul 14 14:23:02.086357 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:02.086346    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f88b920e-1381-4fed-a482-02f3d2ae7c48" containerName="maas-api"
Jul 14 14:23:02.086404 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:02.086358    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f88b920e-1381-4fed-a482-02f3d2ae7c48" containerName="maas-api"
Jul 14 14:23:02.086441 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:02.086425    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f88b920e-1381-4fed-a482-02f3d2ae7c48" containerName="maas-api"
Jul 14 14:23:02.089683 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:02.089668    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9cad56cb-bd5dcf69f-wm4jm"
Jul 14 14:23:02.092226 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:02.092209    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 14:23:02.093187 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:02.093156    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-znm9j\""
Jul 14 14:23:02.093187 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:02.093175    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 14 14:23:02.093367 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:02.093185    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-9cad56cb-serving-cert\""
Jul 14 14:23:02.103215 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:02.103186    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9cad56cb-bd5dcf69f-wm4jm"]
Jul 14 14:23:02.152031 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:02.151985    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:23:02.152246 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:02.152228    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/060429fe-871e-449c-83a2-5da60adfae26-maas-api-tls\") pod \"maas-api-e2e-derive-9cad56cb-bd5dcf69f-wm4jm\" (UID: \"060429fe-871e-449c-83a2-5da60adfae26\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9cad56cb-bd5dcf69f-wm4jm"
Jul 14 14:23:02.152302 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:02.152271    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/060429fe-871e-449c-83a2-5da60adfae26-openshift-service-ca\") pod \"maas-api-e2e-derive-9cad56cb-bd5dcf69f-wm4jm\" (UID: \"060429fe-871e-449c-83a2-5da60adfae26\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9cad56cb-bd5dcf69f-wm4jm"
Jul 14 14:23:02.152380 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:02.152365    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcvnm\" (UniqueName: \"kubernetes.io/projected/060429fe-871e-449c-83a2-5da60adfae26-kube-api-access-kcvnm\") pod \"maas-api-e2e-derive-9cad56cb-bd5dcf69f-wm4jm\" (UID: \"060429fe-871e-449c-83a2-5da60adfae26\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9cad56cb-bd5dcf69f-wm4jm"
Jul 14 14:23:02.252814 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:02.252780    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/060429fe-871e-449c-83a2-5da60adfae26-openshift-service-ca\") pod \"maas-api-e2e-derive-9cad56cb-bd5dcf69f-wm4jm\" (UID: \"060429fe-871e-449c-83a2-5da60adfae26\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9cad56cb-bd5dcf69f-wm4jm"
Jul 14 14:23:02.253003 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:02.252851    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcvnm\" (UniqueName: \"kubernetes.io/projected/060429fe-871e-449c-83a2-5da60adfae26-kube-api-access-kcvnm\") pod \"maas-api-e2e-derive-9cad56cb-bd5dcf69f-wm4jm\" (UID: \"060429fe-871e-449c-83a2-5da60adfae26\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9cad56cb-bd5dcf69f-wm4jm"
Jul 14 14:23:02.253003 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:02.252928    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/060429fe-871e-449c-83a2-5da60adfae26-maas-api-tls\") pod \"maas-api-e2e-derive-9cad56cb-bd5dcf69f-wm4jm\" (UID: \"060429fe-871e-449c-83a2-5da60adfae26\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9cad56cb-bd5dcf69f-wm4jm"
Jul 14 14:23:02.253077 ip-10-0-128-127 kubenswrapper[2576]: E0714 14:23:02.253025    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-9cad56cb-serving-cert: secret "maas-api-e2e-derive-9cad56cb-serving-cert" not found
Jul 14 14:23:02.253116 ip-10-0-128-127 kubenswrapper[2576]: E0714 14:23:02.253095    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/060429fe-871e-449c-83a2-5da60adfae26-maas-api-tls podName:060429fe-871e-449c-83a2-5da60adfae26 nodeName:}" failed. No retries permitted until 2026-07-14 14:23:02.753074951 +0000 UTC m=+1730.603512446 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/060429fe-871e-449c-83a2-5da60adfae26-maas-api-tls") pod "maas-api-e2e-derive-9cad56cb-bd5dcf69f-wm4jm" (UID: "060429fe-871e-449c-83a2-5da60adfae26") : secret "maas-api-e2e-derive-9cad56cb-serving-cert" not found
Jul 14 14:23:02.253443 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:02.253422    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/060429fe-871e-449c-83a2-5da60adfae26-openshift-service-ca\") pod \"maas-api-e2e-derive-9cad56cb-bd5dcf69f-wm4jm\" (UID: \"060429fe-871e-449c-83a2-5da60adfae26\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9cad56cb-bd5dcf69f-wm4jm"
Jul 14 14:23:02.263293 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:02.263261    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcvnm\" (UniqueName: \"kubernetes.io/projected/060429fe-871e-449c-83a2-5da60adfae26-kube-api-access-kcvnm\") pod \"maas-api-e2e-derive-9cad56cb-bd5dcf69f-wm4jm\" (UID: \"060429fe-871e-449c-83a2-5da60adfae26\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9cad56cb-bd5dcf69f-wm4jm"
Jul 14 14:23:02.755394 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:02.755352    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/060429fe-871e-449c-83a2-5da60adfae26-maas-api-tls\") pod \"maas-api-e2e-derive-9cad56cb-bd5dcf69f-wm4jm\" (UID: \"060429fe-871e-449c-83a2-5da60adfae26\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9cad56cb-bd5dcf69f-wm4jm"
Jul 14 14:23:02.758150 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:02.758124    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/060429fe-871e-449c-83a2-5da60adfae26-maas-api-tls\") pod \"maas-api-e2e-derive-9cad56cb-bd5dcf69f-wm4jm\" (UID: \"060429fe-871e-449c-83a2-5da60adfae26\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9cad56cb-bd5dcf69f-wm4jm"
Jul 14 14:23:02.763309 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:02.763275    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:23:02.923222 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:02.923176    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:23:02.999681 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:02.999650    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9cad56cb-bd5dcf69f-wm4jm"
Jul 14 14:23:03.139635 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:03.139423    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9cad56cb-bd5dcf69f-wm4jm"]
Jul 14 14:23:03.142447 ip-10-0-128-127 kubenswrapper[2576]: W0714 14:23:03.142410    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod060429fe_871e_449c_83a2_5da60adfae26.slice/crio-ac20a146349d4390ad45fc4228c8e011fb661c43c1527a757626b6edbd8fa9d7 WatchSource:0}: Error finding container ac20a146349d4390ad45fc4228c8e011fb661c43c1527a757626b6edbd8fa9d7: Status 404 returned error can't find the container with id ac20a146349d4390ad45fc4228c8e011fb661c43c1527a757626b6edbd8fa9d7
Jul 14 14:23:04.127239 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:04.127203    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9cad56cb-bd5dcf69f-wm4jm" event={"ID":"060429fe-871e-449c-83a2-5da60adfae26","Type":"ContainerStarted","Data":"36c5c79b806a3ea5880ed2ab0ffa8a07c3cbc3a51451a25fa99369eb4e5e9131"}
Jul 14 14:23:04.127421 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:04.127290    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9cad56cb-bd5dcf69f-wm4jm"
Jul 14 14:23:04.127421 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:04.127320    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9cad56cb-bd5dcf69f-wm4jm" event={"ID":"060429fe-871e-449c-83a2-5da60adfae26","Type":"ContainerStarted","Data":"ac20a146349d4390ad45fc4228c8e011fb661c43c1527a757626b6edbd8fa9d7"}
Jul 14 14:23:04.147251 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:04.147201    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9cad56cb-bd5dcf69f-wm4jm" podStartSLOduration=1.758727302 podStartE2EDuration="2.147185194s" totalImagesPullingTime="388.457892ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:23:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:23:04.146267396 +0000 UTC m=+1731.996704899" watchObservedRunningTime="2026-07-14 14:23:04.147185194 +0000 UTC m=+1731.997622696"
Jul 14 14:23:07.278427 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:07.278387    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:23:07.376789 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:07.376718    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:23:09.245949 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:09.245913    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:23:10.137049 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:10.137017    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9cad56cb-bd5dcf69f-wm4jm"
Jul 14 14:23:10.716235 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:10.716194    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:23:26.729112 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:26.729071    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:23:26.825442 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:26.825402    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:23:27.582495 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:27.582459    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:23:27.684125 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:27.684085    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:23:35.657288 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:23:35.657228    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:24:07.001059 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:07.000968    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9cad56cb-bd5dcf69f-wm4jm"]
Jul 14 14:24:07.001987 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:07.001204    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9cad56cb-bd5dcf69f-wm4jm" podUID="060429fe-871e-449c-83a2-5da60adfae26" containerName="maas-api" containerID="cri-o://36c5c79b806a3ea5880ed2ab0ffa8a07c3cbc3a51451a25fa99369eb4e5e9131" gracePeriod=30
Jul 14 14:24:07.035557 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:07.035523    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:24:07.250024 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:07.249993    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9cad56cb-bd5dcf69f-wm4jm"
Jul 14 14:24:07.329248 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:07.329216    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kcvnm\" (UniqueName: \"kubernetes.io/projected/060429fe-871e-449c-83a2-5da60adfae26-kube-api-access-kcvnm\") pod \"060429fe-871e-449c-83a2-5da60adfae26\" (UID: \"060429fe-871e-449c-83a2-5da60adfae26\") "
Jul 14 14:24:07.329448 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:07.329283    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/060429fe-871e-449c-83a2-5da60adfae26-maas-api-tls\") pod \"060429fe-871e-449c-83a2-5da60adfae26\" (UID: \"060429fe-871e-449c-83a2-5da60adfae26\") "
Jul 14 14:24:07.329448 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:07.329334    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/060429fe-871e-449c-83a2-5da60adfae26-openshift-service-ca\") pod \"060429fe-871e-449c-83a2-5da60adfae26\" (UID: \"060429fe-871e-449c-83a2-5da60adfae26\") "
Jul 14 14:24:07.329778 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:07.329747    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/060429fe-871e-449c-83a2-5da60adfae26-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "060429fe-871e-449c-83a2-5da60adfae26" (UID: "060429fe-871e-449c-83a2-5da60adfae26"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 14:24:07.331444 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:07.331409    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/060429fe-871e-449c-83a2-5da60adfae26-kube-api-access-kcvnm" (OuterVolumeSpecName: "kube-api-access-kcvnm") pod "060429fe-871e-449c-83a2-5da60adfae26" (UID: "060429fe-871e-449c-83a2-5da60adfae26"). InnerVolumeSpecName "kube-api-access-kcvnm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 14:24:07.331538 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:07.331493    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/060429fe-871e-449c-83a2-5da60adfae26-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "060429fe-871e-449c-83a2-5da60adfae26" (UID: "060429fe-871e-449c-83a2-5da60adfae26"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 14:24:07.407584 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:07.407541    2576 generic.go:356] "Generic (PLEG): container finished" podID="060429fe-871e-449c-83a2-5da60adfae26" containerID="36c5c79b806a3ea5880ed2ab0ffa8a07c3cbc3a51451a25fa99369eb4e5e9131" exitCode=0
Jul 14 14:24:07.407584 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:07.407587    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9cad56cb-bd5dcf69f-wm4jm"
Jul 14 14:24:07.407856 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:07.407632    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9cad56cb-bd5dcf69f-wm4jm" event={"ID":"060429fe-871e-449c-83a2-5da60adfae26","Type":"ContainerDied","Data":"36c5c79b806a3ea5880ed2ab0ffa8a07c3cbc3a51451a25fa99369eb4e5e9131"}
Jul 14 14:24:07.407856 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:07.407694    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9cad56cb-bd5dcf69f-wm4jm" event={"ID":"060429fe-871e-449c-83a2-5da60adfae26","Type":"ContainerDied","Data":"ac20a146349d4390ad45fc4228c8e011fb661c43c1527a757626b6edbd8fa9d7"}
Jul 14 14:24:07.407856 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:07.407736    2576 scope.go:117] "RemoveContainer" containerID="36c5c79b806a3ea5880ed2ab0ffa8a07c3cbc3a51451a25fa99369eb4e5e9131"
Jul 14 14:24:07.417202 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:07.417179    2576 scope.go:117] "RemoveContainer" containerID="36c5c79b806a3ea5880ed2ab0ffa8a07c3cbc3a51451a25fa99369eb4e5e9131"
Jul 14 14:24:07.417527 ip-10-0-128-127 kubenswrapper[2576]: E0714 14:24:07.417501    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36c5c79b806a3ea5880ed2ab0ffa8a07c3cbc3a51451a25fa99369eb4e5e9131\": container with ID starting with 36c5c79b806a3ea5880ed2ab0ffa8a07c3cbc3a51451a25fa99369eb4e5e9131 not found: ID does not exist" containerID="36c5c79b806a3ea5880ed2ab0ffa8a07c3cbc3a51451a25fa99369eb4e5e9131"
Jul 14 14:24:07.417615 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:07.417545    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36c5c79b806a3ea5880ed2ab0ffa8a07c3cbc3a51451a25fa99369eb4e5e9131"} err="failed to get container status \"36c5c79b806a3ea5880ed2ab0ffa8a07c3cbc3a51451a25fa99369eb4e5e9131\": rpc error: code = NotFound desc = could not find container \"36c5c79b806a3ea5880ed2ab0ffa8a07c3cbc3a51451a25fa99369eb4e5e9131\": container with ID starting with 36c5c79b806a3ea5880ed2ab0ffa8a07c3cbc3a51451a25fa99369eb4e5e9131 not found: ID does not exist"
Jul 14 14:24:07.430960 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:07.430889    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/060429fe-871e-449c-83a2-5da60adfae26-maas-api-tls\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:24:07.430960 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:07.430918    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/060429fe-871e-449c-83a2-5da60adfae26-openshift-service-ca\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:24:07.430960 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:07.430935    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kcvnm\" (UniqueName: \"kubernetes.io/projected/060429fe-871e-449c-83a2-5da60adfae26-kube-api-access-kcvnm\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:24:07.431432 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:07.430999    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9cad56cb-bd5dcf69f-wm4jm"]
Jul 14 14:24:07.433200 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:07.433174    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9cad56cb-bd5dcf69f-wm4jm"]
Jul 14 14:24:08.688687 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:08.688653    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="060429fe-871e-449c-83a2-5da60adfae26" path="/var/lib/kubelet/pods/060429fe-871e-449c-83a2-5da60adfae26/volumes"
Jul 14 14:24:09.611343 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:09.611309    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:24:11.657797 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:11.657758    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:24:12.660576 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:12.660541    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sh6rt_5600e3e1-e775-482e-bc72-76d732552705/ovn-acl-logging/0.log"
Jul 14 14:24:12.661624 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:12.661609    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sh6rt_5600e3e1-e775-482e-bc72-76d732552705/ovn-acl-logging/0.log"
Jul 14 14:24:17.955119 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:17.955070    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:24:18.123316 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:18.123276    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:24:21.874054 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:21.874014    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:24:22.559920 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:22.559881    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:24:22.650083 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:22.650044    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:24:22.760163 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:22.760125    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:24:24.512838 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:24.512782    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:24:28.647617 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:28.647127    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:24:31.146534 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:31.146501    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:24:31.273494 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:31.273455    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:24:31.365779 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:31.365746    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:24:31.466442 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:31.466339    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:24:32.024170 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:32.024119    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:24:58.053693 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:24:58.053646    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:25:05.197248 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:25:05.197189    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:25:05.417222 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:25:05.417187    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:25:05.559643 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:25:05.559568    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:25:22.356013 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:25:22.355972    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:25:28.750452 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:25:28.750398    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:25:28.810532 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:25:28.810479    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:25:28.922794 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:25:28.922738    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:25:29.016012 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:25:29.015890    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:25:39.541922 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:25:39.541827    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:25:40.450858 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:25:40.450821    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:25:46.435131 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:25:46.435096    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:25:46.498529 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:25:46.498487    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:25:46.592187 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:25:46.592156    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:25:46.711517 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:25:46.711414    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:25:47.779909 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:25:47.779879    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:25:47.880562 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:25:47.880528    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:25:48.986335 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:25:48.986298    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:25:49.125271 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:25:49.125234    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:25:49.651788 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:25:49.651753    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:25:55.598323 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:25:55.598260    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:25:55.672163 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:25:55.672124    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:25:56.389056 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:25:56.389011    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:25:56.725996 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:25:56.725904    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:26:02.534539 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:02.534501    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:26:02.637104 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:02.637066    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:26:02.993108 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:02.993005    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:26:03.131583 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:03.131539    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:26:05.520229 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:05.520184    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:26:06.302374 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:06.302335    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:26:06.407755 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:06.407722    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:26:06.530298 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:06.530264    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-32d459c1-85c5dbb757-lvnbl"]
Jul 14 14:26:06.530715 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:06.530615    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="060429fe-871e-449c-83a2-5da60adfae26" containerName="maas-api"
Jul 14 14:26:06.530715 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:06.530631    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="060429fe-871e-449c-83a2-5da60adfae26" containerName="maas-api"
Jul 14 14:26:06.530715 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:06.530690    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="060429fe-871e-449c-83a2-5da60adfae26" containerName="maas-api"
Jul 14 14:26:06.533750 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:06.533732    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-32d459c1-85c5dbb757-lvnbl"
Jul 14 14:26:06.536185 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:06.536160    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-32d459c1-serving-cert\""
Jul 14 14:26:06.536460 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:06.536438    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 14:26:06.536584 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:06.536568    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-znm9j\""
Jul 14 14:26:06.538005 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:06.537985    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 14 14:26:06.568482 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:06.568363    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:26:06.595826 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:06.595794    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-32d459c1-85c5dbb757-lvnbl"]
Jul 14 14:26:06.607237 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:06.607202    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cztkd\" (UniqueName: \"kubernetes.io/projected/9a47ba40-fffe-497d-8be9-ceca1e584377-kube-api-access-cztkd\") pod \"maas-api-e2e-mt-32d459c1-85c5dbb757-lvnbl\" (UID: \"9a47ba40-fffe-497d-8be9-ceca1e584377\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-32d459c1-85c5dbb757-lvnbl"
Jul 14 14:26:06.607237 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:06.607236    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a47ba40-fffe-497d-8be9-ceca1e584377-maas-api-tls\") pod \"maas-api-e2e-mt-32d459c1-85c5dbb757-lvnbl\" (UID: \"9a47ba40-fffe-497d-8be9-ceca1e584377\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-32d459c1-85c5dbb757-lvnbl"
Jul 14 14:26:06.607498 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:06.607255    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a47ba40-fffe-497d-8be9-ceca1e584377-openshift-service-ca\") pod \"maas-api-e2e-mt-32d459c1-85c5dbb757-lvnbl\" (UID: \"9a47ba40-fffe-497d-8be9-ceca1e584377\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-32d459c1-85c5dbb757-lvnbl"
Jul 14 14:26:06.708275 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:06.708239    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cztkd\" (UniqueName: \"kubernetes.io/projected/9a47ba40-fffe-497d-8be9-ceca1e584377-kube-api-access-cztkd\") pod \"maas-api-e2e-mt-32d459c1-85c5dbb757-lvnbl\" (UID: \"9a47ba40-fffe-497d-8be9-ceca1e584377\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-32d459c1-85c5dbb757-lvnbl"
Jul 14 14:26:06.708275 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:06.708278    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a47ba40-fffe-497d-8be9-ceca1e584377-maas-api-tls\") pod \"maas-api-e2e-mt-32d459c1-85c5dbb757-lvnbl\" (UID: \"9a47ba40-fffe-497d-8be9-ceca1e584377\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-32d459c1-85c5dbb757-lvnbl"
Jul 14 14:26:06.708499 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:06.708298    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a47ba40-fffe-497d-8be9-ceca1e584377-openshift-service-ca\") pod \"maas-api-e2e-mt-32d459c1-85c5dbb757-lvnbl\" (UID: \"9a47ba40-fffe-497d-8be9-ceca1e584377\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-32d459c1-85c5dbb757-lvnbl"
Jul 14 14:26:06.708499 ip-10-0-128-127 kubenswrapper[2576]: E0714 14:26:06.708421    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-32d459c1-serving-cert: secret "maas-api-e2e-mt-32d459c1-serving-cert" not found
Jul 14 14:26:06.708573 ip-10-0-128-127 kubenswrapper[2576]: E0714 14:26:06.708506    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a47ba40-fffe-497d-8be9-ceca1e584377-maas-api-tls podName:9a47ba40-fffe-497d-8be9-ceca1e584377 nodeName:}" failed. No retries permitted until 2026-07-14 14:26:07.208484392 +0000 UTC m=+1915.058921877 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9a47ba40-fffe-497d-8be9-ceca1e584377-maas-api-tls") pod "maas-api-e2e-mt-32d459c1-85c5dbb757-lvnbl" (UID: "9a47ba40-fffe-497d-8be9-ceca1e584377") : secret "maas-api-e2e-mt-32d459c1-serving-cert" not found
Jul 14 14:26:06.709071 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:06.709051    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a47ba40-fffe-497d-8be9-ceca1e584377-openshift-service-ca\") pod \"maas-api-e2e-mt-32d459c1-85c5dbb757-lvnbl\" (UID: \"9a47ba40-fffe-497d-8be9-ceca1e584377\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-32d459c1-85c5dbb757-lvnbl"
Jul 14 14:26:06.717425 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:06.717389    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cztkd\" (UniqueName: \"kubernetes.io/projected/9a47ba40-fffe-497d-8be9-ceca1e584377-kube-api-access-cztkd\") pod \"maas-api-e2e-mt-32d459c1-85c5dbb757-lvnbl\" (UID: \"9a47ba40-fffe-497d-8be9-ceca1e584377\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-32d459c1-85c5dbb757-lvnbl"
Jul 14 14:26:06.821059 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:06.820971    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:26:07.213798 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:07.213706    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a47ba40-fffe-497d-8be9-ceca1e584377-maas-api-tls\") pod \"maas-api-e2e-mt-32d459c1-85c5dbb757-lvnbl\" (UID: \"9a47ba40-fffe-497d-8be9-ceca1e584377\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-32d459c1-85c5dbb757-lvnbl"
Jul 14 14:26:07.216187 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:07.216161    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a47ba40-fffe-497d-8be9-ceca1e584377-maas-api-tls\") pod \"maas-api-e2e-mt-32d459c1-85c5dbb757-lvnbl\" (UID: \"9a47ba40-fffe-497d-8be9-ceca1e584377\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-32d459c1-85c5dbb757-lvnbl"
Jul 14 14:26:07.255790 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:07.255748    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:26:07.355859 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:07.355818    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:26:07.444721 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:07.444687    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-32d459c1-85c5dbb757-lvnbl"
Jul 14 14:26:07.586285 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:07.586258    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-32d459c1-85c5dbb757-lvnbl"]
Jul 14 14:26:07.588301 ip-10-0-128-127 kubenswrapper[2576]: W0714 14:26:07.588274    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a47ba40_fffe_497d_8be9_ceca1e584377.slice/crio-f8b154476dac0140d07b47966d4b03c5eab1066405a5ddb286d47acf081cbab7 WatchSource:0}: Error finding container f8b154476dac0140d07b47966d4b03c5eab1066405a5ddb286d47acf081cbab7: Status 404 returned error can't find the container with id f8b154476dac0140d07b47966d4b03c5eab1066405a5ddb286d47acf081cbab7
Jul 14 14:26:07.948346 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:07.948308    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-32d459c1-85c5dbb757-lvnbl" event={"ID":"9a47ba40-fffe-497d-8be9-ceca1e584377","Type":"ContainerStarted","Data":"f8b154476dac0140d07b47966d4b03c5eab1066405a5ddb286d47acf081cbab7"}
Jul 14 14:26:08.954418 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:08.954373    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-32d459c1-85c5dbb757-lvnbl" event={"ID":"9a47ba40-fffe-497d-8be9-ceca1e584377","Type":"ContainerStarted","Data":"7b4770beb0775ceb2a4616c708e037e7a2412dd7c373a1dc3eff19571de3facc"}
Jul 14 14:26:08.954820 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:08.954555    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-32d459c1-85c5dbb757-lvnbl"
Jul 14 14:26:08.994364 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:08.994311    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-32d459c1-85c5dbb757-lvnbl" podStartSLOduration=2.4283787390000002 podStartE2EDuration="2.994293435s" totalImagesPullingTime="565.914696ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:26:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:26:08.992157365 +0000 UTC m=+1916.842594867" watchObservedRunningTime="2026-07-14 14:26:08.994293435 +0000 UTC m=+1916.844730938"
Jul 14 14:26:10.746369 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:10.746329    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:26:11.446632 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:11.446553    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:26:11.538829 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:11.538794    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:26:11.910737 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:11.910692    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:26:14.965138 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:14.965109    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-32d459c1-85c5dbb757-lvnbl"
Jul 14 14:26:15.526431 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:15.526394    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:26:18.568294 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:18.568256    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:26:18.653411 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:18.653375    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:26:29.513672 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:29.513631    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-32d459c1-85c5dbb757-lvnbl"]
Jul 14 14:26:29.514822 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:29.514624    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-32d459c1-85c5dbb757-lvnbl" podUID="9a47ba40-fffe-497d-8be9-ceca1e584377" containerName="maas-api" containerID="cri-o://7b4770beb0775ceb2a4616c708e037e7a2412dd7c373a1dc3eff19571de3facc" gracePeriod=30
Jul 14 14:26:29.551252 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:29.551215    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:26:29.769822 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:29.769802    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-32d459c1-85c5dbb757-lvnbl"
Jul 14 14:26:29.930808 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:29.930768    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a47ba40-fffe-497d-8be9-ceca1e584377-openshift-service-ca\") pod \"9a47ba40-fffe-497d-8be9-ceca1e584377\" (UID: \"9a47ba40-fffe-497d-8be9-ceca1e584377\") "
Jul 14 14:26:29.930993 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:29.930845    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cztkd\" (UniqueName: \"kubernetes.io/projected/9a47ba40-fffe-497d-8be9-ceca1e584377-kube-api-access-cztkd\") pod \"9a47ba40-fffe-497d-8be9-ceca1e584377\" (UID: \"9a47ba40-fffe-497d-8be9-ceca1e584377\") "
Jul 14 14:26:29.930993 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:29.930917    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a47ba40-fffe-497d-8be9-ceca1e584377-maas-api-tls\") pod \"9a47ba40-fffe-497d-8be9-ceca1e584377\" (UID: \"9a47ba40-fffe-497d-8be9-ceca1e584377\") "
Jul 14 14:26:29.931231 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:29.931198    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a47ba40-fffe-497d-8be9-ceca1e584377-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9a47ba40-fffe-497d-8be9-ceca1e584377" (UID: "9a47ba40-fffe-497d-8be9-ceca1e584377"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 14:26:29.933016 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:29.932981    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a47ba40-fffe-497d-8be9-ceca1e584377-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9a47ba40-fffe-497d-8be9-ceca1e584377" (UID: "9a47ba40-fffe-497d-8be9-ceca1e584377"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 14:26:29.933144 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:29.933029    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a47ba40-fffe-497d-8be9-ceca1e584377-kube-api-access-cztkd" (OuterVolumeSpecName: "kube-api-access-cztkd") pod "9a47ba40-fffe-497d-8be9-ceca1e584377" (UID: "9a47ba40-fffe-497d-8be9-ceca1e584377"). InnerVolumeSpecName "kube-api-access-cztkd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 14:26:30.031804 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:30.031716    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a47ba40-fffe-497d-8be9-ceca1e584377-maas-api-tls\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:26:30.031804 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:30.031750    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a47ba40-fffe-497d-8be9-ceca1e584377-openshift-service-ca\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:26:30.031804 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:30.031760    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cztkd\" (UniqueName: \"kubernetes.io/projected/9a47ba40-fffe-497d-8be9-ceca1e584377-kube-api-access-cztkd\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:26:30.045737 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:30.045704    2576 generic.go:356] "Generic (PLEG): container finished" podID="9a47ba40-fffe-497d-8be9-ceca1e584377" containerID="7b4770beb0775ceb2a4616c708e037e7a2412dd7c373a1dc3eff19571de3facc" exitCode=0
Jul 14 14:26:30.045895 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:30.045750    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-32d459c1-85c5dbb757-lvnbl"
Jul 14 14:26:30.045895 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:30.045785    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-32d459c1-85c5dbb757-lvnbl" event={"ID":"9a47ba40-fffe-497d-8be9-ceca1e584377","Type":"ContainerDied","Data":"7b4770beb0775ceb2a4616c708e037e7a2412dd7c373a1dc3eff19571de3facc"}
Jul 14 14:26:30.045895 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:30.045838    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-32d459c1-85c5dbb757-lvnbl" event={"ID":"9a47ba40-fffe-497d-8be9-ceca1e584377","Type":"ContainerDied","Data":"f8b154476dac0140d07b47966d4b03c5eab1066405a5ddb286d47acf081cbab7"}
Jul 14 14:26:30.045895 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:30.045860    2576 scope.go:117] "RemoveContainer" containerID="7b4770beb0775ceb2a4616c708e037e7a2412dd7c373a1dc3eff19571de3facc"
Jul 14 14:26:30.055201 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:30.055182    2576 scope.go:117] "RemoveContainer" containerID="7b4770beb0775ceb2a4616c708e037e7a2412dd7c373a1dc3eff19571de3facc"
Jul 14 14:26:30.055509 ip-10-0-128-127 kubenswrapper[2576]: E0714 14:26:30.055488    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b4770beb0775ceb2a4616c708e037e7a2412dd7c373a1dc3eff19571de3facc\": container with ID starting with 7b4770beb0775ceb2a4616c708e037e7a2412dd7c373a1dc3eff19571de3facc not found: ID does not exist" containerID="7b4770beb0775ceb2a4616c708e037e7a2412dd7c373a1dc3eff19571de3facc"
Jul 14 14:26:30.055574 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:30.055523    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b4770beb0775ceb2a4616c708e037e7a2412dd7c373a1dc3eff19571de3facc"} err="failed to get container status \"7b4770beb0775ceb2a4616c708e037e7a2412dd7c373a1dc3eff19571de3facc\": rpc error: code = NotFound desc = could not find container \"7b4770beb0775ceb2a4616c708e037e7a2412dd7c373a1dc3eff19571de3facc\": container with ID starting with 7b4770beb0775ceb2a4616c708e037e7a2412dd7c373a1dc3eff19571de3facc not found: ID does not exist"
Jul 14 14:26:30.069130 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:30.069097    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-32d459c1-85c5dbb757-lvnbl"]
Jul 14 14:26:30.072924 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:30.072898    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-32d459c1-85c5dbb757-lvnbl"]
Jul 14 14:26:30.688857 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:30.688820    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a47ba40-fffe-497d-8be9-ceca1e584377" path="/var/lib/kubelet/pods/9a47ba40-fffe-497d-8be9-ceca1e584377/volumes"
Jul 14 14:26:32.403175 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:32.403136    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:26:34.100800 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:34.100754    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:26:40.988026 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:40.987986    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:26:41.089297 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:41.089267    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:26:44.843355 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:44.843305    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:26:45.109304 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:45.109206    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:26:45.248791 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:45.248745    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:26:45.345683 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:45.345652    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:26:45.486080 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:45.485992    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:26:51.351472 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:51.351431    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:26:51.455652 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:51.455618    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:26:51.551348 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:51.551307    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:26:51.765286 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:51.765190    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:26:56.854253 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:56.854211    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:26:56.954862 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:56.954828    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:26:57.920303 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:57.920260    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:26:58.021768 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:26:58.021726    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:27:05.062041 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:05.061947    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:27:05.434571 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:05.434474    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:27:05.722725 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:05.722627    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:27:08.613675 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:08.613633    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5122206c-6c478686c7-hjnm8"]
Jul 14 14:27:08.614157 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:08.614140    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a47ba40-fffe-497d-8be9-ceca1e584377" containerName="maas-api"
Jul 14 14:27:08.614199 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:08.614161    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a47ba40-fffe-497d-8be9-ceca1e584377" containerName="maas-api"
Jul 14 14:27:08.614245 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:08.614233    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a47ba40-fffe-497d-8be9-ceca1e584377" containerName="maas-api"
Jul 14 14:27:08.617655 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:08.617633    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5122206c-6c478686c7-hjnm8"
Jul 14 14:27:08.620316 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:08.620275    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-5122206c-serving-cert\""
Jul 14 14:27:08.620316 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:08.620293    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-znm9j\""
Jul 14 14:27:08.620539 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:08.620293    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 14:27:08.621086 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:08.621063    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 14 14:27:08.626754 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:08.626730    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5122206c-6c478686c7-hjnm8"]
Jul 14 14:27:08.645696 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:08.645658    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2nv7\" (UniqueName: \"kubernetes.io/projected/33f862e6-d0d6-44b7-8133-90518e81a4d0-kube-api-access-b2nv7\") pod \"maas-api-e2e-mt-5122206c-6c478686c7-hjnm8\" (UID: \"33f862e6-d0d6-44b7-8133-90518e81a4d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5122206c-6c478686c7-hjnm8"
Jul 14 14:27:08.645880 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:08.645704    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33f862e6-d0d6-44b7-8133-90518e81a4d0-openshift-service-ca\") pod \"maas-api-e2e-mt-5122206c-6c478686c7-hjnm8\" (UID: \"33f862e6-d0d6-44b7-8133-90518e81a4d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5122206c-6c478686c7-hjnm8"
Jul 14 14:27:08.645880 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:08.645785    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33f862e6-d0d6-44b7-8133-90518e81a4d0-maas-api-tls\") pod \"maas-api-e2e-mt-5122206c-6c478686c7-hjnm8\" (UID: \"33f862e6-d0d6-44b7-8133-90518e81a4d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5122206c-6c478686c7-hjnm8"
Jul 14 14:27:08.746394 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:08.746355    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2nv7\" (UniqueName: \"kubernetes.io/projected/33f862e6-d0d6-44b7-8133-90518e81a4d0-kube-api-access-b2nv7\") pod \"maas-api-e2e-mt-5122206c-6c478686c7-hjnm8\" (UID: \"33f862e6-d0d6-44b7-8133-90518e81a4d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5122206c-6c478686c7-hjnm8"
Jul 14 14:27:08.746394 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:08.746405    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33f862e6-d0d6-44b7-8133-90518e81a4d0-openshift-service-ca\") pod \"maas-api-e2e-mt-5122206c-6c478686c7-hjnm8\" (UID: \"33f862e6-d0d6-44b7-8133-90518e81a4d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5122206c-6c478686c7-hjnm8"
Jul 14 14:27:08.746738 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:08.746464    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33f862e6-d0d6-44b7-8133-90518e81a4d0-maas-api-tls\") pod \"maas-api-e2e-mt-5122206c-6c478686c7-hjnm8\" (UID: \"33f862e6-d0d6-44b7-8133-90518e81a4d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5122206c-6c478686c7-hjnm8"
Jul 14 14:27:08.746738 ip-10-0-128-127 kubenswrapper[2576]: E0714 14:27:08.746641    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-5122206c-serving-cert: secret "maas-api-e2e-mt-5122206c-serving-cert" not found
Jul 14 14:27:08.746851 ip-10-0-128-127 kubenswrapper[2576]: E0714 14:27:08.746746    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33f862e6-d0d6-44b7-8133-90518e81a4d0-maas-api-tls podName:33f862e6-d0d6-44b7-8133-90518e81a4d0 nodeName:}" failed. No retries permitted until 2026-07-14 14:27:09.246721784 +0000 UTC m=+1977.097159282 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/33f862e6-d0d6-44b7-8133-90518e81a4d0-maas-api-tls") pod "maas-api-e2e-mt-5122206c-6c478686c7-hjnm8" (UID: "33f862e6-d0d6-44b7-8133-90518e81a4d0") : secret "maas-api-e2e-mt-5122206c-serving-cert" not found
Jul 14 14:27:08.747103 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:08.747079    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33f862e6-d0d6-44b7-8133-90518e81a4d0-openshift-service-ca\") pod \"maas-api-e2e-mt-5122206c-6c478686c7-hjnm8\" (UID: \"33f862e6-d0d6-44b7-8133-90518e81a4d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5122206c-6c478686c7-hjnm8"
Jul 14 14:27:08.762972 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:08.762935    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2nv7\" (UniqueName: \"kubernetes.io/projected/33f862e6-d0d6-44b7-8133-90518e81a4d0-kube-api-access-b2nv7\") pod \"maas-api-e2e-mt-5122206c-6c478686c7-hjnm8\" (UID: \"33f862e6-d0d6-44b7-8133-90518e81a4d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5122206c-6c478686c7-hjnm8"
Jul 14 14:27:09.252038 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:09.251988    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33f862e6-d0d6-44b7-8133-90518e81a4d0-maas-api-tls\") pod \"maas-api-e2e-mt-5122206c-6c478686c7-hjnm8\" (UID: \"33f862e6-d0d6-44b7-8133-90518e81a4d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5122206c-6c478686c7-hjnm8"
Jul 14 14:27:09.254543 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:09.254507    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33f862e6-d0d6-44b7-8133-90518e81a4d0-maas-api-tls\") pod \"maas-api-e2e-mt-5122206c-6c478686c7-hjnm8\" (UID: \"33f862e6-d0d6-44b7-8133-90518e81a4d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5122206c-6c478686c7-hjnm8"
Jul 14 14:27:09.366677 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:09.366636    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:27:09.460460 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:09.460409    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:27:09.530230 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:09.530206    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5122206c-6c478686c7-hjnm8"
Jul 14 14:27:09.666230 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:09.666203    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5122206c-6c478686c7-hjnm8"]
Jul 14 14:27:09.668501 ip-10-0-128-127 kubenswrapper[2576]: W0714 14:27:09.668465    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33f862e6_d0d6_44b7_8133_90518e81a4d0.slice/crio-d18aa8010b3bed3fd0ebe43d91bbd699dbc52182854bd86308b374d731584aaa WatchSource:0}: Error finding container d18aa8010b3bed3fd0ebe43d91bbd699dbc52182854bd86308b374d731584aaa: Status 404 returned error can't find the container with id d18aa8010b3bed3fd0ebe43d91bbd699dbc52182854bd86308b374d731584aaa
Jul 14 14:27:09.669882 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:09.669860    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 14:27:10.231405 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:10.231360    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5122206c-6c478686c7-hjnm8" event={"ID":"33f862e6-d0d6-44b7-8133-90518e81a4d0","Type":"ContainerStarted","Data":"d18aa8010b3bed3fd0ebe43d91bbd699dbc52182854bd86308b374d731584aaa"}
Jul 14 14:27:11.237886 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:11.237848    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5122206c-6c478686c7-hjnm8" event={"ID":"33f862e6-d0d6-44b7-8133-90518e81a4d0","Type":"ContainerStarted","Data":"c9a1b8ae9c6edd2a787d4b1fa3795d2da18c4a4cec39bb47a242dfc5ab2f2703"}
Jul 14 14:27:11.238283 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:11.237919    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5122206c-6c478686c7-hjnm8"
Jul 14 14:27:11.259224 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:11.259167    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5122206c-6c478686c7-hjnm8" podStartSLOduration=1.89764425 podStartE2EDuration="3.259146506s" totalImagesPullingTime="1.361502256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:27:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:27:11.258088371 +0000 UTC m=+1979.108525891" watchObservedRunningTime="2026-07-14 14:27:11.259146506 +0000 UTC m=+1979.109584090"
Jul 14 14:27:13.177706 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:13.177664    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:27:14.617868 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:14.617833    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:27:17.247741 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:17.247706    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5122206c-6c478686c7-hjnm8"
Jul 14 14:27:30.488485 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:30.488441    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-5122206c/e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr"]
Jul 14 14:27:30.491938 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:30.491916    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5122206c/e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr"
Jul 14 14:27:30.497258 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:30.497233    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-5122206c\"/\"default-dockercfg-2mmg4\""
Jul 14 14:27:30.498055 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:30.498025    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-5122206c\"/\"openshift-service-ca.crt\""
Jul 14 14:27:30.498221 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:30.498198    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-5122206c\"/\"e2e-lifecycle-model-5122206c-kserve-self-signed-certs\""
Jul 14 14:27:30.498550 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:30.498512    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-5122206c\"/\"kube-root-ca.crt\""
Jul 14 14:27:30.508507 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:30.508478    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-5122206c/e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr"]
Jul 14 14:27:30.519123 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:30.519079    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:27:30.653472 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:30.653430    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0bbca734-f94f-4a6c-aab5-0c86d6351882-dshm\") pod \"e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr\" (UID: \"0bbca734-f94f-4a6c-aab5-0c86d6351882\") " pod="ai-tenant-e2e-mt-5122206c/e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr"
Jul 14 14:27:30.653472 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:30.653469    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0bbca734-f94f-4a6c-aab5-0c86d6351882-tmp-dir\") pod \"e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr\" (UID: \"0bbca734-f94f-4a6c-aab5-0c86d6351882\") " pod="ai-tenant-e2e-mt-5122206c/e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr"
Jul 14 14:27:30.653738 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:30.653491    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6kcb\" (UniqueName: \"kubernetes.io/projected/0bbca734-f94f-4a6c-aab5-0c86d6351882-kube-api-access-x6kcb\") pod \"e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr\" (UID: \"0bbca734-f94f-4a6c-aab5-0c86d6351882\") " pod="ai-tenant-e2e-mt-5122206c/e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr"
Jul 14 14:27:30.653738 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:30.653637    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0bbca734-f94f-4a6c-aab5-0c86d6351882-model-cache\") pod \"e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr\" (UID: \"0bbca734-f94f-4a6c-aab5-0c86d6351882\") " pod="ai-tenant-e2e-mt-5122206c/e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr"
Jul 14 14:27:30.653738 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:30.653679    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0bbca734-f94f-4a6c-aab5-0c86d6351882-home\") pod \"e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr\" (UID: \"0bbca734-f94f-4a6c-aab5-0c86d6351882\") " pod="ai-tenant-e2e-mt-5122206c/e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr"
Jul 14 14:27:30.653738 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:30.653723    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0bbca734-f94f-4a6c-aab5-0c86d6351882-tls-certs\") pod \"e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr\" (UID: \"0bbca734-f94f-4a6c-aab5-0c86d6351882\") " pod="ai-tenant-e2e-mt-5122206c/e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr"
Jul 14 14:27:30.754453 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:30.754349    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0bbca734-f94f-4a6c-aab5-0c86d6351882-model-cache\") pod \"e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr\" (UID: \"0bbca734-f94f-4a6c-aab5-0c86d6351882\") " pod="ai-tenant-e2e-mt-5122206c/e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr"
Jul 14 14:27:30.754453 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:30.754402    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0bbca734-f94f-4a6c-aab5-0c86d6351882-home\") pod \"e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr\" (UID: \"0bbca734-f94f-4a6c-aab5-0c86d6351882\") " pod="ai-tenant-e2e-mt-5122206c/e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr"
Jul 14 14:27:30.754453 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:30.754449    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0bbca734-f94f-4a6c-aab5-0c86d6351882-tls-certs\") pod \"e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr\" (UID: \"0bbca734-f94f-4a6c-aab5-0c86d6351882\") " pod="ai-tenant-e2e-mt-5122206c/e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr"
Jul 14 14:27:30.754755 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:30.754490    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0bbca734-f94f-4a6c-aab5-0c86d6351882-dshm\") pod \"e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr\" (UID: \"0bbca734-f94f-4a6c-aab5-0c86d6351882\") " pod="ai-tenant-e2e-mt-5122206c/e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr"
Jul 14 14:27:30.754755 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:30.754627    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0bbca734-f94f-4a6c-aab5-0c86d6351882-tmp-dir\") pod \"e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr\" (UID: \"0bbca734-f94f-4a6c-aab5-0c86d6351882\") " pod="ai-tenant-e2e-mt-5122206c/e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr"
Jul 14 14:27:30.754755 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:30.754657    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6kcb\" (UniqueName: \"kubernetes.io/projected/0bbca734-f94f-4a6c-aab5-0c86d6351882-kube-api-access-x6kcb\") pod \"e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr\" (UID: \"0bbca734-f94f-4a6c-aab5-0c86d6351882\") " pod="ai-tenant-e2e-mt-5122206c/e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr"
Jul 14 14:27:30.754901 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:30.754879    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0bbca734-f94f-4a6c-aab5-0c86d6351882-model-cache\") pod \"e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr\" (UID: \"0bbca734-f94f-4a6c-aab5-0c86d6351882\") " pod="ai-tenant-e2e-mt-5122206c/e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr"
Jul 14 14:27:30.754950 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:30.754908    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0bbca734-f94f-4a6c-aab5-0c86d6351882-home\") pod \"e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr\" (UID: \"0bbca734-f94f-4a6c-aab5-0c86d6351882\") " pod="ai-tenant-e2e-mt-5122206c/e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr"
Jul 14 14:27:30.754994 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:30.754962    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0bbca734-f94f-4a6c-aab5-0c86d6351882-tmp-dir\") pod \"e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr\" (UID: \"0bbca734-f94f-4a6c-aab5-0c86d6351882\") " pod="ai-tenant-e2e-mt-5122206c/e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr"
Jul 14 14:27:30.756898 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:30.756864    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0bbca734-f94f-4a6c-aab5-0c86d6351882-dshm\") pod \"e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr\" (UID: \"0bbca734-f94f-4a6c-aab5-0c86d6351882\") " pod="ai-tenant-e2e-mt-5122206c/e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr"
Jul 14 14:27:30.757177 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:30.757156    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0bbca734-f94f-4a6c-aab5-0c86d6351882-tls-certs\") pod \"e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr\" (UID: \"0bbca734-f94f-4a6c-aab5-0c86d6351882\") " pod="ai-tenant-e2e-mt-5122206c/e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr"
Jul 14 14:27:30.763843 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:30.763815    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6kcb\" (UniqueName: \"kubernetes.io/projected/0bbca734-f94f-4a6c-aab5-0c86d6351882-kube-api-access-x6kcb\") pod \"e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr\" (UID: \"0bbca734-f94f-4a6c-aab5-0c86d6351882\") " pod="ai-tenant-e2e-mt-5122206c/e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr"
Jul 14 14:27:30.803714 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:30.803672    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5122206c/e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr"
Jul 14 14:27:30.939070 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:30.939042    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-5122206c/e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr"]
Jul 14 14:27:30.941916 ip-10-0-128-127 kubenswrapper[2576]: W0714 14:27:30.941878    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0bbca734_f94f_4a6c_aab5_0c86d6351882.slice/crio-d6f8488ee6b2396bd28ede2fd7077efafff764aa42235eec0ca0dc9439504d4f WatchSource:0}: Error finding container d6f8488ee6b2396bd28ede2fd7077efafff764aa42235eec0ca0dc9439504d4f: Status 404 returned error can't find the container with id d6f8488ee6b2396bd28ede2fd7077efafff764aa42235eec0ca0dc9439504d4f
Jul 14 14:27:31.327897 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:31.327862    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5122206c/e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr" event={"ID":"0bbca734-f94f-4a6c-aab5-0c86d6351882","Type":"ContainerStarted","Data":"d4514ae4eabdb5d687a1e22968ca49e45293b8817b967e383c96d539e0bd42f7"}
Jul 14 14:27:31.328070 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:31.327903    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5122206c/e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr" event={"ID":"0bbca734-f94f-4a6c-aab5-0c86d6351882","Type":"ContainerStarted","Data":"d6f8488ee6b2396bd28ede2fd7077efafff764aa42235eec0ca0dc9439504d4f"}
Jul 14 14:27:31.352218 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:31.352162    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-5122206c/e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr" podStartSLOduration=1.3521421120000001 podStartE2EDuration="1.352142112s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:27:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:27:31.350856536 +0000 UTC m=+1999.201294040" watchObservedRunningTime="2026-07-14 14:27:31.352142112 +0000 UTC m=+1999.202579615"
Jul 14 14:27:34.118632 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:34.118572    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:27:40.804105 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:40.804066    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-5122206c/e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr"
Jul 14 14:27:40.804690 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:40.804221    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-5122206c/e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr"
Jul 14 14:27:40.817313 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:40.817284    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-5122206c/e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr"
Jul 14 14:27:41.381935 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:41.381902    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-5122206c/e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr"
Jul 14 14:27:48.109323 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:27:48.109284    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:28:02.445270 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:02.445228    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5122206c-6c478686c7-hjnm8"]
Jul 14 14:28:02.445743 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:02.445533    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5122206c-6c478686c7-hjnm8" podUID="33f862e6-d0d6-44b7-8133-90518e81a4d0" containerName="maas-api" containerID="cri-o://c9a1b8ae9c6edd2a787d4b1fa3795d2da18c4a4cec39bb47a242dfc5ab2f2703" gracePeriod=30
Jul 14 14:28:02.544060 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:02.544020    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:28:02.699093 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:02.699033    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5122206c-6c478686c7-hjnm8"
Jul 14 14:28:02.857752 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:02.857690    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33f862e6-d0d6-44b7-8133-90518e81a4d0-openshift-service-ca\") pod \"33f862e6-d0d6-44b7-8133-90518e81a4d0\" (UID: \"33f862e6-d0d6-44b7-8133-90518e81a4d0\") "
Jul 14 14:28:02.857951 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:02.857814    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33f862e6-d0d6-44b7-8133-90518e81a4d0-maas-api-tls\") pod \"33f862e6-d0d6-44b7-8133-90518e81a4d0\" (UID: \"33f862e6-d0d6-44b7-8133-90518e81a4d0\") "
Jul 14 14:28:02.857951 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:02.857857    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b2nv7\" (UniqueName: \"kubernetes.io/projected/33f862e6-d0d6-44b7-8133-90518e81a4d0-kube-api-access-b2nv7\") pod \"33f862e6-d0d6-44b7-8133-90518e81a4d0\" (UID: \"33f862e6-d0d6-44b7-8133-90518e81a4d0\") "
Jul 14 14:28:02.858231 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:02.858199    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33f862e6-d0d6-44b7-8133-90518e81a4d0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "33f862e6-d0d6-44b7-8133-90518e81a4d0" (UID: "33f862e6-d0d6-44b7-8133-90518e81a4d0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 14:28:02.859999 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:02.859967    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33f862e6-d0d6-44b7-8133-90518e81a4d0-kube-api-access-b2nv7" (OuterVolumeSpecName: "kube-api-access-b2nv7") pod "33f862e6-d0d6-44b7-8133-90518e81a4d0" (UID: "33f862e6-d0d6-44b7-8133-90518e81a4d0"). InnerVolumeSpecName "kube-api-access-b2nv7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 14:28:02.860099 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:02.860002    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33f862e6-d0d6-44b7-8133-90518e81a4d0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "33f862e6-d0d6-44b7-8133-90518e81a4d0" (UID: "33f862e6-d0d6-44b7-8133-90518e81a4d0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 14:28:02.959384 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:02.959285    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33f862e6-d0d6-44b7-8133-90518e81a4d0-maas-api-tls\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:28:02.959384 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:02.959323    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2nv7\" (UniqueName: \"kubernetes.io/projected/33f862e6-d0d6-44b7-8133-90518e81a4d0-kube-api-access-b2nv7\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:28:02.959384 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:02.959333    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33f862e6-d0d6-44b7-8133-90518e81a4d0-openshift-service-ca\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:28:03.478057 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:03.478019    2576 generic.go:356] "Generic (PLEG): container finished" podID="33f862e6-d0d6-44b7-8133-90518e81a4d0" containerID="c9a1b8ae9c6edd2a787d4b1fa3795d2da18c4a4cec39bb47a242dfc5ab2f2703" exitCode=0
Jul 14 14:28:03.478472 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:03.478081    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5122206c-6c478686c7-hjnm8"
Jul 14 14:28:03.478472 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:03.478103    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5122206c-6c478686c7-hjnm8" event={"ID":"33f862e6-d0d6-44b7-8133-90518e81a4d0","Type":"ContainerDied","Data":"c9a1b8ae9c6edd2a787d4b1fa3795d2da18c4a4cec39bb47a242dfc5ab2f2703"}
Jul 14 14:28:03.478472 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:03.478175    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5122206c-6c478686c7-hjnm8" event={"ID":"33f862e6-d0d6-44b7-8133-90518e81a4d0","Type":"ContainerDied","Data":"d18aa8010b3bed3fd0ebe43d91bbd699dbc52182854bd86308b374d731584aaa"}
Jul 14 14:28:03.478472 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:03.478199    2576 scope.go:117] "RemoveContainer" containerID="c9a1b8ae9c6edd2a787d4b1fa3795d2da18c4a4cec39bb47a242dfc5ab2f2703"
Jul 14 14:28:03.487062 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:03.487045    2576 scope.go:117] "RemoveContainer" containerID="c9a1b8ae9c6edd2a787d4b1fa3795d2da18c4a4cec39bb47a242dfc5ab2f2703"
Jul 14 14:28:03.487311 ip-10-0-128-127 kubenswrapper[2576]: E0714 14:28:03.487293    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9a1b8ae9c6edd2a787d4b1fa3795d2da18c4a4cec39bb47a242dfc5ab2f2703\": container with ID starting with c9a1b8ae9c6edd2a787d4b1fa3795d2da18c4a4cec39bb47a242dfc5ab2f2703 not found: ID does not exist" containerID="c9a1b8ae9c6edd2a787d4b1fa3795d2da18c4a4cec39bb47a242dfc5ab2f2703"
Jul 14 14:28:03.487352 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:03.487319    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9a1b8ae9c6edd2a787d4b1fa3795d2da18c4a4cec39bb47a242dfc5ab2f2703"} err="failed to get container status \"c9a1b8ae9c6edd2a787d4b1fa3795d2da18c4a4cec39bb47a242dfc5ab2f2703\": rpc error: code = NotFound desc = could not find container \"c9a1b8ae9c6edd2a787d4b1fa3795d2da18c4a4cec39bb47a242dfc5ab2f2703\": container with ID starting with c9a1b8ae9c6edd2a787d4b1fa3795d2da18c4a4cec39bb47a242dfc5ab2f2703 not found: ID does not exist"
Jul 14 14:28:03.507389 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:03.507353    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5122206c-6c478686c7-hjnm8"]
Jul 14 14:28:03.514899 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:03.514872    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5122206c-6c478686c7-hjnm8"]
Jul 14 14:28:04.688474 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:04.688436    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33f862e6-d0d6-44b7-8133-90518e81a4d0" path="/var/lib/kubelet/pods/33f862e6-d0d6-44b7-8133-90518e81a4d0/volumes"
Jul 14 14:28:06.016048 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:06.016010    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:28:06.919123 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:06.919082    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:28:11.913192 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:11.913150    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-5122206c/e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr"]
Jul 14 14:28:12.071354 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:12.071316    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:28:26.933363 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:26.933324    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-5122206c/e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr" podUID="0bbca734-f94f-4a6c-aab5-0c86d6351882" containerName="main" containerID="cri-o://d4514ae4eabdb5d687a1e22968ca49e45293b8817b967e383c96d539e0bd42f7" gracePeriod=45
Jul 14 14:28:27.183243 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:27.183217    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5122206c/e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr"
Jul 14 14:28:27.268004 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:27.267898    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0bbca734-f94f-4a6c-aab5-0c86d6351882-home\") pod \"0bbca734-f94f-4a6c-aab5-0c86d6351882\" (UID: \"0bbca734-f94f-4a6c-aab5-0c86d6351882\") "
Jul 14 14:28:27.268004 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:27.267939    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0bbca734-f94f-4a6c-aab5-0c86d6351882-model-cache\") pod \"0bbca734-f94f-4a6c-aab5-0c86d6351882\" (UID: \"0bbca734-f94f-4a6c-aab5-0c86d6351882\") "
Jul 14 14:28:27.268004 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:27.267974    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0bbca734-f94f-4a6c-aab5-0c86d6351882-dshm\") pod \"0bbca734-f94f-4a6c-aab5-0c86d6351882\" (UID: \"0bbca734-f94f-4a6c-aab5-0c86d6351882\") "
Jul 14 14:28:27.268004 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:27.268002    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0bbca734-f94f-4a6c-aab5-0c86d6351882-tmp-dir\") pod \"0bbca734-f94f-4a6c-aab5-0c86d6351882\" (UID: \"0bbca734-f94f-4a6c-aab5-0c86d6351882\") "
Jul 14 14:28:27.268328 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:27.268023    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6kcb\" (UniqueName: \"kubernetes.io/projected/0bbca734-f94f-4a6c-aab5-0c86d6351882-kube-api-access-x6kcb\") pod \"0bbca734-f94f-4a6c-aab5-0c86d6351882\" (UID: \"0bbca734-f94f-4a6c-aab5-0c86d6351882\") "
Jul 14 14:28:27.268328 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:27.268055    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0bbca734-f94f-4a6c-aab5-0c86d6351882-tls-certs\") pod \"0bbca734-f94f-4a6c-aab5-0c86d6351882\" (UID: \"0bbca734-f94f-4a6c-aab5-0c86d6351882\") "
Jul 14 14:28:27.268328 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:27.268154    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bbca734-f94f-4a6c-aab5-0c86d6351882-home" (OuterVolumeSpecName: "home") pod "0bbca734-f94f-4a6c-aab5-0c86d6351882" (UID: "0bbca734-f94f-4a6c-aab5-0c86d6351882"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 14:28:27.268491 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:27.268344    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bbca734-f94f-4a6c-aab5-0c86d6351882-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0bbca734-f94f-4a6c-aab5-0c86d6351882" (UID: "0bbca734-f94f-4a6c-aab5-0c86d6351882"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 14:28:27.268491 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:27.268388    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bbca734-f94f-4a6c-aab5-0c86d6351882-model-cache" (OuterVolumeSpecName: "model-cache") pod "0bbca734-f94f-4a6c-aab5-0c86d6351882" (UID: "0bbca734-f94f-4a6c-aab5-0c86d6351882"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 14:28:27.268491 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:27.268395    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0bbca734-f94f-4a6c-aab5-0c86d6351882-home\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:28:27.270360 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:27.270325    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bbca734-f94f-4a6c-aab5-0c86d6351882-dshm" (OuterVolumeSpecName: "dshm") pod "0bbca734-f94f-4a6c-aab5-0c86d6351882" (UID: "0bbca734-f94f-4a6c-aab5-0c86d6351882"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 14:28:27.270697 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:27.270383    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0bbca734-f94f-4a6c-aab5-0c86d6351882-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0bbca734-f94f-4a6c-aab5-0c86d6351882" (UID: "0bbca734-f94f-4a6c-aab5-0c86d6351882"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 14:28:27.270697 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:27.270397    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bbca734-f94f-4a6c-aab5-0c86d6351882-kube-api-access-x6kcb" (OuterVolumeSpecName: "kube-api-access-x6kcb") pod "0bbca734-f94f-4a6c-aab5-0c86d6351882" (UID: "0bbca734-f94f-4a6c-aab5-0c86d6351882"). InnerVolumeSpecName "kube-api-access-x6kcb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 14:28:27.369009 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:27.368958    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0bbca734-f94f-4a6c-aab5-0c86d6351882-model-cache\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:28:27.369009 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:27.368995    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0bbca734-f94f-4a6c-aab5-0c86d6351882-dshm\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:28:27.369009 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:27.369006    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0bbca734-f94f-4a6c-aab5-0c86d6351882-tmp-dir\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:28:27.369009 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:27.369014    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6kcb\" (UniqueName: \"kubernetes.io/projected/0bbca734-f94f-4a6c-aab5-0c86d6351882-kube-api-access-x6kcb\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:28:27.369009 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:27.369024    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0bbca734-f94f-4a6c-aab5-0c86d6351882-tls-certs\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:28:27.584618 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:27.584571    2576 generic.go:356] "Generic (PLEG): container finished" podID="0bbca734-f94f-4a6c-aab5-0c86d6351882" containerID="d4514ae4eabdb5d687a1e22968ca49e45293b8817b967e383c96d539e0bd42f7" exitCode=0
Jul 14 14:28:27.584791 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:27.584628    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5122206c/e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr"
Jul 14 14:28:27.584791 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:27.584661    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5122206c/e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr" event={"ID":"0bbca734-f94f-4a6c-aab5-0c86d6351882","Type":"ContainerDied","Data":"d4514ae4eabdb5d687a1e22968ca49e45293b8817b967e383c96d539e0bd42f7"}
Jul 14 14:28:27.584791 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:27.584710    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5122206c/e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr" event={"ID":"0bbca734-f94f-4a6c-aab5-0c86d6351882","Type":"ContainerDied","Data":"d6f8488ee6b2396bd28ede2fd7077efafff764aa42235eec0ca0dc9439504d4f"}
Jul 14 14:28:27.584791 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:27.584729    2576 scope.go:117] "RemoveContainer" containerID="d4514ae4eabdb5d687a1e22968ca49e45293b8817b967e383c96d539e0bd42f7"
Jul 14 14:28:27.595504 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:27.595481    2576 scope.go:117] "RemoveContainer" containerID="d4514ae4eabdb5d687a1e22968ca49e45293b8817b967e383c96d539e0bd42f7"
Jul 14 14:28:27.595988 ip-10-0-128-127 kubenswrapper[2576]: E0714 14:28:27.595965    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4514ae4eabdb5d687a1e22968ca49e45293b8817b967e383c96d539e0bd42f7\": container with ID starting with d4514ae4eabdb5d687a1e22968ca49e45293b8817b967e383c96d539e0bd42f7 not found: ID does not exist" containerID="d4514ae4eabdb5d687a1e22968ca49e45293b8817b967e383c96d539e0bd42f7"
Jul 14 14:28:27.596074 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:27.595997    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4514ae4eabdb5d687a1e22968ca49e45293b8817b967e383c96d539e0bd42f7"} err="failed to get container status \"d4514ae4eabdb5d687a1e22968ca49e45293b8817b967e383c96d539e0bd42f7\": rpc error: code = NotFound desc = could not find container \"d4514ae4eabdb5d687a1e22968ca49e45293b8817b967e383c96d539e0bd42f7\": container with ID starting with d4514ae4eabdb5d687a1e22968ca49e45293b8817b967e383c96d539e0bd42f7 not found: ID does not exist"
Jul 14 14:28:27.623501 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:27.623466    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-5122206c/e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr"]
Jul 14 14:28:27.631003 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:27.630975    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-5122206c/e2e-lifecycle-model-5122206c-kserve-75f6c958c9-k7btr"]
Jul 14 14:28:27.772156 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:27.772116    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:28:27.876051 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:27.875961    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:28:28.689018 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:28.688985    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0bbca734-f94f-4a6c-aab5-0c86d6351882" path="/var/lib/kubelet/pods/0bbca734-f94f-4a6c-aab5-0c86d6351882/volumes"
Jul 14 14:28:48.828841 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:48.828753    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:28:49.139368 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:49.139280    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:28:49.242306 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:49.242246    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:28:50.029325 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:50.029282    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:28:56.843479 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:56.843436    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:28:56.987311 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:56.987264    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:28:59.830349 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:28:59.830282    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:29:00.096437 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:29:00.096341    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:29:01.365153 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:29:01.365117    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:29:01.525584 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:29:01.525549    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:29:01.649095 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:29:01.648995    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:29:02.560154 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:29:02.560121    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:29:02.680297 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:29:02.680238    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:29:02.810479 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:29:02.810380    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:29:04.544071 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:29:04.544028    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:29:10.536129 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:29:10.536088    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:29:12.687850 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:29:12.687822    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sh6rt_5600e3e1-e775-482e-bc72-76d732552705/ovn-acl-logging/0.log"
Jul 14 14:29:12.690262 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:29:12.690243    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sh6rt_5600e3e1-e775-482e-bc72-76d732552705/ovn-acl-logging/0.log"
Jul 14 14:29:12.742283 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:29:12.742250    2576 scope.go:117] "RemoveContainer" containerID="9ab1ecb295ffb38adce8d883b09e6cef95328f0d03c8d234b8bcd341d1cabab4"
Jul 14 14:29:12.817193 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:29:12.817150    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:29:13.062940 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:29:13.062895    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:29:13.780499 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:29:13.780459    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:29:20.919582 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:29:20.919536    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:29:21.423471 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:29:21.423423    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:29:21.573027 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:29:21.572983    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:29:22.002182 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:29:22.002150    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:29:32.330699 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:29:32.330657    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:29:32.786942 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:29:32.786904    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:29:53.825429 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:29:53.825388    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:30:00.673477 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:00.673436    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:30:04.565909 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:04.565826    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0bd8ad-6f9cddfb7c-lgjxs"]
Jul 14 14:30:04.566263 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:04.566151    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33f862e6-d0d6-44b7-8133-90518e81a4d0" containerName="maas-api"
Jul 14 14:30:04.566263 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:04.566162    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="33f862e6-d0d6-44b7-8133-90518e81a4d0" containerName="maas-api"
Jul 14 14:30:04.566263 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:04.566172    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bbca734-f94f-4a6c-aab5-0c86d6351882" containerName="main"
Jul 14 14:30:04.566263 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:04.566177    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bbca734-f94f-4a6c-aab5-0c86d6351882" containerName="main"
Jul 14 14:30:04.566263 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:04.566228    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bbca734-f94f-4a6c-aab5-0c86d6351882" containerName="main"
Jul 14 14:30:04.566263 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:04.566236    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="33f862e6-d0d6-44b7-8133-90518e81a4d0" containerName="maas-api"
Jul 14 14:30:04.569080 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:04.569059    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0bd8ad-6f9cddfb7c-lgjxs"
Jul 14 14:30:04.571728 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:04.571702    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 14 14:30:04.572464 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:04.572438    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-0bd8ad-serving-cert\""
Jul 14 14:30:04.572641 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:04.572460    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 14:30:04.572641 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:04.572449    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-znm9j\""
Jul 14 14:30:04.579897 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:04.579873    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0bd8ad-6f9cddfb7c-lgjxs"]
Jul 14 14:30:04.649347 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:04.649306    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916-openshift-service-ca\") pod \"maas-api-e2e-inf-a-0bd8ad-6f9cddfb7c-lgjxs\" (UID: \"6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0bd8ad-6f9cddfb7c-lgjxs"
Jul 14 14:30:04.649583 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:04.649356    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916-maas-api-tls\") pod \"maas-api-e2e-inf-a-0bd8ad-6f9cddfb7c-lgjxs\" (UID: \"6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0bd8ad-6f9cddfb7c-lgjxs"
Jul 14 14:30:04.649583 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:04.649473    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckjpz\" (UniqueName: \"kubernetes.io/projected/6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916-kube-api-access-ckjpz\") pod \"maas-api-e2e-inf-a-0bd8ad-6f9cddfb7c-lgjxs\" (UID: \"6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0bd8ad-6f9cddfb7c-lgjxs"
Jul 14 14:30:04.750737 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:04.750696    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckjpz\" (UniqueName: \"kubernetes.io/projected/6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916-kube-api-access-ckjpz\") pod \"maas-api-e2e-inf-a-0bd8ad-6f9cddfb7c-lgjxs\" (UID: \"6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0bd8ad-6f9cddfb7c-lgjxs"
Jul 14 14:30:04.750927 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:04.750753    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916-openshift-service-ca\") pod \"maas-api-e2e-inf-a-0bd8ad-6f9cddfb7c-lgjxs\" (UID: \"6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0bd8ad-6f9cddfb7c-lgjxs"
Jul 14 14:30:04.750927 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:04.750785    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916-maas-api-tls\") pod \"maas-api-e2e-inf-a-0bd8ad-6f9cddfb7c-lgjxs\" (UID: \"6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0bd8ad-6f9cddfb7c-lgjxs"
Jul 14 14:30:04.750927 ip-10-0-128-127 kubenswrapper[2576]: E0714 14:30:04.750884    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-0bd8ad-serving-cert: secret "maas-api-e2e-inf-a-0bd8ad-serving-cert" not found
Jul 14 14:30:04.751033 ip-10-0-128-127 kubenswrapper[2576]: E0714 14:30:04.750947    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916-maas-api-tls podName:6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916 nodeName:}" failed. No retries permitted until 2026-07-14 14:30:05.250930991 +0000 UTC m=+2153.101368471 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916-maas-api-tls") pod "maas-api-e2e-inf-a-0bd8ad-6f9cddfb7c-lgjxs" (UID: "6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916") : secret "maas-api-e2e-inf-a-0bd8ad-serving-cert" not found
Jul 14 14:30:04.751443 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:04.751420    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916-openshift-service-ca\") pod \"maas-api-e2e-inf-a-0bd8ad-6f9cddfb7c-lgjxs\" (UID: \"6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0bd8ad-6f9cddfb7c-lgjxs"
Jul 14 14:30:04.766745 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:04.766714    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckjpz\" (UniqueName: \"kubernetes.io/projected/6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916-kube-api-access-ckjpz\") pod \"maas-api-e2e-inf-a-0bd8ad-6f9cddfb7c-lgjxs\" (UID: \"6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0bd8ad-6f9cddfb7c-lgjxs"
Jul 14 14:30:05.254940 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:05.254902    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916-maas-api-tls\") pod \"maas-api-e2e-inf-a-0bd8ad-6f9cddfb7c-lgjxs\" (UID: \"6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0bd8ad-6f9cddfb7c-lgjxs"
Jul 14 14:30:05.257420 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:05.257384    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916-maas-api-tls\") pod \"maas-api-e2e-inf-a-0bd8ad-6f9cddfb7c-lgjxs\" (UID: \"6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0bd8ad-6f9cddfb7c-lgjxs"
Jul 14 14:30:05.481773 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:05.481733    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0bd8ad-6f9cddfb7c-lgjxs"
Jul 14 14:30:05.621502 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:05.621464    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0bd8ad-6f9cddfb7c-lgjxs"]
Jul 14 14:30:05.625027 ip-10-0-128-127 kubenswrapper[2576]: W0714 14:30:05.624939    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d009e6e_68e3_4bed_9f0e_cd4d2c0f4916.slice/crio-212fac783e491751378c001b8be2c3697d3d7cc94c74f4119f284d71ea8d8515 WatchSource:0}: Error finding container 212fac783e491751378c001b8be2c3697d3d7cc94c74f4119f284d71ea8d8515: Status 404 returned error can't find the container with id 212fac783e491751378c001b8be2c3697d3d7cc94c74f4119f284d71ea8d8515
Jul 14 14:30:06.018380 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:06.018284    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0bd8ad-6f9cddfb7c-lgjxs" event={"ID":"6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916","Type":"ContainerStarted","Data":"212fac783e491751378c001b8be2c3697d3d7cc94c74f4119f284d71ea8d8515"}
Jul 14 14:30:06.507220 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:06.507184    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:30:07.024576 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:07.024535    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0bd8ad-6f9cddfb7c-lgjxs" event={"ID":"6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916","Type":"ContainerStarted","Data":"9dea0df6b27860c8e30a10f575a10d9fdb525b467251ad35aa963cc84588c71c"}
Jul 14 14:30:07.025085 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:07.024679    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0bd8ad-6f9cddfb7c-lgjxs"
Jul 14 14:30:07.044152 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:07.044096    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0bd8ad-6f9cddfb7c-lgjxs" podStartSLOduration=2.501126972 podStartE2EDuration="3.044074791s" totalImagesPullingTime="542.947819ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:30:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:30:07.04286377 +0000 UTC m=+2154.893301276" watchObservedRunningTime="2026-07-14 14:30:07.044074791 +0000 UTC m=+2154.894512293"
Jul 14 14:30:09.064182 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:09.064147    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:30:09.423446 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:09.423354    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:30:09.840586 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:09.840551    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:30:10.220506 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:10.220416    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:30:12.123585 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:12.123550    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:30:12.522124 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:12.522085    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:30:12.605560 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:12.605524    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:30:12.608928 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:12.608897    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-94a6fc-bb779b8f6-tbvhx"]
Jul 14 14:30:12.612554 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:12.612530    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a6fc-bb779b8f6-tbvhx"
Jul 14 14:30:12.615091 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:12.615062    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-94a6fc-serving-cert\""
Jul 14 14:30:12.624990 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:12.624959    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-94a6fc-bb779b8f6-tbvhx"]
Jul 14 14:30:12.723362 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:12.723317    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eeea8f9c-420d-4ebc-b2a2-9996b270681c-maas-api-tls\") pod \"maas-api-e2e-inf-b-94a6fc-bb779b8f6-tbvhx\" (UID: \"eeea8f9c-420d-4ebc-b2a2-9996b270681c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a6fc-bb779b8f6-tbvhx"
Jul 14 14:30:12.723362 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:12.723369    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eeea8f9c-420d-4ebc-b2a2-9996b270681c-openshift-service-ca\") pod \"maas-api-e2e-inf-b-94a6fc-bb779b8f6-tbvhx\" (UID: \"eeea8f9c-420d-4ebc-b2a2-9996b270681c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a6fc-bb779b8f6-tbvhx"
Jul 14 14:30:12.723580 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:12.723387    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x47dl\" (UniqueName: \"kubernetes.io/projected/eeea8f9c-420d-4ebc-b2a2-9996b270681c-kube-api-access-x47dl\") pod \"maas-api-e2e-inf-b-94a6fc-bb779b8f6-tbvhx\" (UID: \"eeea8f9c-420d-4ebc-b2a2-9996b270681c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a6fc-bb779b8f6-tbvhx"
Jul 14 14:30:12.824336 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:12.824244    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eeea8f9c-420d-4ebc-b2a2-9996b270681c-maas-api-tls\") pod \"maas-api-e2e-inf-b-94a6fc-bb779b8f6-tbvhx\" (UID: \"eeea8f9c-420d-4ebc-b2a2-9996b270681c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a6fc-bb779b8f6-tbvhx"
Jul 14 14:30:12.824336 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:12.824290    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eeea8f9c-420d-4ebc-b2a2-9996b270681c-openshift-service-ca\") pod \"maas-api-e2e-inf-b-94a6fc-bb779b8f6-tbvhx\" (UID: \"eeea8f9c-420d-4ebc-b2a2-9996b270681c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a6fc-bb779b8f6-tbvhx"
Jul 14 14:30:12.824336 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:12.824322    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x47dl\" (UniqueName: \"kubernetes.io/projected/eeea8f9c-420d-4ebc-b2a2-9996b270681c-kube-api-access-x47dl\") pod \"maas-api-e2e-inf-b-94a6fc-bb779b8f6-tbvhx\" (UID: \"eeea8f9c-420d-4ebc-b2a2-9996b270681c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a6fc-bb779b8f6-tbvhx"
Jul 14 14:30:12.824721 ip-10-0-128-127 kubenswrapper[2576]: E0714 14:30:12.824695    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-94a6fc-serving-cert: secret "maas-api-e2e-inf-b-94a6fc-serving-cert" not found
Jul 14 14:30:12.824786 ip-10-0-128-127 kubenswrapper[2576]: E0714 14:30:12.824764    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eeea8f9c-420d-4ebc-b2a2-9996b270681c-maas-api-tls podName:eeea8f9c-420d-4ebc-b2a2-9996b270681c nodeName:}" failed. No retries permitted until 2026-07-14 14:30:13.324747008 +0000 UTC m=+2161.175184489 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/eeea8f9c-420d-4ebc-b2a2-9996b270681c-maas-api-tls") pod "maas-api-e2e-inf-b-94a6fc-bb779b8f6-tbvhx" (UID: "eeea8f9c-420d-4ebc-b2a2-9996b270681c") : secret "maas-api-e2e-inf-b-94a6fc-serving-cert" not found
Jul 14 14:30:12.824978 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:12.824961    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eeea8f9c-420d-4ebc-b2a2-9996b270681c-openshift-service-ca\") pod \"maas-api-e2e-inf-b-94a6fc-bb779b8f6-tbvhx\" (UID: \"eeea8f9c-420d-4ebc-b2a2-9996b270681c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a6fc-bb779b8f6-tbvhx"
Jul 14 14:30:12.833241 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:12.833212    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x47dl\" (UniqueName: \"kubernetes.io/projected/eeea8f9c-420d-4ebc-b2a2-9996b270681c-kube-api-access-x47dl\") pod \"maas-api-e2e-inf-b-94a6fc-bb779b8f6-tbvhx\" (UID: \"eeea8f9c-420d-4ebc-b2a2-9996b270681c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a6fc-bb779b8f6-tbvhx"
Jul 14 14:30:12.984695 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:12.984657    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:30:13.034167 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:13.034130    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0bd8ad-6f9cddfb7c-lgjxs"
Jul 14 14:30:13.125679 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:13.125565    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:30:13.329082 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:13.329040    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eeea8f9c-420d-4ebc-b2a2-9996b270681c-maas-api-tls\") pod \"maas-api-e2e-inf-b-94a6fc-bb779b8f6-tbvhx\" (UID: \"eeea8f9c-420d-4ebc-b2a2-9996b270681c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a6fc-bb779b8f6-tbvhx"
Jul 14 14:30:13.331495 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:13.331470    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eeea8f9c-420d-4ebc-b2a2-9996b270681c-maas-api-tls\") pod \"maas-api-e2e-inf-b-94a6fc-bb779b8f6-tbvhx\" (UID: \"eeea8f9c-420d-4ebc-b2a2-9996b270681c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a6fc-bb779b8f6-tbvhx"
Jul 14 14:30:13.524542 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:13.524507    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a6fc-bb779b8f6-tbvhx"
Jul 14 14:30:13.580575 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:13.580205    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:30:13.666887 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:13.666860    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-94a6fc-bb779b8f6-tbvhx"]
Jul 14 14:30:13.669497 ip-10-0-128-127 kubenswrapper[2576]: W0714 14:30:13.669458    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeeea8f9c_420d_4ebc_b2a2_9996b270681c.slice/crio-078adec8fa6e337e7b2350bc054d526aaf3b1745675292b09f7213d24a04bb8a WatchSource:0}: Error finding container 078adec8fa6e337e7b2350bc054d526aaf3b1745675292b09f7213d24a04bb8a: Status 404 returned error can't find the container with id 078adec8fa6e337e7b2350bc054d526aaf3b1745675292b09f7213d24a04bb8a
Jul 14 14:30:13.674933 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:13.674892    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:30:14.056214 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:14.056186    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a6fc-bb779b8f6-tbvhx" event={"ID":"eeea8f9c-420d-4ebc-b2a2-9996b270681c","Type":"ContainerStarted","Data":"078adec8fa6e337e7b2350bc054d526aaf3b1745675292b09f7213d24a04bb8a"}
Jul 14 14:30:15.061669 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:15.061627    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a6fc-bb779b8f6-tbvhx" event={"ID":"eeea8f9c-420d-4ebc-b2a2-9996b270681c","Type":"ContainerStarted","Data":"e21f071449ddc92029bb246c770334aa373298e68454a61538483587baa7f735"}
Jul 14 14:30:15.062078 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:15.061701    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a6fc-bb779b8f6-tbvhx"
Jul 14 14:30:15.083765 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:15.083701    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a6fc-bb779b8f6-tbvhx" podStartSLOduration=2.70180934 podStartE2EDuration="3.083681922s" totalImagesPullingTime="381.872582ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:30:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:30:15.082191918 +0000 UTC m=+2162.932629422" watchObservedRunningTime="2026-07-14 14:30:15.083681922 +0000 UTC m=+2162.934119424"
Jul 14 14:30:17.164175 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:17.164136    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:30:19.487368 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:19.487322    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-0bd8ad/test-model-0bd8ad-kserve-5747687f-6c5qt"]
Jul 14 14:30:19.491218 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:19.491194    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-0bd8ad/test-model-0bd8ad-kserve-5747687f-6c5qt"
Jul 14 14:30:19.494107 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:19.494086    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-0bd8ad\"/\"openshift-service-ca.crt\""
Jul 14 14:30:19.495270 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:19.495251    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-0bd8ad\"/\"test-model-0bd8ad-kserve-self-signed-certs\""
Jul 14 14:30:19.495518 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:19.495485    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-0bd8ad\"/\"kube-root-ca.crt\""
Jul 14 14:30:19.495705 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:19.495687    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-0bd8ad\"/\"default-dockercfg-j8mkn\""
Jul 14 14:30:19.505328 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:19.505300    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-0bd8ad/test-model-0bd8ad-kserve-5747687f-6c5qt"]
Jul 14 14:30:19.588345 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:19.588306    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ds4q7\" (UniqueName: \"kubernetes.io/projected/69cf130d-9c33-40d5-af9d-3438e8909871-kube-api-access-ds4q7\") pod \"test-model-0bd8ad-kserve-5747687f-6c5qt\" (UID: \"69cf130d-9c33-40d5-af9d-3438e8909871\") " pod="ai-tenant-e2e-inf-a-0bd8ad/test-model-0bd8ad-kserve-5747687f-6c5qt"
Jul 14 14:30:19.588509 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:19.588356    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69cf130d-9c33-40d5-af9d-3438e8909871-model-cache\") pod \"test-model-0bd8ad-kserve-5747687f-6c5qt\" (UID: \"69cf130d-9c33-40d5-af9d-3438e8909871\") " pod="ai-tenant-e2e-inf-a-0bd8ad/test-model-0bd8ad-kserve-5747687f-6c5qt"
Jul 14 14:30:19.588509 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:19.588388    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69cf130d-9c33-40d5-af9d-3438e8909871-tmp-dir\") pod \"test-model-0bd8ad-kserve-5747687f-6c5qt\" (UID: \"69cf130d-9c33-40d5-af9d-3438e8909871\") " pod="ai-tenant-e2e-inf-a-0bd8ad/test-model-0bd8ad-kserve-5747687f-6c5qt"
Jul 14 14:30:19.588509 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:19.588417    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69cf130d-9c33-40d5-af9d-3438e8909871-home\") pod \"test-model-0bd8ad-kserve-5747687f-6c5qt\" (UID: \"69cf130d-9c33-40d5-af9d-3438e8909871\") " pod="ai-tenant-e2e-inf-a-0bd8ad/test-model-0bd8ad-kserve-5747687f-6c5qt"
Jul 14 14:30:19.588509 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:19.588455    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69cf130d-9c33-40d5-af9d-3438e8909871-dshm\") pod \"test-model-0bd8ad-kserve-5747687f-6c5qt\" (UID: \"69cf130d-9c33-40d5-af9d-3438e8909871\") " pod="ai-tenant-e2e-inf-a-0bd8ad/test-model-0bd8ad-kserve-5747687f-6c5qt"
Jul 14 14:30:19.588509 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:19.588486    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69cf130d-9c33-40d5-af9d-3438e8909871-tls-certs\") pod \"test-model-0bd8ad-kserve-5747687f-6c5qt\" (UID: \"69cf130d-9c33-40d5-af9d-3438e8909871\") " pod="ai-tenant-e2e-inf-a-0bd8ad/test-model-0bd8ad-kserve-5747687f-6c5qt"
Jul 14 14:30:19.603127 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:19.603085    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:30:19.689700 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:19.689665    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ds4q7\" (UniqueName: \"kubernetes.io/projected/69cf130d-9c33-40d5-af9d-3438e8909871-kube-api-access-ds4q7\") pod \"test-model-0bd8ad-kserve-5747687f-6c5qt\" (UID: \"69cf130d-9c33-40d5-af9d-3438e8909871\") " pod="ai-tenant-e2e-inf-a-0bd8ad/test-model-0bd8ad-kserve-5747687f-6c5qt"
Jul 14 14:30:19.689700 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:19.689707    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69cf130d-9c33-40d5-af9d-3438e8909871-model-cache\") pod \"test-model-0bd8ad-kserve-5747687f-6c5qt\" (UID: \"69cf130d-9c33-40d5-af9d-3438e8909871\") " pod="ai-tenant-e2e-inf-a-0bd8ad/test-model-0bd8ad-kserve-5747687f-6c5qt"
Jul 14 14:30:19.689907 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:19.689736    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69cf130d-9c33-40d5-af9d-3438e8909871-tmp-dir\") pod \"test-model-0bd8ad-kserve-5747687f-6c5qt\" (UID: \"69cf130d-9c33-40d5-af9d-3438e8909871\") " pod="ai-tenant-e2e-inf-a-0bd8ad/test-model-0bd8ad-kserve-5747687f-6c5qt"
Jul 14 14:30:19.689907 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:19.689845    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69cf130d-9c33-40d5-af9d-3438e8909871-home\") pod \"test-model-0bd8ad-kserve-5747687f-6c5qt\" (UID: \"69cf130d-9c33-40d5-af9d-3438e8909871\") " pod="ai-tenant-e2e-inf-a-0bd8ad/test-model-0bd8ad-kserve-5747687f-6c5qt"
Jul 14 14:30:19.689907 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:19.689883    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69cf130d-9c33-40d5-af9d-3438e8909871-dshm\") pod \"test-model-0bd8ad-kserve-5747687f-6c5qt\" (UID: \"69cf130d-9c33-40d5-af9d-3438e8909871\") " pod="ai-tenant-e2e-inf-a-0bd8ad/test-model-0bd8ad-kserve-5747687f-6c5qt"
Jul 14 14:30:19.690005 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:19.689936    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69cf130d-9c33-40d5-af9d-3438e8909871-tls-certs\") pod \"test-model-0bd8ad-kserve-5747687f-6c5qt\" (UID: \"69cf130d-9c33-40d5-af9d-3438e8909871\") " pod="ai-tenant-e2e-inf-a-0bd8ad/test-model-0bd8ad-kserve-5747687f-6c5qt"
Jul 14 14:30:19.690152 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:19.690133    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69cf130d-9c33-40d5-af9d-3438e8909871-model-cache\") pod \"test-model-0bd8ad-kserve-5747687f-6c5qt\" (UID: \"69cf130d-9c33-40d5-af9d-3438e8909871\") " pod="ai-tenant-e2e-inf-a-0bd8ad/test-model-0bd8ad-kserve-5747687f-6c5qt"
Jul 14 14:30:19.690191 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:19.690148    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69cf130d-9c33-40d5-af9d-3438e8909871-tmp-dir\") pod \"test-model-0bd8ad-kserve-5747687f-6c5qt\" (UID: \"69cf130d-9c33-40d5-af9d-3438e8909871\") " pod="ai-tenant-e2e-inf-a-0bd8ad/test-model-0bd8ad-kserve-5747687f-6c5qt"
Jul 14 14:30:19.690223 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:19.690198    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69cf130d-9c33-40d5-af9d-3438e8909871-home\") pod \"test-model-0bd8ad-kserve-5747687f-6c5qt\" (UID: \"69cf130d-9c33-40d5-af9d-3438e8909871\") " pod="ai-tenant-e2e-inf-a-0bd8ad/test-model-0bd8ad-kserve-5747687f-6c5qt"
Jul 14 14:30:19.692229 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:19.692206    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69cf130d-9c33-40d5-af9d-3438e8909871-dshm\") pod \"test-model-0bd8ad-kserve-5747687f-6c5qt\" (UID: \"69cf130d-9c33-40d5-af9d-3438e8909871\") " pod="ai-tenant-e2e-inf-a-0bd8ad/test-model-0bd8ad-kserve-5747687f-6c5qt"
Jul 14 14:30:19.692415 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:19.692398    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69cf130d-9c33-40d5-af9d-3438e8909871-tls-certs\") pod \"test-model-0bd8ad-kserve-5747687f-6c5qt\" (UID: \"69cf130d-9c33-40d5-af9d-3438e8909871\") " pod="ai-tenant-e2e-inf-a-0bd8ad/test-model-0bd8ad-kserve-5747687f-6c5qt"
Jul 14 14:30:19.698619 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:19.698565    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ds4q7\" (UniqueName: \"kubernetes.io/projected/69cf130d-9c33-40d5-af9d-3438e8909871-kube-api-access-ds4q7\") pod \"test-model-0bd8ad-kserve-5747687f-6c5qt\" (UID: \"69cf130d-9c33-40d5-af9d-3438e8909871\") " pod="ai-tenant-e2e-inf-a-0bd8ad/test-model-0bd8ad-kserve-5747687f-6c5qt"
Jul 14 14:30:19.804187 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:19.804153    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-0bd8ad/test-model-0bd8ad-kserve-5747687f-6c5qt"
Jul 14 14:30:19.946939 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:19.946755    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-0bd8ad/test-model-0bd8ad-kserve-5747687f-6c5qt"]
Jul 14 14:30:19.949719 ip-10-0-128-127 kubenswrapper[2576]: W0714 14:30:19.949687    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod69cf130d_9c33_40d5_af9d_3438e8909871.slice/crio-0bc8ba9af3f7628c457f21f468c8e66a1f70448ebcde542588cd9e4c9824f156 WatchSource:0}: Error finding container 0bc8ba9af3f7628c457f21f468c8e66a1f70448ebcde542588cd9e4c9824f156: Status 404 returned error can't find the container with id 0bc8ba9af3f7628c457f21f468c8e66a1f70448ebcde542588cd9e4c9824f156
Jul 14 14:30:20.088383 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:20.088299    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-0bd8ad/test-model-0bd8ad-kserve-5747687f-6c5qt" event={"ID":"69cf130d-9c33-40d5-af9d-3438e8909871","Type":"ContainerStarted","Data":"4a93257fa17860115d66420479df4d3a4735fa3cd0432b6d88f5435df395e46d"}
Jul 14 14:30:20.088383 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:20.088346    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-0bd8ad/test-model-0bd8ad-kserve-5747687f-6c5qt" event={"ID":"69cf130d-9c33-40d5-af9d-3438e8909871","Type":"ContainerStarted","Data":"0bc8ba9af3f7628c457f21f468c8e66a1f70448ebcde542588cd9e4c9824f156"}
Jul 14 14:30:20.107857 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:20.107796    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-0bd8ad/test-model-0bd8ad-kserve-5747687f-6c5qt" podStartSLOduration=1.107778722 podStartE2EDuration="1.107778722s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:30:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:30:20.105941513 +0000 UTC m=+2167.956379018" watchObservedRunningTime="2026-07-14 14:30:20.107778722 +0000 UTC m=+2167.958216224"
Jul 14 14:30:21.072606 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:21.072560    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a6fc-bb779b8f6-tbvhx"
Jul 14 14:30:23.412378 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:23.412340    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:30:26.290915 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:26.290873    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:30:26.590174 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:26.590065    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:30:29.805115 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:29.805070    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-0bd8ad/test-model-0bd8ad-kserve-5747687f-6c5qt"
Jul 14 14:30:29.805526 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:29.805135    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-0bd8ad/test-model-0bd8ad-kserve-5747687f-6c5qt"
Jul 14 14:30:29.817548 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:29.817515    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-0bd8ad/test-model-0bd8ad-kserve-5747687f-6c5qt"
Jul 14 14:30:30.147049 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:30.146956    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-0bd8ad/test-model-0bd8ad-kserve-5747687f-6c5qt"
Jul 14 14:30:33.925812 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:33.925779    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:30:34.274195 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:34.274161    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:30:34.374740 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:34.374701    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:30:44.940363 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:44.940329    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:30:45.040392 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:45.040352    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:30:45.944104 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:45.944064    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:30:46.010886 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:46.010851    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:30:52.499929 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:52.499890    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-94a6fc/test-model-94a6fc-kserve-57ddd74699-lklnk"]
Jul 14 14:30:52.503104 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:52.503080    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-94a6fc/test-model-94a6fc-kserve-57ddd74699-lklnk"
Jul 14 14:30:52.506115 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:52.506089    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-94a6fc\"/\"default-dockercfg-d5pq9\""
Jul 14 14:30:52.507059 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:52.507033    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-94a6fc\"/\"openshift-service-ca.crt\""
Jul 14 14:30:52.507192 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:52.507080    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-94a6fc\"/\"test-model-94a6fc-kserve-self-signed-certs\""
Jul 14 14:30:52.507192 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:52.507120    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-94a6fc\"/\"kube-root-ca.crt\""
Jul 14 14:30:52.518891 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:52.518854    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-94a6fc/test-model-94a6fc-kserve-57ddd74699-lklnk"]
Jul 14 14:30:52.532265 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:52.532232    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:30:52.594291 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:52.594252    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cf951df-9e8c-4af3-9daf-402266bb8151-dshm\") pod \"test-model-94a6fc-kserve-57ddd74699-lklnk\" (UID: \"4cf951df-9e8c-4af3-9daf-402266bb8151\") " pod="ai-tenant-e2e-inf-b-94a6fc/test-model-94a6fc-kserve-57ddd74699-lklnk"
Jul 14 14:30:52.594291 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:52.594296    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cf951df-9e8c-4af3-9daf-402266bb8151-tls-certs\") pod \"test-model-94a6fc-kserve-57ddd74699-lklnk\" (UID: \"4cf951df-9e8c-4af3-9daf-402266bb8151\") " pod="ai-tenant-e2e-inf-b-94a6fc/test-model-94a6fc-kserve-57ddd74699-lklnk"
Jul 14 14:30:52.594514 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:52.594379    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cf951df-9e8c-4af3-9daf-402266bb8151-tmp-dir\") pod \"test-model-94a6fc-kserve-57ddd74699-lklnk\" (UID: \"4cf951df-9e8c-4af3-9daf-402266bb8151\") " pod="ai-tenant-e2e-inf-b-94a6fc/test-model-94a6fc-kserve-57ddd74699-lklnk"
Jul 14 14:30:52.594514 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:52.594403    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cf951df-9e8c-4af3-9daf-402266bb8151-model-cache\") pod \"test-model-94a6fc-kserve-57ddd74699-lklnk\" (UID: \"4cf951df-9e8c-4af3-9daf-402266bb8151\") " pod="ai-tenant-e2e-inf-b-94a6fc/test-model-94a6fc-kserve-57ddd74699-lklnk"
Jul 14 14:30:52.594514 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:52.594419    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kskrf\" (UniqueName: \"kubernetes.io/projected/4cf951df-9e8c-4af3-9daf-402266bb8151-kube-api-access-kskrf\") pod \"test-model-94a6fc-kserve-57ddd74699-lklnk\" (UID: \"4cf951df-9e8c-4af3-9daf-402266bb8151\") " pod="ai-tenant-e2e-inf-b-94a6fc/test-model-94a6fc-kserve-57ddd74699-lklnk"
Jul 14 14:30:52.594514 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:52.594491    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cf951df-9e8c-4af3-9daf-402266bb8151-home\") pod \"test-model-94a6fc-kserve-57ddd74699-lklnk\" (UID: \"4cf951df-9e8c-4af3-9daf-402266bb8151\") " pod="ai-tenant-e2e-inf-b-94a6fc/test-model-94a6fc-kserve-57ddd74699-lklnk"
Jul 14 14:30:52.695411 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:52.695379    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cf951df-9e8c-4af3-9daf-402266bb8151-model-cache\") pod \"test-model-94a6fc-kserve-57ddd74699-lklnk\" (UID: \"4cf951df-9e8c-4af3-9daf-402266bb8151\") " pod="ai-tenant-e2e-inf-b-94a6fc/test-model-94a6fc-kserve-57ddd74699-lklnk"
Jul 14 14:30:52.695411 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:52.695412    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kskrf\" (UniqueName: \"kubernetes.io/projected/4cf951df-9e8c-4af3-9daf-402266bb8151-kube-api-access-kskrf\") pod \"test-model-94a6fc-kserve-57ddd74699-lklnk\" (UID: \"4cf951df-9e8c-4af3-9daf-402266bb8151\") " pod="ai-tenant-e2e-inf-b-94a6fc/test-model-94a6fc-kserve-57ddd74699-lklnk"
Jul 14 14:30:52.695665 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:52.695450    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cf951df-9e8c-4af3-9daf-402266bb8151-home\") pod \"test-model-94a6fc-kserve-57ddd74699-lklnk\" (UID: \"4cf951df-9e8c-4af3-9daf-402266bb8151\") " pod="ai-tenant-e2e-inf-b-94a6fc/test-model-94a6fc-kserve-57ddd74699-lklnk"
Jul 14 14:30:52.695665 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:52.695497    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cf951df-9e8c-4af3-9daf-402266bb8151-dshm\") pod \"test-model-94a6fc-kserve-57ddd74699-lklnk\" (UID: \"4cf951df-9e8c-4af3-9daf-402266bb8151\") " pod="ai-tenant-e2e-inf-b-94a6fc/test-model-94a6fc-kserve-57ddd74699-lklnk"
Jul 14 14:30:52.695665 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:52.695527    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cf951df-9e8c-4af3-9daf-402266bb8151-tls-certs\") pod \"test-model-94a6fc-kserve-57ddd74699-lklnk\" (UID: \"4cf951df-9e8c-4af3-9daf-402266bb8151\") " pod="ai-tenant-e2e-inf-b-94a6fc/test-model-94a6fc-kserve-57ddd74699-lklnk"
Jul 14 14:30:52.695665 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:52.695608    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cf951df-9e8c-4af3-9daf-402266bb8151-tmp-dir\") pod \"test-model-94a6fc-kserve-57ddd74699-lklnk\" (UID: \"4cf951df-9e8c-4af3-9daf-402266bb8151\") " pod="ai-tenant-e2e-inf-b-94a6fc/test-model-94a6fc-kserve-57ddd74699-lklnk"
Jul 14 14:30:52.695911 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:52.695875    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cf951df-9e8c-4af3-9daf-402266bb8151-model-cache\") pod \"test-model-94a6fc-kserve-57ddd74699-lklnk\" (UID: \"4cf951df-9e8c-4af3-9daf-402266bb8151\") " pod="ai-tenant-e2e-inf-b-94a6fc/test-model-94a6fc-kserve-57ddd74699-lklnk"
Jul 14 14:30:52.695911 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:52.695887    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cf951df-9e8c-4af3-9daf-402266bb8151-home\") pod \"test-model-94a6fc-kserve-57ddd74699-lklnk\" (UID: \"4cf951df-9e8c-4af3-9daf-402266bb8151\") " pod="ai-tenant-e2e-inf-b-94a6fc/test-model-94a6fc-kserve-57ddd74699-lklnk"
Jul 14 14:30:52.696075 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:52.695962    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cf951df-9e8c-4af3-9daf-402266bb8151-tmp-dir\") pod \"test-model-94a6fc-kserve-57ddd74699-lklnk\" (UID: \"4cf951df-9e8c-4af3-9daf-402266bb8151\") " pod="ai-tenant-e2e-inf-b-94a6fc/test-model-94a6fc-kserve-57ddd74699-lklnk"
Jul 14 14:30:52.697875 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:52.697853    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cf951df-9e8c-4af3-9daf-402266bb8151-dshm\") pod \"test-model-94a6fc-kserve-57ddd74699-lklnk\" (UID: \"4cf951df-9e8c-4af3-9daf-402266bb8151\") " pod="ai-tenant-e2e-inf-b-94a6fc/test-model-94a6fc-kserve-57ddd74699-lklnk"
Jul 14 14:30:52.698129 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:52.698110    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cf951df-9e8c-4af3-9daf-402266bb8151-tls-certs\") pod \"test-model-94a6fc-kserve-57ddd74699-lklnk\" (UID: \"4cf951df-9e8c-4af3-9daf-402266bb8151\") " pod="ai-tenant-e2e-inf-b-94a6fc/test-model-94a6fc-kserve-57ddd74699-lklnk"
Jul 14 14:30:52.704952 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:52.704929    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kskrf\" (UniqueName: \"kubernetes.io/projected/4cf951df-9e8c-4af3-9daf-402266bb8151-kube-api-access-kskrf\") pod \"test-model-94a6fc-kserve-57ddd74699-lklnk\" (UID: \"4cf951df-9e8c-4af3-9daf-402266bb8151\") " pod="ai-tenant-e2e-inf-b-94a6fc/test-model-94a6fc-kserve-57ddd74699-lklnk"
Jul 14 14:30:52.812927 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:52.812883    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-94a6fc/test-model-94a6fc-kserve-57ddd74699-lklnk"
Jul 14 14:30:52.953478 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:52.953439    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-94a6fc/test-model-94a6fc-kserve-57ddd74699-lklnk"]
Jul 14 14:30:52.956734 ip-10-0-128-127 kubenswrapper[2576]: W0714 14:30:52.956709    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4cf951df_9e8c_4af3_9daf_402266bb8151.slice/crio-07e4414441383cedf9590508a908daa2059f401ab928d068636a845146693a3e WatchSource:0}: Error finding container 07e4414441383cedf9590508a908daa2059f401ab928d068636a845146693a3e: Status 404 returned error can't find the container with id 07e4414441383cedf9590508a908daa2059f401ab928d068636a845146693a3e
Jul 14 14:30:53.241671 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:53.241549    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-94a6fc/test-model-94a6fc-kserve-57ddd74699-lklnk" event={"ID":"4cf951df-9e8c-4af3-9daf-402266bb8151","Type":"ContainerStarted","Data":"422e07088441b4038b52c82ca32f9cda6d74b5f15c353ac252ad1460c6498c54"}
Jul 14 14:30:53.241671 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:53.241623    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-94a6fc/test-model-94a6fc-kserve-57ddd74699-lklnk" event={"ID":"4cf951df-9e8c-4af3-9daf-402266bb8151","Type":"ContainerStarted","Data":"07e4414441383cedf9590508a908daa2059f401ab928d068636a845146693a3e"}
Jul 14 14:30:53.264083 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:53.264025    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-94a6fc/test-model-94a6fc-kserve-57ddd74699-lklnk" podStartSLOduration=1.264009081 podStartE2EDuration="1.264009081s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:30:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:30:53.262865996 +0000 UTC m=+2201.113303513" watchObservedRunningTime="2026-07-14 14:30:53.264009081 +0000 UTC m=+2201.114446585"
Jul 14 14:30:56.621878 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:30:56.621838    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:31:02.813947 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:02.813895    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-94a6fc/test-model-94a6fc-kserve-57ddd74699-lklnk"
Jul 14 14:31:02.814418 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:02.813993    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-94a6fc/test-model-94a6fc-kserve-57ddd74699-lklnk"
Jul 14 14:31:02.826844 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:02.826811    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-94a6fc/test-model-94a6fc-kserve-57ddd74699-lklnk"
Jul 14 14:31:03.300487 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:03.300456    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-94a6fc/test-model-94a6fc-kserve-57ddd74699-lklnk"
Jul 14 14:31:11.358791 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:11.358753    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-0bd8ad/test-model-0bd8ad-kserve-5747687f-6c5qt"]
Jul 14 14:31:11.378238 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:11.378194    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:31:12.476345 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:12.476301    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-94a6fc/test-model-94a6fc-kserve-57ddd74699-lklnk"]
Jul 14 14:31:12.495239 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:12.495208    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:31:13.220258 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:13.220157    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:31:13.333823 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:13.333791    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:31:15.038262 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:15.038223    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:31:23.223260 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:23.223217    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0bd8ad-6f9cddfb7c-lgjxs"]
Jul 14 14:31:23.223742 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:23.223435    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0bd8ad-6f9cddfb7c-lgjxs" podUID="6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916" containerName="maas-api" containerID="cri-o://9dea0df6b27860c8e30a10f575a10d9fdb525b467251ad35aa963cc84588c71c" gracePeriod=30
Jul 14 14:31:23.290067 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:23.290013    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:31:23.382280 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:23.382242    2576 generic.go:356] "Generic (PLEG): container finished" podID="6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916" containerID="9dea0df6b27860c8e30a10f575a10d9fdb525b467251ad35aa963cc84588c71c" exitCode=0
Jul 14 14:31:23.382446 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:23.382321    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0bd8ad-6f9cddfb7c-lgjxs" event={"ID":"6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916","Type":"ContainerDied","Data":"9dea0df6b27860c8e30a10f575a10d9fdb525b467251ad35aa963cc84588c71c"}
Jul 14 14:31:23.473979 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:23.473905    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0bd8ad-6f9cddfb7c-lgjxs"
Jul 14 14:31:23.570580 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:23.570542    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916-openshift-service-ca\") pod \"6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916\" (UID: \"6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916\") "
Jul 14 14:31:23.570800 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:23.570662    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916-maas-api-tls\") pod \"6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916\" (UID: \"6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916\") "
Jul 14 14:31:23.570800 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:23.570764    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ckjpz\" (UniqueName: \"kubernetes.io/projected/6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916-kube-api-access-ckjpz\") pod \"6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916\" (UID: \"6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916\") "
Jul 14 14:31:23.570938 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:23.570910    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916" (UID: "6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 14:31:23.571052 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:23.571035    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916-openshift-service-ca\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:31:23.572745 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:23.572718    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916" (UID: "6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 14:31:23.572859 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:23.572828    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916-kube-api-access-ckjpz" (OuterVolumeSpecName: "kube-api-access-ckjpz") pod "6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916" (UID: "6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916"). InnerVolumeSpecName "kube-api-access-ckjpz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 14:31:23.671870 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:23.671834    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916-maas-api-tls\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:31:23.671870 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:23.671863    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ckjpz\" (UniqueName: \"kubernetes.io/projected/6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916-kube-api-access-ckjpz\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:31:24.388162 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:24.388123    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0bd8ad-6f9cddfb7c-lgjxs" event={"ID":"6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916","Type":"ContainerDied","Data":"212fac783e491751378c001b8be2c3697d3d7cc94c74f4119f284d71ea8d8515"}
Jul 14 14:31:24.388629 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:24.388187    2576 scope.go:117] "RemoveContainer" containerID="9dea0df6b27860c8e30a10f575a10d9fdb525b467251ad35aa963cc84588c71c"
Jul 14 14:31:24.388629 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:24.388145    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0bd8ad-6f9cddfb7c-lgjxs"
Jul 14 14:31:24.411810 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:24.411774    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0bd8ad-6f9cddfb7c-lgjxs"]
Jul 14 14:31:24.415928 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:24.415894    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0bd8ad-6f9cddfb7c-lgjxs"]
Jul 14 14:31:24.688835 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:24.688759    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916" path="/var/lib/kubelet/pods/6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916/volumes"
Jul 14 14:31:26.379199 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:26.379153    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-0bd8ad/test-model-0bd8ad-kserve-5747687f-6c5qt" podUID="69cf130d-9c33-40d5-af9d-3438e8909871" containerName="main" containerID="cri-o://4a93257fa17860115d66420479df4d3a4735fa3cd0432b6d88f5435df395e46d" gracePeriod=45
Jul 14 14:31:26.635310 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:26.635230    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-0bd8ad/test-model-0bd8ad-kserve-5747687f-6c5qt"
Jul 14 14:31:26.696735 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:26.696703    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ds4q7\" (UniqueName: \"kubernetes.io/projected/69cf130d-9c33-40d5-af9d-3438e8909871-kube-api-access-ds4q7\") pod \"69cf130d-9c33-40d5-af9d-3438e8909871\" (UID: \"69cf130d-9c33-40d5-af9d-3438e8909871\") "
Jul 14 14:31:26.696906 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:26.696772    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69cf130d-9c33-40d5-af9d-3438e8909871-tls-certs\") pod \"69cf130d-9c33-40d5-af9d-3438e8909871\" (UID: \"69cf130d-9c33-40d5-af9d-3438e8909871\") "
Jul 14 14:31:26.696906 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:26.696797    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69cf130d-9c33-40d5-af9d-3438e8909871-dshm\") pod \"69cf130d-9c33-40d5-af9d-3438e8909871\" (UID: \"69cf130d-9c33-40d5-af9d-3438e8909871\") "
Jul 14 14:31:26.696906 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:26.696816    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69cf130d-9c33-40d5-af9d-3438e8909871-home\") pod \"69cf130d-9c33-40d5-af9d-3438e8909871\" (UID: \"69cf130d-9c33-40d5-af9d-3438e8909871\") "
Jul 14 14:31:26.696906 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:26.696850    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69cf130d-9c33-40d5-af9d-3438e8909871-model-cache\") pod \"69cf130d-9c33-40d5-af9d-3438e8909871\" (UID: \"69cf130d-9c33-40d5-af9d-3438e8909871\") "
Jul 14 14:31:26.697115 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:26.696945    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69cf130d-9c33-40d5-af9d-3438e8909871-tmp-dir\") pod \"69cf130d-9c33-40d5-af9d-3438e8909871\" (UID: \"69cf130d-9c33-40d5-af9d-3438e8909871\") "
Jul 14 14:31:26.697115 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:26.697082    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69cf130d-9c33-40d5-af9d-3438e8909871-home" (OuterVolumeSpecName: "home") pod "69cf130d-9c33-40d5-af9d-3438e8909871" (UID: "69cf130d-9c33-40d5-af9d-3438e8909871"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 14:31:26.697115 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:26.697095    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69cf130d-9c33-40d5-af9d-3438e8909871-model-cache" (OuterVolumeSpecName: "model-cache") pod "69cf130d-9c33-40d5-af9d-3438e8909871" (UID: "69cf130d-9c33-40d5-af9d-3438e8909871"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 14:31:26.697319 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:26.697287    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69cf130d-9c33-40d5-af9d-3438e8909871-home\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:31:26.697319 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:26.697308    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69cf130d-9c33-40d5-af9d-3438e8909871-model-cache\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:31:26.697319 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:26.697282    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69cf130d-9c33-40d5-af9d-3438e8909871-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "69cf130d-9c33-40d5-af9d-3438e8909871" (UID: "69cf130d-9c33-40d5-af9d-3438e8909871"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 14:31:26.699054 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:26.699025    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69cf130d-9c33-40d5-af9d-3438e8909871-dshm" (OuterVolumeSpecName: "dshm") pod "69cf130d-9c33-40d5-af9d-3438e8909871" (UID: "69cf130d-9c33-40d5-af9d-3438e8909871"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 14:31:26.699186 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:26.699083    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69cf130d-9c33-40d5-af9d-3438e8909871-kube-api-access-ds4q7" (OuterVolumeSpecName: "kube-api-access-ds4q7") pod "69cf130d-9c33-40d5-af9d-3438e8909871" (UID: "69cf130d-9c33-40d5-af9d-3438e8909871"). InnerVolumeSpecName "kube-api-access-ds4q7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 14:31:26.699186 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:26.699122    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69cf130d-9c33-40d5-af9d-3438e8909871-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "69cf130d-9c33-40d5-af9d-3438e8909871" (UID: "69cf130d-9c33-40d5-af9d-3438e8909871"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 14:31:26.798380 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:26.798337    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69cf130d-9c33-40d5-af9d-3438e8909871-tls-certs\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:31:26.798380 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:26.798371    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69cf130d-9c33-40d5-af9d-3438e8909871-dshm\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:31:26.798380 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:26.798379    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69cf130d-9c33-40d5-af9d-3438e8909871-tmp-dir\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:31:26.798380 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:26.798389    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ds4q7\" (UniqueName: \"kubernetes.io/projected/69cf130d-9c33-40d5-af9d-3438e8909871-kube-api-access-ds4q7\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:31:27.402364 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:27.402324    2576 generic.go:356] "Generic (PLEG): container finished" podID="69cf130d-9c33-40d5-af9d-3438e8909871" containerID="4a93257fa17860115d66420479df4d3a4735fa3cd0432b6d88f5435df395e46d" exitCode=0
Jul 14 14:31:27.402878 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:27.402373    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-0bd8ad/test-model-0bd8ad-kserve-5747687f-6c5qt"
Jul 14 14:31:27.402878 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:27.402393    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-0bd8ad/test-model-0bd8ad-kserve-5747687f-6c5qt" event={"ID":"69cf130d-9c33-40d5-af9d-3438e8909871","Type":"ContainerDied","Data":"4a93257fa17860115d66420479df4d3a4735fa3cd0432b6d88f5435df395e46d"}
Jul 14 14:31:27.402878 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:27.402431    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-0bd8ad/test-model-0bd8ad-kserve-5747687f-6c5qt" event={"ID":"69cf130d-9c33-40d5-af9d-3438e8909871","Type":"ContainerDied","Data":"0bc8ba9af3f7628c457f21f468c8e66a1f70448ebcde542588cd9e4c9824f156"}
Jul 14 14:31:27.402878 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:27.402454    2576 scope.go:117] "RemoveContainer" containerID="4a93257fa17860115d66420479df4d3a4735fa3cd0432b6d88f5435df395e46d"
Jul 14 14:31:27.411787 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:27.411765    2576 scope.go:117] "RemoveContainer" containerID="4a93257fa17860115d66420479df4d3a4735fa3cd0432b6d88f5435df395e46d"
Jul 14 14:31:27.412126 ip-10-0-128-127 kubenswrapper[2576]: E0714 14:31:27.412098    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a93257fa17860115d66420479df4d3a4735fa3cd0432b6d88f5435df395e46d\": container with ID starting with 4a93257fa17860115d66420479df4d3a4735fa3cd0432b6d88f5435df395e46d not found: ID does not exist" containerID="4a93257fa17860115d66420479df4d3a4735fa3cd0432b6d88f5435df395e46d"
Jul 14 14:31:27.412197 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:27.412144    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a93257fa17860115d66420479df4d3a4735fa3cd0432b6d88f5435df395e46d"} err="failed to get container status \"4a93257fa17860115d66420479df4d3a4735fa3cd0432b6d88f5435df395e46d\": rpc error: code = NotFound desc = could not find container \"4a93257fa17860115d66420479df4d3a4735fa3cd0432b6d88f5435df395e46d\": container with ID starting with 4a93257fa17860115d66420479df4d3a4735fa3cd0432b6d88f5435df395e46d not found: ID does not exist"
Jul 14 14:31:27.426444 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:27.426405    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-0bd8ad/test-model-0bd8ad-kserve-5747687f-6c5qt"]
Jul 14 14:31:27.430820 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:27.430790    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-0bd8ad/test-model-0bd8ad-kserve-5747687f-6c5qt"]
Jul 14 14:31:27.496613 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:27.496544    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-94a6fc/test-model-94a6fc-kserve-57ddd74699-lklnk" podUID="4cf951df-9e8c-4af3-9daf-402266bb8151" containerName="main" containerID="cri-o://422e07088441b4038b52c82ca32f9cda6d74b5f15c353ac252ad1460c6498c54" gracePeriod=45
Jul 14 14:31:27.685363 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:27.685328    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:31:27.747331 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:27.747302    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-94a6fc/test-model-94a6fc-kserve-57ddd74699-lklnk"
Jul 14 14:31:27.806342 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:27.806308    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cf951df-9e8c-4af3-9daf-402266bb8151-tls-certs\") pod \"4cf951df-9e8c-4af3-9daf-402266bb8151\" (UID: \"4cf951df-9e8c-4af3-9daf-402266bb8151\") "
Jul 14 14:31:27.806549 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:27.806355    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cf951df-9e8c-4af3-9daf-402266bb8151-dshm\") pod \"4cf951df-9e8c-4af3-9daf-402266bb8151\" (UID: \"4cf951df-9e8c-4af3-9daf-402266bb8151\") "
Jul 14 14:31:27.806549 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:27.806397    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cf951df-9e8c-4af3-9daf-402266bb8151-tmp-dir\") pod \"4cf951df-9e8c-4af3-9daf-402266bb8151\" (UID: \"4cf951df-9e8c-4af3-9daf-402266bb8151\") "
Jul 14 14:31:27.806549 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:27.806434    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cf951df-9e8c-4af3-9daf-402266bb8151-model-cache\") pod \"4cf951df-9e8c-4af3-9daf-402266bb8151\" (UID: \"4cf951df-9e8c-4af3-9daf-402266bb8151\") "
Jul 14 14:31:27.806549 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:27.806469    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kskrf\" (UniqueName: \"kubernetes.io/projected/4cf951df-9e8c-4af3-9daf-402266bb8151-kube-api-access-kskrf\") pod \"4cf951df-9e8c-4af3-9daf-402266bb8151\" (UID: \"4cf951df-9e8c-4af3-9daf-402266bb8151\") "
Jul 14 14:31:27.806549 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:27.806492    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cf951df-9e8c-4af3-9daf-402266bb8151-home\") pod \"4cf951df-9e8c-4af3-9daf-402266bb8151\" (UID: \"4cf951df-9e8c-4af3-9daf-402266bb8151\") "
Jul 14 14:31:27.806871 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:27.806741    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cf951df-9e8c-4af3-9daf-402266bb8151-model-cache" (OuterVolumeSpecName: "model-cache") pod "4cf951df-9e8c-4af3-9daf-402266bb8151" (UID: "4cf951df-9e8c-4af3-9daf-402266bb8151"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 14:31:27.806871 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:27.806760    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cf951df-9e8c-4af3-9daf-402266bb8151-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4cf951df-9e8c-4af3-9daf-402266bb8151" (UID: "4cf951df-9e8c-4af3-9daf-402266bb8151"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 14:31:27.806994 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:27.806908    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cf951df-9e8c-4af3-9daf-402266bb8151-home" (OuterVolumeSpecName: "home") pod "4cf951df-9e8c-4af3-9daf-402266bb8151" (UID: "4cf951df-9e8c-4af3-9daf-402266bb8151"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 14:31:27.808799 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:27.808767    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cf951df-9e8c-4af3-9daf-402266bb8151-dshm" (OuterVolumeSpecName: "dshm") pod "4cf951df-9e8c-4af3-9daf-402266bb8151" (UID: "4cf951df-9e8c-4af3-9daf-402266bb8151"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 14:31:27.808898 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:27.808825    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4cf951df-9e8c-4af3-9daf-402266bb8151-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4cf951df-9e8c-4af3-9daf-402266bb8151" (UID: "4cf951df-9e8c-4af3-9daf-402266bb8151"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 14:31:27.808898 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:27.808874    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cf951df-9e8c-4af3-9daf-402266bb8151-kube-api-access-kskrf" (OuterVolumeSpecName: "kube-api-access-kskrf") pod "4cf951df-9e8c-4af3-9daf-402266bb8151" (UID: "4cf951df-9e8c-4af3-9daf-402266bb8151"). InnerVolumeSpecName "kube-api-access-kskrf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 14:31:27.908053 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:27.908013    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kskrf\" (UniqueName: \"kubernetes.io/projected/4cf951df-9e8c-4af3-9daf-402266bb8151-kube-api-access-kskrf\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:31:27.908053 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:27.908044    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cf951df-9e8c-4af3-9daf-402266bb8151-home\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:31:27.908053 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:27.908056    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cf951df-9e8c-4af3-9daf-402266bb8151-tls-certs\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:31:27.908053 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:27.908065    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cf951df-9e8c-4af3-9daf-402266bb8151-dshm\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:31:27.908328 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:27.908074    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cf951df-9e8c-4af3-9daf-402266bb8151-tmp-dir\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:31:27.908328 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:27.908082    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cf951df-9e8c-4af3-9daf-402266bb8151-model-cache\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:31:28.407974 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:28.407934    2576 generic.go:356] "Generic (PLEG): container finished" podID="4cf951df-9e8c-4af3-9daf-402266bb8151" containerID="422e07088441b4038b52c82ca32f9cda6d74b5f15c353ac252ad1460c6498c54" exitCode=0
Jul 14 14:31:28.408482 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:28.407983    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-94a6fc/test-model-94a6fc-kserve-57ddd74699-lklnk"
Jul 14 14:31:28.408482 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:28.408023    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-94a6fc/test-model-94a6fc-kserve-57ddd74699-lklnk" event={"ID":"4cf951df-9e8c-4af3-9daf-402266bb8151","Type":"ContainerDied","Data":"422e07088441b4038b52c82ca32f9cda6d74b5f15c353ac252ad1460c6498c54"}
Jul 14 14:31:28.408482 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:28.408068    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-94a6fc/test-model-94a6fc-kserve-57ddd74699-lklnk" event={"ID":"4cf951df-9e8c-4af3-9daf-402266bb8151","Type":"ContainerDied","Data":"07e4414441383cedf9590508a908daa2059f401ab928d068636a845146693a3e"}
Jul 14 14:31:28.408482 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:28.408090    2576 scope.go:117] "RemoveContainer" containerID="422e07088441b4038b52c82ca32f9cda6d74b5f15c353ac252ad1460c6498c54"
Jul 14 14:31:28.420049 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:28.420023    2576 scope.go:117] "RemoveContainer" containerID="422e07088441b4038b52c82ca32f9cda6d74b5f15c353ac252ad1460c6498c54"
Jul 14 14:31:28.420365 ip-10-0-128-127 kubenswrapper[2576]: E0714 14:31:28.420342    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"422e07088441b4038b52c82ca32f9cda6d74b5f15c353ac252ad1460c6498c54\": container with ID starting with 422e07088441b4038b52c82ca32f9cda6d74b5f15c353ac252ad1460c6498c54 not found: ID does not exist" containerID="422e07088441b4038b52c82ca32f9cda6d74b5f15c353ac252ad1460c6498c54"
Jul 14 14:31:28.420438 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:28.420374    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"422e07088441b4038b52c82ca32f9cda6d74b5f15c353ac252ad1460c6498c54"} err="failed to get container status \"422e07088441b4038b52c82ca32f9cda6d74b5f15c353ac252ad1460c6498c54\": rpc error: code = NotFound desc = could not find container \"422e07088441b4038b52c82ca32f9cda6d74b5f15c353ac252ad1460c6498c54\": container with ID starting with 422e07088441b4038b52c82ca32f9cda6d74b5f15c353ac252ad1460c6498c54 not found: ID does not exist"
Jul 14 14:31:28.434457 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:28.434424    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-94a6fc/test-model-94a6fc-kserve-57ddd74699-lklnk"]
Jul 14 14:31:28.438215 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:28.438188    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-94a6fc/test-model-94a6fc-kserve-57ddd74699-lklnk"]
Jul 14 14:31:28.689303 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:28.689221    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4cf951df-9e8c-4af3-9daf-402266bb8151" path="/var/lib/kubelet/pods/4cf951df-9e8c-4af3-9daf-402266bb8151/volumes"
Jul 14 14:31:28.689628 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:28.689614    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69cf130d-9c33-40d5-af9d-3438e8909871" path="/var/lib/kubelet/pods/69cf130d-9c33-40d5-af9d-3438e8909871/volumes"
Jul 14 14:31:32.189519 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:32.189483    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:31:33.740916 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:33.740878    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:31:33.816974 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:33.816936    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:31:34.512222 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:34.512186    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:31:38.820732 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:38.820696    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:31:38.894758 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:38.894723    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:31:39.526282 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:39.526235    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:31:39.627883 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:39.627851    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:31:49.701482 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:49.701443    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-94a6fc-bb779b8f6-tbvhx"]
Jul 14 14:31:49.701991 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:49.701753    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a6fc-bb779b8f6-tbvhx" podUID="eeea8f9c-420d-4ebc-b2a2-9996b270681c" containerName="maas-api" containerID="cri-o://e21f071449ddc92029bb246c770334aa373298e68454a61538483587baa7f735" gracePeriod=30
Jul 14 14:31:49.736268 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:49.736233    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:31:49.951002 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:49.950975    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a6fc-bb779b8f6-tbvhx"
Jul 14 14:31:50.115829 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:50.115788    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x47dl\" (UniqueName: \"kubernetes.io/projected/eeea8f9c-420d-4ebc-b2a2-9996b270681c-kube-api-access-x47dl\") pod \"eeea8f9c-420d-4ebc-b2a2-9996b270681c\" (UID: \"eeea8f9c-420d-4ebc-b2a2-9996b270681c\") "
Jul 14 14:31:50.116040 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:50.115852    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eeea8f9c-420d-4ebc-b2a2-9996b270681c-maas-api-tls\") pod \"eeea8f9c-420d-4ebc-b2a2-9996b270681c\" (UID: \"eeea8f9c-420d-4ebc-b2a2-9996b270681c\") "
Jul 14 14:31:50.116040 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:50.115871    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eeea8f9c-420d-4ebc-b2a2-9996b270681c-openshift-service-ca\") pod \"eeea8f9c-420d-4ebc-b2a2-9996b270681c\" (UID: \"eeea8f9c-420d-4ebc-b2a2-9996b270681c\") "
Jul 14 14:31:50.116345 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:50.116309    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eeea8f9c-420d-4ebc-b2a2-9996b270681c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "eeea8f9c-420d-4ebc-b2a2-9996b270681c" (UID: "eeea8f9c-420d-4ebc-b2a2-9996b270681c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 14:31:50.118042 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:50.118017    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eeea8f9c-420d-4ebc-b2a2-9996b270681c-kube-api-access-x47dl" (OuterVolumeSpecName: "kube-api-access-x47dl") pod "eeea8f9c-420d-4ebc-b2a2-9996b270681c" (UID: "eeea8f9c-420d-4ebc-b2a2-9996b270681c"). InnerVolumeSpecName "kube-api-access-x47dl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 14:31:50.118146 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:50.118013    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eeea8f9c-420d-4ebc-b2a2-9996b270681c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "eeea8f9c-420d-4ebc-b2a2-9996b270681c" (UID: "eeea8f9c-420d-4ebc-b2a2-9996b270681c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 14:31:50.217394 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:50.217356    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x47dl\" (UniqueName: \"kubernetes.io/projected/eeea8f9c-420d-4ebc-b2a2-9996b270681c-kube-api-access-x47dl\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:31:50.217394 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:50.217388    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eeea8f9c-420d-4ebc-b2a2-9996b270681c-maas-api-tls\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:31:50.217394 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:50.217398    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eeea8f9c-420d-4ebc-b2a2-9996b270681c-openshift-service-ca\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:31:50.509707 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:50.509617    2576 generic.go:356] "Generic (PLEG): container finished" podID="eeea8f9c-420d-4ebc-b2a2-9996b270681c" containerID="e21f071449ddc92029bb246c770334aa373298e68454a61538483587baa7f735" exitCode=0
Jul 14 14:31:50.509707 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:50.509665    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a6fc-bb779b8f6-tbvhx"
Jul 14 14:31:50.509891 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:50.509707    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a6fc-bb779b8f6-tbvhx" event={"ID":"eeea8f9c-420d-4ebc-b2a2-9996b270681c","Type":"ContainerDied","Data":"e21f071449ddc92029bb246c770334aa373298e68454a61538483587baa7f735"}
Jul 14 14:31:50.509891 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:50.509768    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a6fc-bb779b8f6-tbvhx" event={"ID":"eeea8f9c-420d-4ebc-b2a2-9996b270681c","Type":"ContainerDied","Data":"078adec8fa6e337e7b2350bc054d526aaf3b1745675292b09f7213d24a04bb8a"}
Jul 14 14:31:50.509891 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:50.509797    2576 scope.go:117] "RemoveContainer" containerID="e21f071449ddc92029bb246c770334aa373298e68454a61538483587baa7f735"
Jul 14 14:31:50.518717 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:50.518694    2576 scope.go:117] "RemoveContainer" containerID="e21f071449ddc92029bb246c770334aa373298e68454a61538483587baa7f735"
Jul 14 14:31:50.518996 ip-10-0-128-127 kubenswrapper[2576]: E0714 14:31:50.518977    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e21f071449ddc92029bb246c770334aa373298e68454a61538483587baa7f735\": container with ID starting with e21f071449ddc92029bb246c770334aa373298e68454a61538483587baa7f735 not found: ID does not exist" containerID="e21f071449ddc92029bb246c770334aa373298e68454a61538483587baa7f735"
Jul 14 14:31:50.519042 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:50.519005    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e21f071449ddc92029bb246c770334aa373298e68454a61538483587baa7f735"} err="failed to get container status \"e21f071449ddc92029bb246c770334aa373298e68454a61538483587baa7f735\": rpc error: code = NotFound desc = could not find container \"e21f071449ddc92029bb246c770334aa373298e68454a61538483587baa7f735\": container with ID starting with e21f071449ddc92029bb246c770334aa373298e68454a61538483587baa7f735 not found: ID does not exist"
Jul 14 14:31:50.532733 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:50.532703    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-94a6fc-bb779b8f6-tbvhx"]
Jul 14 14:31:50.536486 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:50.536461    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-94a6fc-bb779b8f6-tbvhx"]
Jul 14 14:31:50.688401 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:50.688372    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eeea8f9c-420d-4ebc-b2a2-9996b270681c" path="/var/lib/kubelet/pods/eeea8f9c-420d-4ebc-b2a2-9996b270681c/volumes"
Jul 14 14:31:52.827000 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:52.826967    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:31:54.074785 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:31:54.074744    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:32:00.273346 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:00.273291    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:32:00.374571 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:00.374534    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:32:04.864580 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:04.864536    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:32:05.121581 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:05.121496    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:32:05.537636 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:05.537581    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:32:05.981293 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:05.981203    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:32:06.277812 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:06.277752    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:32:09.260798 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:09.260752    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:32:09.344607 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:09.344566    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:32:09.349066 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:09.349033    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a7298c-746bb6fcdc-fz8ql"]
Jul 14 14:32:09.349400 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:09.349386    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cf951df-9e8c-4af3-9daf-402266bb8151" containerName="main"
Jul 14 14:32:09.349447 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:09.349402    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cf951df-9e8c-4af3-9daf-402266bb8151" containerName="main"
Jul 14 14:32:09.349447 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:09.349416    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eeea8f9c-420d-4ebc-b2a2-9996b270681c" containerName="maas-api"
Jul 14 14:32:09.349447 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:09.349422    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="eeea8f9c-420d-4ebc-b2a2-9996b270681c" containerName="maas-api"
Jul 14 14:32:09.349447 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:09.349434    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69cf130d-9c33-40d5-af9d-3438e8909871" containerName="main"
Jul 14 14:32:09.349447 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:09.349440    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="69cf130d-9c33-40d5-af9d-3438e8909871" containerName="main"
Jul 14 14:32:09.349447 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:09.349446    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916" containerName="maas-api"
Jul 14 14:32:09.349654 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:09.349453    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916" containerName="maas-api"
Jul 14 14:32:09.349654 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:09.349500    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="eeea8f9c-420d-4ebc-b2a2-9996b270681c" containerName="maas-api"
Jul 14 14:32:09.349654 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:09.349507    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="69cf130d-9c33-40d5-af9d-3438e8909871" containerName="main"
Jul 14 14:32:09.349654 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:09.349515    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4cf951df-9e8c-4af3-9daf-402266bb8151" containerName="main"
Jul 14 14:32:09.349654 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:09.349521    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d009e6e-68e3-4bed-9f0e-cd4d2c0f4916" containerName="maas-api"
Jul 14 14:32:09.354076 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:09.354054    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7298c-746bb6fcdc-fz8ql"
Jul 14 14:32:09.356691 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:09.356664    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 14 14:32:09.356910 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:09.356671    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-a7298c-serving-cert\""
Jul 14 14:32:09.356910 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:09.356886    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-znm9j\""
Jul 14 14:32:09.357724 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:09.357709    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 14:32:09.361861 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:09.361824    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a7298c-746bb6fcdc-fz8ql"]
Jul 14 14:32:09.382492 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:09.382461    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9t2s\" (UniqueName: \"kubernetes.io/projected/7fb93465-21a9-482b-83b2-90f13ca12f31-kube-api-access-n9t2s\") pod \"maas-api-e2e-api-a-a7298c-746bb6fcdc-fz8ql\" (UID: \"7fb93465-21a9-482b-83b2-90f13ca12f31\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7298c-746bb6fcdc-fz8ql"
Jul 14 14:32:09.382711 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:09.382508    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fb93465-21a9-482b-83b2-90f13ca12f31-openshift-service-ca\") pod \"maas-api-e2e-api-a-a7298c-746bb6fcdc-fz8ql\" (UID: \"7fb93465-21a9-482b-83b2-90f13ca12f31\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7298c-746bb6fcdc-fz8ql"
Jul 14 14:32:09.382711 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:09.382567    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7fb93465-21a9-482b-83b2-90f13ca12f31-maas-api-tls\") pod \"maas-api-e2e-api-a-a7298c-746bb6fcdc-fz8ql\" (UID: \"7fb93465-21a9-482b-83b2-90f13ca12f31\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7298c-746bb6fcdc-fz8ql"
Jul 14 14:32:09.419991 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:09.419950    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:32:09.483219 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:09.483180    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9t2s\" (UniqueName: \"kubernetes.io/projected/7fb93465-21a9-482b-83b2-90f13ca12f31-kube-api-access-n9t2s\") pod \"maas-api-e2e-api-a-a7298c-746bb6fcdc-fz8ql\" (UID: \"7fb93465-21a9-482b-83b2-90f13ca12f31\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7298c-746bb6fcdc-fz8ql"
Jul 14 14:32:09.483390 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:09.483241    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fb93465-21a9-482b-83b2-90f13ca12f31-openshift-service-ca\") pod \"maas-api-e2e-api-a-a7298c-746bb6fcdc-fz8ql\" (UID: \"7fb93465-21a9-482b-83b2-90f13ca12f31\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7298c-746bb6fcdc-fz8ql"
Jul 14 14:32:09.483390 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:09.483289    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7fb93465-21a9-482b-83b2-90f13ca12f31-maas-api-tls\") pod \"maas-api-e2e-api-a-a7298c-746bb6fcdc-fz8ql\" (UID: \"7fb93465-21a9-482b-83b2-90f13ca12f31\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7298c-746bb6fcdc-fz8ql"
Jul 14 14:32:09.483390 ip-10-0-128-127 kubenswrapper[2576]: E0714 14:32:09.483380    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-a7298c-serving-cert: secret "maas-api-e2e-api-a-a7298c-serving-cert" not found
Jul 14 14:32:09.483491 ip-10-0-128-127 kubenswrapper[2576]: E0714 14:32:09.483455    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fb93465-21a9-482b-83b2-90f13ca12f31-maas-api-tls podName:7fb93465-21a9-482b-83b2-90f13ca12f31 nodeName:}" failed. No retries permitted until 2026-07-14 14:32:09.983434198 +0000 UTC m=+2277.833871682 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7fb93465-21a9-482b-83b2-90f13ca12f31-maas-api-tls") pod "maas-api-e2e-api-a-a7298c-746bb6fcdc-fz8ql" (UID: "7fb93465-21a9-482b-83b2-90f13ca12f31") : secret "maas-api-e2e-api-a-a7298c-serving-cert" not found
Jul 14 14:32:09.483964 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:09.483945    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fb93465-21a9-482b-83b2-90f13ca12f31-openshift-service-ca\") pod \"maas-api-e2e-api-a-a7298c-746bb6fcdc-fz8ql\" (UID: \"7fb93465-21a9-482b-83b2-90f13ca12f31\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7298c-746bb6fcdc-fz8ql"
Jul 14 14:32:09.492374 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:09.492346    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9t2s\" (UniqueName: \"kubernetes.io/projected/7fb93465-21a9-482b-83b2-90f13ca12f31-kube-api-access-n9t2s\") pod \"maas-api-e2e-api-a-a7298c-746bb6fcdc-fz8ql\" (UID: \"7fb93465-21a9-482b-83b2-90f13ca12f31\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7298c-746bb6fcdc-fz8ql"
Jul 14 14:32:09.989229 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:09.989189    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7fb93465-21a9-482b-83b2-90f13ca12f31-maas-api-tls\") pod \"maas-api-e2e-api-a-a7298c-746bb6fcdc-fz8ql\" (UID: \"7fb93465-21a9-482b-83b2-90f13ca12f31\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7298c-746bb6fcdc-fz8ql"
Jul 14 14:32:09.991604 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:09.991567    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7fb93465-21a9-482b-83b2-90f13ca12f31-maas-api-tls\") pod \"maas-api-e2e-api-a-a7298c-746bb6fcdc-fz8ql\" (UID: \"7fb93465-21a9-482b-83b2-90f13ca12f31\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7298c-746bb6fcdc-fz8ql"
Jul 14 14:32:10.000359 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:10.000325    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:32:10.100108 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:10.100059    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:32:10.207297 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:10.207263    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:32:10.267173 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:10.267084    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7298c-746bb6fcdc-fz8ql"
Jul 14 14:32:10.406878 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:10.406838    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a7298c-746bb6fcdc-fz8ql"]
Jul 14 14:32:10.409748 ip-10-0-128-127 kubenswrapper[2576]: W0714 14:32:10.409709    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fb93465_21a9_482b_83b2_90f13ca12f31.slice/crio-4da67d51b5e1652bd14d14fdaa27593b4de9c26984ac49450c714288235ea9e9 WatchSource:0}: Error finding container 4da67d51b5e1652bd14d14fdaa27593b4de9c26984ac49450c714288235ea9e9: Status 404 returned error can't find the container with id 4da67d51b5e1652bd14d14fdaa27593b4de9c26984ac49450c714288235ea9e9
Jul 14 14:32:10.410926 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:10.410905    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 14:32:10.602671 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:10.602632    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7298c-746bb6fcdc-fz8ql" event={"ID":"7fb93465-21a9-482b-83b2-90f13ca12f31","Type":"ContainerStarted","Data":"4da67d51b5e1652bd14d14fdaa27593b4de9c26984ac49450c714288235ea9e9"}
Jul 14 14:32:13.118374 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:13.118341    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:32:13.622488 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:13.622454    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7298c-746bb6fcdc-fz8ql" event={"ID":"7fb93465-21a9-482b-83b2-90f13ca12f31","Type":"ContainerStarted","Data":"5c3e62e3b47d935b9918d02c2752aa3d927ffa4202f1c4571355f8338b564d10"}
Jul 14 14:32:13.622716 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:13.622569    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7298c-746bb6fcdc-fz8ql"
Jul 14 14:32:13.642583 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:13.642524    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7298c-746bb6fcdc-fz8ql" podStartSLOduration=2.364309474 podStartE2EDuration="4.642503934s" totalImagesPullingTime="2.27819446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:32:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:32:13.640747101 +0000 UTC m=+2281.491184604" watchObservedRunningTime="2026-07-14 14:32:13.642503934 +0000 UTC m=+2281.492941437"
Jul 14 14:32:13.878718 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:13.878579    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:32:14.414471 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:14.414430    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:32:14.779250 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:14.779215    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:32:15.080482 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:15.080367    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:32:18.342075 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:18.342026    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:32:18.415158 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:18.415123    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9343ac-56979b549-2mt7s"]
Jul 14 14:32:18.418586 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:18.418561    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9343ac-56979b549-2mt7s"
Jul 14 14:32:18.421195 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:18.421161    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-9343ac-serving-cert\""
Jul 14 14:32:18.431572 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:18.431542    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9343ac-56979b549-2mt7s"]
Jul 14 14:32:18.438132 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:18.438105    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:32:18.467566 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:18.467476    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkslz\" (UniqueName: \"kubernetes.io/projected/6ebbf673-662b-4489-ad47-e165da8ef178-kube-api-access-pkslz\") pod \"maas-api-e2e-api-b-9343ac-56979b549-2mt7s\" (UID: \"6ebbf673-662b-4489-ad47-e165da8ef178\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9343ac-56979b549-2mt7s"
Jul 14 14:32:18.467566 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:18.467535    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ebbf673-662b-4489-ad47-e165da8ef178-openshift-service-ca\") pod \"maas-api-e2e-api-b-9343ac-56979b549-2mt7s\" (UID: \"6ebbf673-662b-4489-ad47-e165da8ef178\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9343ac-56979b549-2mt7s"
Jul 14 14:32:18.467838 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:18.467637    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ebbf673-662b-4489-ad47-e165da8ef178-maas-api-tls\") pod \"maas-api-e2e-api-b-9343ac-56979b549-2mt7s\" (UID: \"6ebbf673-662b-4489-ad47-e165da8ef178\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9343ac-56979b549-2mt7s"
Jul 14 14:32:18.505093 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:18.505048    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:32:18.568817 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:18.568781    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkslz\" (UniqueName: \"kubernetes.io/projected/6ebbf673-662b-4489-ad47-e165da8ef178-kube-api-access-pkslz\") pod \"maas-api-e2e-api-b-9343ac-56979b549-2mt7s\" (UID: \"6ebbf673-662b-4489-ad47-e165da8ef178\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9343ac-56979b549-2mt7s"
Jul 14 14:32:18.569035 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:18.568827    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ebbf673-662b-4489-ad47-e165da8ef178-openshift-service-ca\") pod \"maas-api-e2e-api-b-9343ac-56979b549-2mt7s\" (UID: \"6ebbf673-662b-4489-ad47-e165da8ef178\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9343ac-56979b549-2mt7s"
Jul 14 14:32:18.569035 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:18.568858    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ebbf673-662b-4489-ad47-e165da8ef178-maas-api-tls\") pod \"maas-api-e2e-api-b-9343ac-56979b549-2mt7s\" (UID: \"6ebbf673-662b-4489-ad47-e165da8ef178\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9343ac-56979b549-2mt7s"
Jul 14 14:32:18.569035 ip-10-0-128-127 kubenswrapper[2576]: E0714 14:32:18.568989    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-9343ac-serving-cert: secret "maas-api-e2e-api-b-9343ac-serving-cert" not found
Jul 14 14:32:18.569219 ip-10-0-128-127 kubenswrapper[2576]: E0714 14:32:18.569052    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ebbf673-662b-4489-ad47-e165da8ef178-maas-api-tls podName:6ebbf673-662b-4489-ad47-e165da8ef178 nodeName:}" failed. No retries permitted until 2026-07-14 14:32:19.069036077 +0000 UTC m=+2286.919473557 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6ebbf673-662b-4489-ad47-e165da8ef178-maas-api-tls") pod "maas-api-e2e-api-b-9343ac-56979b549-2mt7s" (UID: "6ebbf673-662b-4489-ad47-e165da8ef178") : secret "maas-api-e2e-api-b-9343ac-serving-cert" not found
Jul 14 14:32:18.569576 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:18.569550    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ebbf673-662b-4489-ad47-e165da8ef178-openshift-service-ca\") pod \"maas-api-e2e-api-b-9343ac-56979b549-2mt7s\" (UID: \"6ebbf673-662b-4489-ad47-e165da8ef178\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9343ac-56979b549-2mt7s"
Jul 14 14:32:18.578279 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:18.578244    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkslz\" (UniqueName: \"kubernetes.io/projected/6ebbf673-662b-4489-ad47-e165da8ef178-kube-api-access-pkslz\") pod \"maas-api-e2e-api-b-9343ac-56979b549-2mt7s\" (UID: \"6ebbf673-662b-4489-ad47-e165da8ef178\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9343ac-56979b549-2mt7s"
Jul 14 14:32:19.023291 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:19.023226    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:32:19.073854 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:19.073814    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ebbf673-662b-4489-ad47-e165da8ef178-maas-api-tls\") pod \"maas-api-e2e-api-b-9343ac-56979b549-2mt7s\" (UID: \"6ebbf673-662b-4489-ad47-e165da8ef178\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9343ac-56979b549-2mt7s"
Jul 14 14:32:19.076693 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:19.076652    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ebbf673-662b-4489-ad47-e165da8ef178-maas-api-tls\") pod \"maas-api-e2e-api-b-9343ac-56979b549-2mt7s\" (UID: \"6ebbf673-662b-4489-ad47-e165da8ef178\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9343ac-56979b549-2mt7s"
Jul 14 14:32:19.140163 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:19.140122    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:32:19.232161 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:19.232116    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:32:19.330082 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:19.329987    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9343ac-56979b549-2mt7s"
Jul 14 14:32:19.480973 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:19.480930    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9343ac-56979b549-2mt7s"]
Jul 14 14:32:19.483704 ip-10-0-128-127 kubenswrapper[2576]: W0714 14:32:19.483650    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ebbf673_662b_4489_ad47_e165da8ef178.slice/crio-f05c1f9dee541e171c73809a57031024c0b107d1f8ccd8dc3dfb090509de68d0 WatchSource:0}: Error finding container f05c1f9dee541e171c73809a57031024c0b107d1f8ccd8dc3dfb090509de68d0: Status 404 returned error can't find the container with id f05c1f9dee541e171c73809a57031024c0b107d1f8ccd8dc3dfb090509de68d0
Jul 14 14:32:19.634386 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:19.634299    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7298c-746bb6fcdc-fz8ql"
Jul 14 14:32:19.649474 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:19.649437    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9343ac-56979b549-2mt7s" event={"ID":"6ebbf673-662b-4489-ad47-e165da8ef178","Type":"ContainerStarted","Data":"f05c1f9dee541e171c73809a57031024c0b107d1f8ccd8dc3dfb090509de68d0"}
Jul 14 14:32:21.659473 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:21.659438    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9343ac-56979b549-2mt7s" event={"ID":"6ebbf673-662b-4489-ad47-e165da8ef178","Type":"ContainerStarted","Data":"99e9ed50d90c24e38f3fb1a0ec9566c3b01857cafdfe95aa71db41829879f101"}
Jul 14 14:32:21.659909 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:21.659565    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9343ac-56979b549-2mt7s"
Jul 14 14:32:21.687272 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:21.687216    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9343ac-56979b549-2mt7s" podStartSLOduration=2.037363577 podStartE2EDuration="3.68719931s" totalImagesPullingTime="1.649835733s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:32:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:32:21.685994377 +0000 UTC m=+2289.536431880" watchObservedRunningTime="2026-07-14 14:32:21.68719931 +0000 UTC m=+2289.537636812"
Jul 14 14:32:22.892885 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:22.892847    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:32:26.716433 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:26.716395    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:32:27.669061 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:27.669033    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9343ac-56979b549-2mt7s"
Jul 14 14:32:30.994479 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:30.994426    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:32:31.092559 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:31.092522    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:32:41.034365 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:41.034318    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:32:41.143400 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:41.143356    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:32:41.751712 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:41.751667    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a7298c-746bb6fcdc-fz8ql"]
Jul 14 14:32:41.752027 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:41.751966    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7298c-746bb6fcdc-fz8ql" podUID="7fb93465-21a9-482b-83b2-90f13ca12f31" containerName="maas-api" containerID="cri-o://5c3e62e3b47d935b9918d02c2752aa3d927ffa4202f1c4571355f8338b564d10" gracePeriod=30
Jul 14 14:32:41.774810 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:41.774773    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:32:42.003628 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:42.003536    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7298c-746bb6fcdc-fz8ql"
Jul 14 14:32:42.085211 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:42.085170    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7fb93465-21a9-482b-83b2-90f13ca12f31-maas-api-tls\") pod \"7fb93465-21a9-482b-83b2-90f13ca12f31\" (UID: \"7fb93465-21a9-482b-83b2-90f13ca12f31\") "
Jul 14 14:32:42.085703 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:42.085299    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n9t2s\" (UniqueName: \"kubernetes.io/projected/7fb93465-21a9-482b-83b2-90f13ca12f31-kube-api-access-n9t2s\") pod \"7fb93465-21a9-482b-83b2-90f13ca12f31\" (UID: \"7fb93465-21a9-482b-83b2-90f13ca12f31\") "
Jul 14 14:32:42.085703 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:42.085334    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fb93465-21a9-482b-83b2-90f13ca12f31-openshift-service-ca\") pod \"7fb93465-21a9-482b-83b2-90f13ca12f31\" (UID: \"7fb93465-21a9-482b-83b2-90f13ca12f31\") "
Jul 14 14:32:42.085784 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:42.085732    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7fb93465-21a9-482b-83b2-90f13ca12f31-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7fb93465-21a9-482b-83b2-90f13ca12f31" (UID: "7fb93465-21a9-482b-83b2-90f13ca12f31"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 14:32:42.087220 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:42.087187    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7fb93465-21a9-482b-83b2-90f13ca12f31-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7fb93465-21a9-482b-83b2-90f13ca12f31" (UID: "7fb93465-21a9-482b-83b2-90f13ca12f31"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 14:32:42.087334 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:42.087318    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fb93465-21a9-482b-83b2-90f13ca12f31-kube-api-access-n9t2s" (OuterVolumeSpecName: "kube-api-access-n9t2s") pod "7fb93465-21a9-482b-83b2-90f13ca12f31" (UID: "7fb93465-21a9-482b-83b2-90f13ca12f31"). InnerVolumeSpecName "kube-api-access-n9t2s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 14:32:42.186947 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:42.186904    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9t2s\" (UniqueName: \"kubernetes.io/projected/7fb93465-21a9-482b-83b2-90f13ca12f31-kube-api-access-n9t2s\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:32:42.186947 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:42.186941    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fb93465-21a9-482b-83b2-90f13ca12f31-openshift-service-ca\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:32:42.186947 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:42.186952    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7fb93465-21a9-482b-83b2-90f13ca12f31-maas-api-tls\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:32:42.754437 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:42.754394    2576 generic.go:356] "Generic (PLEG): container finished" podID="7fb93465-21a9-482b-83b2-90f13ca12f31" containerID="5c3e62e3b47d935b9918d02c2752aa3d927ffa4202f1c4571355f8338b564d10" exitCode=0
Jul 14 14:32:42.754675 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:42.754444    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7298c-746bb6fcdc-fz8ql"
Jul 14 14:32:42.754675 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:42.754459    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7298c-746bb6fcdc-fz8ql" event={"ID":"7fb93465-21a9-482b-83b2-90f13ca12f31","Type":"ContainerDied","Data":"5c3e62e3b47d935b9918d02c2752aa3d927ffa4202f1c4571355f8338b564d10"}
Jul 14 14:32:42.754675 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:42.754508    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7298c-746bb6fcdc-fz8ql" event={"ID":"7fb93465-21a9-482b-83b2-90f13ca12f31","Type":"ContainerDied","Data":"4da67d51b5e1652bd14d14fdaa27593b4de9c26984ac49450c714288235ea9e9"}
Jul 14 14:32:42.754675 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:42.754531    2576 scope.go:117] "RemoveContainer" containerID="5c3e62e3b47d935b9918d02c2752aa3d927ffa4202f1c4571355f8338b564d10"
Jul 14 14:32:42.765027 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:42.764981    2576 scope.go:117] "RemoveContainer" containerID="5c3e62e3b47d935b9918d02c2752aa3d927ffa4202f1c4571355f8338b564d10"
Jul 14 14:32:42.765472 ip-10-0-128-127 kubenswrapper[2576]: E0714 14:32:42.765451    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c3e62e3b47d935b9918d02c2752aa3d927ffa4202f1c4571355f8338b564d10\": container with ID starting with 5c3e62e3b47d935b9918d02c2752aa3d927ffa4202f1c4571355f8338b564d10 not found: ID does not exist" containerID="5c3e62e3b47d935b9918d02c2752aa3d927ffa4202f1c4571355f8338b564d10"
Jul 14 14:32:42.765521 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:42.765483    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c3e62e3b47d935b9918d02c2752aa3d927ffa4202f1c4571355f8338b564d10"} err="failed to get container status \"5c3e62e3b47d935b9918d02c2752aa3d927ffa4202f1c4571355f8338b564d10\": rpc error: code = NotFound desc = could not find container \"5c3e62e3b47d935b9918d02c2752aa3d927ffa4202f1c4571355f8338b564d10\": container with ID starting with 5c3e62e3b47d935b9918d02c2752aa3d927ffa4202f1c4571355f8338b564d10 not found: ID does not exist"
Jul 14 14:32:42.772283 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:42.772242    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a7298c-746bb6fcdc-fz8ql"]
Jul 14 14:32:42.777280 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:42.777249    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a7298c-746bb6fcdc-fz8ql"]
Jul 14 14:32:44.689132 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:44.689096    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7fb93465-21a9-482b-83b2-90f13ca12f31" path="/var/lib/kubelet/pods/7fb93465-21a9-482b-83b2-90f13ca12f31/volumes"
Jul 14 14:32:45.308377 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:45.308324    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:32:46.668330 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:46.668289    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:32:52.782412 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:52.782377    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:32:52.879758 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:52.879723    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:32:57.457412 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:57.457359    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:32:57.733066 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:32:57.732965    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:33:08.349085 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:08.348988    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9343ac-56979b549-2mt7s"]
Jul 14 14:33:08.349587 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:08.349248    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9343ac-56979b549-2mt7s" podUID="6ebbf673-662b-4489-ad47-e165da8ef178" containerName="maas-api" containerID="cri-o://99e9ed50d90c24e38f3fb1a0ec9566c3b01857cafdfe95aa71db41829879f101" gracePeriod=30
Jul 14 14:33:08.376656 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:08.376623    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:33:08.587162 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:08.587135    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9343ac-56979b549-2mt7s"
Jul 14 14:33:08.712451 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:08.712347    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ebbf673-662b-4489-ad47-e165da8ef178-maas-api-tls\") pod \"6ebbf673-662b-4489-ad47-e165da8ef178\" (UID: \"6ebbf673-662b-4489-ad47-e165da8ef178\") "
Jul 14 14:33:08.712451 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:08.712408    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ebbf673-662b-4489-ad47-e165da8ef178-openshift-service-ca\") pod \"6ebbf673-662b-4489-ad47-e165da8ef178\" (UID: \"6ebbf673-662b-4489-ad47-e165da8ef178\") "
Jul 14 14:33:08.712730 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:08.712544    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pkslz\" (UniqueName: \"kubernetes.io/projected/6ebbf673-662b-4489-ad47-e165da8ef178-kube-api-access-pkslz\") pod \"6ebbf673-662b-4489-ad47-e165da8ef178\" (UID: \"6ebbf673-662b-4489-ad47-e165da8ef178\") "
Jul 14 14:33:08.712860 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:08.712836    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ebbf673-662b-4489-ad47-e165da8ef178-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6ebbf673-662b-4489-ad47-e165da8ef178" (UID: "6ebbf673-662b-4489-ad47-e165da8ef178"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 14:33:08.712931 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:08.712913    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ebbf673-662b-4489-ad47-e165da8ef178-openshift-service-ca\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:33:08.714585 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:08.714556    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ebbf673-662b-4489-ad47-e165da8ef178-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6ebbf673-662b-4489-ad47-e165da8ef178" (UID: "6ebbf673-662b-4489-ad47-e165da8ef178"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 14:33:08.714727 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:08.714627    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ebbf673-662b-4489-ad47-e165da8ef178-kube-api-access-pkslz" (OuterVolumeSpecName: "kube-api-access-pkslz") pod "6ebbf673-662b-4489-ad47-e165da8ef178" (UID: "6ebbf673-662b-4489-ad47-e165da8ef178"). InnerVolumeSpecName "kube-api-access-pkslz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 14:33:08.813956 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:08.813920    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pkslz\" (UniqueName: \"kubernetes.io/projected/6ebbf673-662b-4489-ad47-e165da8ef178-kube-api-access-pkslz\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:33:08.813956 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:08.813948    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ebbf673-662b-4489-ad47-e165da8ef178-maas-api-tls\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:33:08.868871 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:08.868840    2576 generic.go:356] "Generic (PLEG): container finished" podID="6ebbf673-662b-4489-ad47-e165da8ef178" containerID="99e9ed50d90c24e38f3fb1a0ec9566c3b01857cafdfe95aa71db41829879f101" exitCode=0
Jul 14 14:33:08.869047 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:08.868885    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9343ac-56979b549-2mt7s"
Jul 14 14:33:08.869047 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:08.868910    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9343ac-56979b549-2mt7s" event={"ID":"6ebbf673-662b-4489-ad47-e165da8ef178","Type":"ContainerDied","Data":"99e9ed50d90c24e38f3fb1a0ec9566c3b01857cafdfe95aa71db41829879f101"}
Jul 14 14:33:08.869047 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:08.868954    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9343ac-56979b549-2mt7s" event={"ID":"6ebbf673-662b-4489-ad47-e165da8ef178","Type":"ContainerDied","Data":"f05c1f9dee541e171c73809a57031024c0b107d1f8ccd8dc3dfb090509de68d0"}
Jul 14 14:33:08.869047 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:08.868976    2576 scope.go:117] "RemoveContainer" containerID="99e9ed50d90c24e38f3fb1a0ec9566c3b01857cafdfe95aa71db41829879f101"
Jul 14 14:33:08.877884 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:08.877860    2576 scope.go:117] "RemoveContainer" containerID="99e9ed50d90c24e38f3fb1a0ec9566c3b01857cafdfe95aa71db41829879f101"
Jul 14 14:33:08.878202 ip-10-0-128-127 kubenswrapper[2576]: E0714 14:33:08.878181    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99e9ed50d90c24e38f3fb1a0ec9566c3b01857cafdfe95aa71db41829879f101\": container with ID starting with 99e9ed50d90c24e38f3fb1a0ec9566c3b01857cafdfe95aa71db41829879f101 not found: ID does not exist" containerID="99e9ed50d90c24e38f3fb1a0ec9566c3b01857cafdfe95aa71db41829879f101"
Jul 14 14:33:08.878249 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:08.878212    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99e9ed50d90c24e38f3fb1a0ec9566c3b01857cafdfe95aa71db41829879f101"} err="failed to get container status \"99e9ed50d90c24e38f3fb1a0ec9566c3b01857cafdfe95aa71db41829879f101\": rpc error: code = NotFound desc = could not find container \"99e9ed50d90c24e38f3fb1a0ec9566c3b01857cafdfe95aa71db41829879f101\": container with ID starting with 99e9ed50d90c24e38f3fb1a0ec9566c3b01857cafdfe95aa71db41829879f101 not found: ID does not exist"
Jul 14 14:33:08.889572 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:08.889546    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9343ac-56979b549-2mt7s"]
Jul 14 14:33:08.893459 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:08.893433    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9343ac-56979b549-2mt7s"]
Jul 14 14:33:10.689097 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:10.689064    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ebbf673-662b-4489-ad47-e165da8ef178" path="/var/lib/kubelet/pods/6ebbf673-662b-4489-ad47-e165da8ef178/volumes"
Jul 14 14:33:11.803054 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:11.803014    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:33:12.966422 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:12.966385    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:33:19.051180 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:19.051134    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:33:19.114768 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:19.114696    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:33:23.717514 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:23.717478    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:33:23.971547 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:23.971440    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:33:24.392873 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:24.392840    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:33:24.715331 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:24.715237    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:33:24.829989 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:24.829957    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:33:27.630530 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:27.630494    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:33:27.718236 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:27.718203    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8442f2-8448d45f8-xx4s8"]
Jul 14 14:33:27.718523 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:27.718512    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ebbf673-662b-4489-ad47-e165da8ef178" containerName="maas-api"
Jul 14 14:33:27.718568 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:27.718525    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ebbf673-662b-4489-ad47-e165da8ef178" containerName="maas-api"
Jul 14 14:33:27.718568 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:27.718539    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fb93465-21a9-482b-83b2-90f13ca12f31" containerName="maas-api"
Jul 14 14:33:27.718568 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:27.718545    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fb93465-21a9-482b-83b2-90f13ca12f31" containerName="maas-api"
Jul 14 14:33:27.718709 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:27.718627    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ebbf673-662b-4489-ad47-e165da8ef178" containerName="maas-api"
Jul 14 14:33:27.718709 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:27.718638    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fb93465-21a9-482b-83b2-90f13ca12f31" containerName="maas-api"
Jul 14 14:33:27.721631 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:27.721572    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:33:27.721796 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:27.721671    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8442f2-8448d45f8-xx4s8"
Jul 14 14:33:27.724075 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:27.724052    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 14 14:33:27.724214 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:27.724081    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-8442f2-serving-cert\""
Jul 14 14:33:27.724214 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:27.724194    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-znm9j\""
Jul 14 14:33:27.725035 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:27.725017    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 14:33:27.731733 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:27.731701    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8442f2-8448d45f8-xx4s8"]
Jul 14 14:33:27.815549 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:27.815508    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:33:27.885936 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:27.885846    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wrl6\" (UniqueName: \"kubernetes.io/projected/eb9b8831-8748-47c2-b77c-67a9e8e960d3-kube-api-access-7wrl6\") pod \"maas-api-e2e-shared-a-8442f2-8448d45f8-xx4s8\" (UID: \"eb9b8831-8748-47c2-b77c-67a9e8e960d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8442f2-8448d45f8-xx4s8"
Jul 14 14:33:27.885936 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:27.885887    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb9b8831-8748-47c2-b77c-67a9e8e960d3-openshift-service-ca\") pod \"maas-api-e2e-shared-a-8442f2-8448d45f8-xx4s8\" (UID: \"eb9b8831-8748-47c2-b77c-67a9e8e960d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8442f2-8448d45f8-xx4s8"
Jul 14 14:33:27.886123 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:27.885944    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb9b8831-8748-47c2-b77c-67a9e8e960d3-maas-api-tls\") pod \"maas-api-e2e-shared-a-8442f2-8448d45f8-xx4s8\" (UID: \"eb9b8831-8748-47c2-b77c-67a9e8e960d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8442f2-8448d45f8-xx4s8"
Jul 14 14:33:27.987373 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:27.987334    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wrl6\" (UniqueName: \"kubernetes.io/projected/eb9b8831-8748-47c2-b77c-67a9e8e960d3-kube-api-access-7wrl6\") pod \"maas-api-e2e-shared-a-8442f2-8448d45f8-xx4s8\" (UID: \"eb9b8831-8748-47c2-b77c-67a9e8e960d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8442f2-8448d45f8-xx4s8"
Jul 14 14:33:27.987373 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:27.987386    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb9b8831-8748-47c2-b77c-67a9e8e960d3-openshift-service-ca\") pod \"maas-api-e2e-shared-a-8442f2-8448d45f8-xx4s8\" (UID: \"eb9b8831-8748-47c2-b77c-67a9e8e960d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8442f2-8448d45f8-xx4s8"
Jul 14 14:33:27.987711 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:27.987437    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb9b8831-8748-47c2-b77c-67a9e8e960d3-maas-api-tls\") pod \"maas-api-e2e-shared-a-8442f2-8448d45f8-xx4s8\" (UID: \"eb9b8831-8748-47c2-b77c-67a9e8e960d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8442f2-8448d45f8-xx4s8"
Jul 14 14:33:27.987711 ip-10-0-128-127 kubenswrapper[2576]: E0714 14:33:27.987620    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-8442f2-serving-cert: secret "maas-api-e2e-shared-a-8442f2-serving-cert" not found
Jul 14 14:33:27.987711 ip-10-0-128-127 kubenswrapper[2576]: E0714 14:33:27.987694    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb9b8831-8748-47c2-b77c-67a9e8e960d3-maas-api-tls podName:eb9b8831-8748-47c2-b77c-67a9e8e960d3 nodeName:}" failed. No retries permitted until 2026-07-14 14:33:28.487672259 +0000 UTC m=+2356.338109743 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/eb9b8831-8748-47c2-b77c-67a9e8e960d3-maas-api-tls") pod "maas-api-e2e-shared-a-8442f2-8448d45f8-xx4s8" (UID: "eb9b8831-8748-47c2-b77c-67a9e8e960d3") : secret "maas-api-e2e-shared-a-8442f2-serving-cert" not found
Jul 14 14:33:27.988088 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:27.988065    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb9b8831-8748-47c2-b77c-67a9e8e960d3-openshift-service-ca\") pod \"maas-api-e2e-shared-a-8442f2-8448d45f8-xx4s8\" (UID: \"eb9b8831-8748-47c2-b77c-67a9e8e960d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8442f2-8448d45f8-xx4s8"
Jul 14 14:33:27.996671 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:27.996645    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wrl6\" (UniqueName: \"kubernetes.io/projected/eb9b8831-8748-47c2-b77c-67a9e8e960d3-kube-api-access-7wrl6\") pod \"maas-api-e2e-shared-a-8442f2-8448d45f8-xx4s8\" (UID: \"eb9b8831-8748-47c2-b77c-67a9e8e960d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8442f2-8448d45f8-xx4s8"
Jul 14 14:33:28.065111 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:28.065066    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:33:28.439071 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:28.439028    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:33:28.492010 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:28.491959    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb9b8831-8748-47c2-b77c-67a9e8e960d3-maas-api-tls\") pod \"maas-api-e2e-shared-a-8442f2-8448d45f8-xx4s8\" (UID: \"eb9b8831-8748-47c2-b77c-67a9e8e960d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8442f2-8448d45f8-xx4s8"
Jul 14 14:33:28.494533 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:28.494503    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb9b8831-8748-47c2-b77c-67a9e8e960d3-maas-api-tls\") pod \"maas-api-e2e-shared-a-8442f2-8448d45f8-xx4s8\" (UID: \"eb9b8831-8748-47c2-b77c-67a9e8e960d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8442f2-8448d45f8-xx4s8"
Jul 14 14:33:28.539300 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:28.539262    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:33:28.633710 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:28.633673    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8442f2-8448d45f8-xx4s8"
Jul 14 14:33:28.779974 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:28.779941    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8442f2-8448d45f8-xx4s8"]
Jul 14 14:33:28.783276 ip-10-0-128-127 kubenswrapper[2576]: W0714 14:33:28.783245    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb9b8831_8748_47c2_b77c_67a9e8e960d3.slice/crio-8b137dcc6ad597e2caac90d183bf03f6790763d0508197099c68ddba8cdb1c96 WatchSource:0}: Error finding container 8b137dcc6ad597e2caac90d183bf03f6790763d0508197099c68ddba8cdb1c96: Status 404 returned error can't find the container with id 8b137dcc6ad597e2caac90d183bf03f6790763d0508197099c68ddba8cdb1c96
Jul 14 14:33:28.960839 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:28.960803    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8442f2-8448d45f8-xx4s8" event={"ID":"eb9b8831-8748-47c2-b77c-67a9e8e960d3","Type":"ContainerStarted","Data":"8b137dcc6ad597e2caac90d183bf03f6790763d0508197099c68ddba8cdb1c96"}
Jul 14 14:33:29.312967 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:29.312934    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:33:29.408849 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:29.408795    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:33:29.966775 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:29.966735    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8442f2-8448d45f8-xx4s8" event={"ID":"eb9b8831-8748-47c2-b77c-67a9e8e960d3","Type":"ContainerStarted","Data":"404d28779907b8867a4237e570ca5d9fe1bb7faa9a13ec60f5f94895696a1b63"}
Jul 14 14:33:29.967201 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:29.966850    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8442f2-8448d45f8-xx4s8"
Jul 14 14:33:29.986778 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:29.986704    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8442f2-8448d45f8-xx4s8" podStartSLOduration=2.58842257 podStartE2EDuration="2.986682196s" totalImagesPullingTime="398.259626ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:33:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:33:29.986492928 +0000 UTC m=+2357.836930435" watchObservedRunningTime="2026-07-14 14:33:29.986682196 +0000 UTC m=+2357.837119697"
Jul 14 14:33:30.064893 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:30.064851    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:33:30.162778 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:30.162738    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:33:32.279351 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:32.279308    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:33:32.688823 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:32.688721    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:33:32.914643 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:32.914587    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:33:33.038078 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:33.038037    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:33:33.124874 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:33.124842    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:33:35.731745 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:35.731704    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:33:35.834079 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:35.834043    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:33:35.930941 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:35.930903    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f8f81e-5f6f7bd897-j7v5c"]
Jul 14 14:33:35.934630 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:35.934567    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f8f81e-5f6f7bd897-j7v5c"
Jul 14 14:33:35.937106 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:35.937078    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-f8f81e-serving-cert\""
Jul 14 14:33:35.946252 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:35.946225    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f8f81e-5f6f7bd897-j7v5c"]
Jul 14 14:33:35.966248 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:35.966207    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:33:35.978210 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:35.978180    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8442f2-8448d45f8-xx4s8"
Jul 14 14:33:36.067663 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:36.067627    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:33:36.069668 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:36.069643    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53f9753d-c9a0-4e0a-9e5d-e5d268094b1c-maas-api-tls\") pod \"maas-api-e2e-shared-b-f8f81e-5f6f7bd897-j7v5c\" (UID: \"53f9753d-c9a0-4e0a-9e5d-e5d268094b1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f8f81e-5f6f7bd897-j7v5c"
Jul 14 14:33:36.069805 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:36.069677    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53f9753d-c9a0-4e0a-9e5d-e5d268094b1c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-f8f81e-5f6f7bd897-j7v5c\" (UID: \"53f9753d-c9a0-4e0a-9e5d-e5d268094b1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f8f81e-5f6f7bd897-j7v5c"
Jul 14 14:33:36.069805 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:36.069755    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29qsd\" (UniqueName: \"kubernetes.io/projected/53f9753d-c9a0-4e0a-9e5d-e5d268094b1c-kube-api-access-29qsd\") pod \"maas-api-e2e-shared-b-f8f81e-5f6f7bd897-j7v5c\" (UID: \"53f9753d-c9a0-4e0a-9e5d-e5d268094b1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f8f81e-5f6f7bd897-j7v5c"
Jul 14 14:33:36.170809 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:36.170763    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53f9753d-c9a0-4e0a-9e5d-e5d268094b1c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-f8f81e-5f6f7bd897-j7v5c\" (UID: \"53f9753d-c9a0-4e0a-9e5d-e5d268094b1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f8f81e-5f6f7bd897-j7v5c"
Jul 14 14:33:36.170984 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:36.170912    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29qsd\" (UniqueName: \"kubernetes.io/projected/53f9753d-c9a0-4e0a-9e5d-e5d268094b1c-kube-api-access-29qsd\") pod \"maas-api-e2e-shared-b-f8f81e-5f6f7bd897-j7v5c\" (UID: \"53f9753d-c9a0-4e0a-9e5d-e5d268094b1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f8f81e-5f6f7bd897-j7v5c"
Jul 14 14:33:36.171050 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:36.171018    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53f9753d-c9a0-4e0a-9e5d-e5d268094b1c-maas-api-tls\") pod \"maas-api-e2e-shared-b-f8f81e-5f6f7bd897-j7v5c\" (UID: \"53f9753d-c9a0-4e0a-9e5d-e5d268094b1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f8f81e-5f6f7bd897-j7v5c"
Jul 14 14:33:36.171517 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:36.171482    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53f9753d-c9a0-4e0a-9e5d-e5d268094b1c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-f8f81e-5f6f7bd897-j7v5c\" (UID: \"53f9753d-c9a0-4e0a-9e5d-e5d268094b1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f8f81e-5f6f7bd897-j7v5c"
Jul 14 14:33:36.173490 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:36.173471    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53f9753d-c9a0-4e0a-9e5d-e5d268094b1c-maas-api-tls\") pod \"maas-api-e2e-shared-b-f8f81e-5f6f7bd897-j7v5c\" (UID: \"53f9753d-c9a0-4e0a-9e5d-e5d268094b1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f8f81e-5f6f7bd897-j7v5c"
Jul 14 14:33:36.178872 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:36.178852    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29qsd\" (UniqueName: \"kubernetes.io/projected/53f9753d-c9a0-4e0a-9e5d-e5d268094b1c-kube-api-access-29qsd\") pod \"maas-api-e2e-shared-b-f8f81e-5f6f7bd897-j7v5c\" (UID: \"53f9753d-c9a0-4e0a-9e5d-e5d268094b1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f8f81e-5f6f7bd897-j7v5c"
Jul 14 14:33:36.246825 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:36.246782    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f8f81e-5f6f7bd897-j7v5c"
Jul 14 14:33:36.390284 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:36.390244    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f8f81e-5f6f7bd897-j7v5c"]
Jul 14 14:33:36.393703 ip-10-0-128-127 kubenswrapper[2576]: W0714 14:33:36.393670    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53f9753d_c9a0_4e0a_9e5d_e5d268094b1c.slice/crio-2fc9865395f438dd37fe8ede6a91f4b0ff55c4931a62c1014be4e2d199969ffc WatchSource:0}: Error finding container 2fc9865395f438dd37fe8ede6a91f4b0ff55c4931a62c1014be4e2d199969ffc: Status 404 returned error can't find the container with id 2fc9865395f438dd37fe8ede6a91f4b0ff55c4931a62c1014be4e2d199969ffc
Jul 14 14:33:36.511145 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:36.511106    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:33:36.610985 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:36.610889    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:33:36.999322 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:36.999287    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f8f81e-5f6f7bd897-j7v5c" event={"ID":"53f9753d-c9a0-4e0a-9e5d-e5d268094b1c","Type":"ContainerStarted","Data":"f4a2735e5d5d60c5a3843cde728db6267c5005963bb5f9e42e6859c8ad0f6d2e"}
Jul 14 14:33:36.999724 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:36.999338    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f8f81e-5f6f7bd897-j7v5c" event={"ID":"53f9753d-c9a0-4e0a-9e5d-e5d268094b1c","Type":"ContainerStarted","Data":"2fc9865395f438dd37fe8ede6a91f4b0ff55c4931a62c1014be4e2d199969ffc"}
Jul 14 14:33:36.999724 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:36.999385    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f8f81e-5f6f7bd897-j7v5c"
Jul 14 14:33:37.017522 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:37.017468    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f8f81e-5f6f7bd897-j7v5c" podStartSLOduration=1.566973726 podStartE2EDuration="2.017449091s" totalImagesPullingTime="450.475365ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:33:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:33:37.016637729 +0000 UTC m=+2364.867075233" watchObservedRunningTime="2026-07-14 14:33:37.017449091 +0000 UTC m=+2364.867886593"
Jul 14 14:33:40.296567 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:40.296519    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:33:43.009521 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:43.009493    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f8f81e-5f6f7bd897-j7v5c"
Jul 14 14:33:45.512367 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:45.512329    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:33:49.198417 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:49.198369    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:33:49.297100 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:49.297051    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:33:50.689414 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:50.689374    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-8442f2/auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd"]
Jul 14 14:33:50.692805 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:50.692782    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8442f2/auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd"
Jul 14 14:33:50.695254 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:50.695232    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-8442f2\"/\"openshift-service-ca.crt\""
Jul 14 14:33:50.696351 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:50.696330    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-8442f2\"/\"kube-root-ca.crt\""
Jul 14 14:33:50.696470 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:50.696392    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-8442f2\"/\"default-dockercfg-n64l6\""
Jul 14 14:33:50.696546 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:50.696470    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-8442f2\"/\"auth-test-model-8442f2-kserve-self-signed-certs\""
Jul 14 14:33:50.701740 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:50.701717    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8442f2/auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd"]
Jul 14 14:33:50.720137 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:50.720086    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:33:50.805008 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:50.804952    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b8aba73-34f2-4889-890b-46ded76fabba-dshm\") pod \"auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd\" (UID: \"2b8aba73-34f2-4889-890b-46ded76fabba\") " pod="ai-tenant-e2e-shared-a-8442f2/auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd"
Jul 14 14:33:50.805222 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:50.805021    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rhxz\" (UniqueName: \"kubernetes.io/projected/2b8aba73-34f2-4889-890b-46ded76fabba-kube-api-access-9rhxz\") pod \"auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd\" (UID: \"2b8aba73-34f2-4889-890b-46ded76fabba\") " pod="ai-tenant-e2e-shared-a-8442f2/auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd"
Jul 14 14:33:50.805222 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:50.805071    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b8aba73-34f2-4889-890b-46ded76fabba-home\") pod \"auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd\" (UID: \"2b8aba73-34f2-4889-890b-46ded76fabba\") " pod="ai-tenant-e2e-shared-a-8442f2/auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd"
Jul 14 14:33:50.805222 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:50.805113    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b8aba73-34f2-4889-890b-46ded76fabba-model-cache\") pod \"auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd\" (UID: \"2b8aba73-34f2-4889-890b-46ded76fabba\") " pod="ai-tenant-e2e-shared-a-8442f2/auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd"
Jul 14 14:33:50.805222 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:50.805176    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b8aba73-34f2-4889-890b-46ded76fabba-tls-certs\") pod \"auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd\" (UID: \"2b8aba73-34f2-4889-890b-46ded76fabba\") " pod="ai-tenant-e2e-shared-a-8442f2/auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd"
Jul 14 14:33:50.805222 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:50.805206    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b8aba73-34f2-4889-890b-46ded76fabba-tmp-dir\") pod \"auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd\" (UID: \"2b8aba73-34f2-4889-890b-46ded76fabba\") " pod="ai-tenant-e2e-shared-a-8442f2/auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd"
Jul 14 14:33:50.906124 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:50.906078    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b8aba73-34f2-4889-890b-46ded76fabba-tls-certs\") pod \"auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd\" (UID: \"2b8aba73-34f2-4889-890b-46ded76fabba\") " pod="ai-tenant-e2e-shared-a-8442f2/auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd"
Jul 14 14:33:50.906124 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:50.906123    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b8aba73-34f2-4889-890b-46ded76fabba-tmp-dir\") pod \"auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd\" (UID: \"2b8aba73-34f2-4889-890b-46ded76fabba\") " pod="ai-tenant-e2e-shared-a-8442f2/auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd"
Jul 14 14:33:50.906399 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:50.906162    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b8aba73-34f2-4889-890b-46ded76fabba-dshm\") pod \"auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd\" (UID: \"2b8aba73-34f2-4889-890b-46ded76fabba\") " pod="ai-tenant-e2e-shared-a-8442f2/auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd"
Jul 14 14:33:50.906399 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:50.906189    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rhxz\" (UniqueName: \"kubernetes.io/projected/2b8aba73-34f2-4889-890b-46ded76fabba-kube-api-access-9rhxz\") pod \"auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd\" (UID: \"2b8aba73-34f2-4889-890b-46ded76fabba\") " pod="ai-tenant-e2e-shared-a-8442f2/auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd"
Jul 14 14:33:50.906399 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:50.906207    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b8aba73-34f2-4889-890b-46ded76fabba-home\") pod \"auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd\" (UID: \"2b8aba73-34f2-4889-890b-46ded76fabba\") " pod="ai-tenant-e2e-shared-a-8442f2/auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd"
Jul 14 14:33:50.906399 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:50.906235    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b8aba73-34f2-4889-890b-46ded76fabba-model-cache\") pod \"auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd\" (UID: \"2b8aba73-34f2-4889-890b-46ded76fabba\") " pod="ai-tenant-e2e-shared-a-8442f2/auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd"
Jul 14 14:33:50.906732 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:50.906711    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b8aba73-34f2-4889-890b-46ded76fabba-tmp-dir\") pod \"auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd\" (UID: \"2b8aba73-34f2-4889-890b-46ded76fabba\") " pod="ai-tenant-e2e-shared-a-8442f2/auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd"
Jul 14 14:33:50.906797 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:50.906743    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b8aba73-34f2-4889-890b-46ded76fabba-model-cache\") pod \"auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd\" (UID: \"2b8aba73-34f2-4889-890b-46ded76fabba\") " pod="ai-tenant-e2e-shared-a-8442f2/auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd"
Jul 14 14:33:50.906797 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:50.906769    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b8aba73-34f2-4889-890b-46ded76fabba-home\") pod \"auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd\" (UID: \"2b8aba73-34f2-4889-890b-46ded76fabba\") " pod="ai-tenant-e2e-shared-a-8442f2/auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd"
Jul 14 14:33:50.908681 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:50.908663    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b8aba73-34f2-4889-890b-46ded76fabba-dshm\") pod \"auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd\" (UID: \"2b8aba73-34f2-4889-890b-46ded76fabba\") " pod="ai-tenant-e2e-shared-a-8442f2/auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd"
Jul 14 14:33:50.908838 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:50.908823    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b8aba73-34f2-4889-890b-46ded76fabba-tls-certs\") pod \"auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd\" (UID: \"2b8aba73-34f2-4889-890b-46ded76fabba\") " pod="ai-tenant-e2e-shared-a-8442f2/auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd"
Jul 14 14:33:50.914808 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:50.914781    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rhxz\" (UniqueName: \"kubernetes.io/projected/2b8aba73-34f2-4889-890b-46ded76fabba-kube-api-access-9rhxz\") pod \"auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd\" (UID: \"2b8aba73-34f2-4889-890b-46ded76fabba\") " pod="ai-tenant-e2e-shared-a-8442f2/auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd"
Jul 14 14:33:51.003006 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:51.002891    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8442f2/auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd"
Jul 14 14:33:51.140887 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:51.140827    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8442f2/auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd"]
Jul 14 14:33:51.143716 ip-10-0-128-127 kubenswrapper[2576]: W0714 14:33:51.143682    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b8aba73_34f2_4889_890b_46ded76fabba.slice/crio-80feb6fa94595cdffb5da188859e65f0bc0657f1cf5c8b33cd0f98dfff22d8d2 WatchSource:0}: Error finding container 80feb6fa94595cdffb5da188859e65f0bc0657f1cf5c8b33cd0f98dfff22d8d2: Status 404 returned error can't find the container with id 80feb6fa94595cdffb5da188859e65f0bc0657f1cf5c8b33cd0f98dfff22d8d2
Jul 14 14:33:52.077925 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:52.077826    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8442f2/auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd" event={"ID":"2b8aba73-34f2-4889-890b-46ded76fabba","Type":"ContainerStarted","Data":"4e1eda950af17e36a60b15ad07dd93e219579d014b6fae0f90e81dba5705d1fc"}
Jul 14 14:33:52.077925 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:52.077874    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8442f2/auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd" event={"ID":"2b8aba73-34f2-4889-890b-46ded76fabba","Type":"ContainerStarted","Data":"80feb6fa94595cdffb5da188859e65f0bc0657f1cf5c8b33cd0f98dfff22d8d2"}
Jul 14 14:33:52.095516 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:52.095457    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-8442f2/auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd" podStartSLOduration=2.095438809 podStartE2EDuration="2.095438809s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:33:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:33:52.093909715 +0000 UTC m=+2379.944347219" watchObservedRunningTime="2026-07-14 14:33:52.095438809 +0000 UTC m=+2379.945876312"
Jul 14 14:33:52.820528 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:52.820489    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:33:57.237717 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:57.237680    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:33:57.336871 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:33:57.336838    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:34:01.003186 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:01.003122    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-8442f2/auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd"
Jul 14 14:34:01.003186 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:01.003203    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-8442f2/auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd"
Jul 14 14:34:01.015946 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:01.015910    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-8442f2/auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd"
Jul 14 14:34:01.130159 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:01.130128    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-8442f2/auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd"
Jul 14 14:34:11.883871 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:11.883832    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-f8f81e/auth-test-model-f8f81e-kserve-6d6749b667-pbbb2"]
Jul 14 14:34:11.888453 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:11.888436    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f8f81e/auth-test-model-f8f81e-kserve-6d6749b667-pbbb2"
Jul 14 14:34:11.892129 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:11.892107    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f8f81e\"/\"auth-test-model-f8f81e-kserve-self-signed-certs\""
Jul 14 14:34:11.892258 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:11.892152    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f8f81e\"/\"default-dockercfg-4fwrg\""
Jul 14 14:34:11.892258 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:11.892190    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f8f81e\"/\"kube-root-ca.crt\""
Jul 14 14:34:11.892258 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:11.892206    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f8f81e\"/\"openshift-service-ca.crt\""
Jul 14 14:34:11.896921 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:11.896899    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f8f81e/auth-test-model-f8f81e-kserve-6d6749b667-pbbb2"]
Jul 14 14:34:11.915208 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:11.915156    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:34:12.001544 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:12.001500    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7180bd41-6c53-4fc7-af1a-0b37fd547edc-model-cache\") pod \"auth-test-model-f8f81e-kserve-6d6749b667-pbbb2\" (UID: \"7180bd41-6c53-4fc7-af1a-0b37fd547edc\") " pod="ai-tenant-e2e-shared-b-f8f81e/auth-test-model-f8f81e-kserve-6d6749b667-pbbb2"
Jul 14 14:34:12.001769 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:12.001553    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7180bd41-6c53-4fc7-af1a-0b37fd547edc-dshm\") pod \"auth-test-model-f8f81e-kserve-6d6749b667-pbbb2\" (UID: \"7180bd41-6c53-4fc7-af1a-0b37fd547edc\") " pod="ai-tenant-e2e-shared-b-f8f81e/auth-test-model-f8f81e-kserve-6d6749b667-pbbb2"
Jul 14 14:34:12.001769 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:12.001691    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7180bd41-6c53-4fc7-af1a-0b37fd547edc-home\") pod \"auth-test-model-f8f81e-kserve-6d6749b667-pbbb2\" (UID: \"7180bd41-6c53-4fc7-af1a-0b37fd547edc\") " pod="ai-tenant-e2e-shared-b-f8f81e/auth-test-model-f8f81e-kserve-6d6749b667-pbbb2"
Jul 14 14:34:12.001853 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:12.001776    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7180bd41-6c53-4fc7-af1a-0b37fd547edc-tmp-dir\") pod \"auth-test-model-f8f81e-kserve-6d6749b667-pbbb2\" (UID: \"7180bd41-6c53-4fc7-af1a-0b37fd547edc\") " pod="ai-tenant-e2e-shared-b-f8f81e/auth-test-model-f8f81e-kserve-6d6749b667-pbbb2"
Jul 14 14:34:12.001853 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:12.001802    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7180bd41-6c53-4fc7-af1a-0b37fd547edc-tls-certs\") pod \"auth-test-model-f8f81e-kserve-6d6749b667-pbbb2\" (UID: \"7180bd41-6c53-4fc7-af1a-0b37fd547edc\") " pod="ai-tenant-e2e-shared-b-f8f81e/auth-test-model-f8f81e-kserve-6d6749b667-pbbb2"
Jul 14 14:34:12.001853 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:12.001839    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48bqs\" (UniqueName: \"kubernetes.io/projected/7180bd41-6c53-4fc7-af1a-0b37fd547edc-kube-api-access-48bqs\") pod \"auth-test-model-f8f81e-kserve-6d6749b667-pbbb2\" (UID: \"7180bd41-6c53-4fc7-af1a-0b37fd547edc\") " pod="ai-tenant-e2e-shared-b-f8f81e/auth-test-model-f8f81e-kserve-6d6749b667-pbbb2"
Jul 14 14:34:12.103308 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:12.103266    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48bqs\" (UniqueName: \"kubernetes.io/projected/7180bd41-6c53-4fc7-af1a-0b37fd547edc-kube-api-access-48bqs\") pod \"auth-test-model-f8f81e-kserve-6d6749b667-pbbb2\" (UID: \"7180bd41-6c53-4fc7-af1a-0b37fd547edc\") " pod="ai-tenant-e2e-shared-b-f8f81e/auth-test-model-f8f81e-kserve-6d6749b667-pbbb2"
Jul 14 14:34:12.103308 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:12.103311    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7180bd41-6c53-4fc7-af1a-0b37fd547edc-model-cache\") pod \"auth-test-model-f8f81e-kserve-6d6749b667-pbbb2\" (UID: \"7180bd41-6c53-4fc7-af1a-0b37fd547edc\") " pod="ai-tenant-e2e-shared-b-f8f81e/auth-test-model-f8f81e-kserve-6d6749b667-pbbb2"
Jul 14 14:34:12.103536 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:12.103438    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7180bd41-6c53-4fc7-af1a-0b37fd547edc-dshm\") pod \"auth-test-model-f8f81e-kserve-6d6749b667-pbbb2\" (UID: \"7180bd41-6c53-4fc7-af1a-0b37fd547edc\") " pod="ai-tenant-e2e-shared-b-f8f81e/auth-test-model-f8f81e-kserve-6d6749b667-pbbb2"
Jul 14 14:34:12.103536 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:12.103521    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7180bd41-6c53-4fc7-af1a-0b37fd547edc-home\") pod \"auth-test-model-f8f81e-kserve-6d6749b667-pbbb2\" (UID: \"7180bd41-6c53-4fc7-af1a-0b37fd547edc\") " pod="ai-tenant-e2e-shared-b-f8f81e/auth-test-model-f8f81e-kserve-6d6749b667-pbbb2"
Jul 14 14:34:12.103642 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:12.103611    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7180bd41-6c53-4fc7-af1a-0b37fd547edc-tmp-dir\") pod \"auth-test-model-f8f81e-kserve-6d6749b667-pbbb2\" (UID: \"7180bd41-6c53-4fc7-af1a-0b37fd547edc\") " pod="ai-tenant-e2e-shared-b-f8f81e/auth-test-model-f8f81e-kserve-6d6749b667-pbbb2"
Jul 14 14:34:12.103692 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:12.103644    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7180bd41-6c53-4fc7-af1a-0b37fd547edc-tls-certs\") pod \"auth-test-model-f8f81e-kserve-6d6749b667-pbbb2\" (UID: \"7180bd41-6c53-4fc7-af1a-0b37fd547edc\") " pod="ai-tenant-e2e-shared-b-f8f81e/auth-test-model-f8f81e-kserve-6d6749b667-pbbb2"
Jul 14 14:34:12.103800 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:12.103772    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7180bd41-6c53-4fc7-af1a-0b37fd547edc-model-cache\") pod \"auth-test-model-f8f81e-kserve-6d6749b667-pbbb2\" (UID: \"7180bd41-6c53-4fc7-af1a-0b37fd547edc\") " pod="ai-tenant-e2e-shared-b-f8f81e/auth-test-model-f8f81e-kserve-6d6749b667-pbbb2"
Jul 14 14:34:12.103954 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:12.103893    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7180bd41-6c53-4fc7-af1a-0b37fd547edc-home\") pod \"auth-test-model-f8f81e-kserve-6d6749b667-pbbb2\" (UID: \"7180bd41-6c53-4fc7-af1a-0b37fd547edc\") " pod="ai-tenant-e2e-shared-b-f8f81e/auth-test-model-f8f81e-kserve-6d6749b667-pbbb2"
Jul 14 14:34:12.103954 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:12.103939    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7180bd41-6c53-4fc7-af1a-0b37fd547edc-tmp-dir\") pod \"auth-test-model-f8f81e-kserve-6d6749b667-pbbb2\" (UID: \"7180bd41-6c53-4fc7-af1a-0b37fd547edc\") " pod="ai-tenant-e2e-shared-b-f8f81e/auth-test-model-f8f81e-kserve-6d6749b667-pbbb2"
Jul 14 14:34:12.105787 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:12.105761    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7180bd41-6c53-4fc7-af1a-0b37fd547edc-dshm\") pod \"auth-test-model-f8f81e-kserve-6d6749b667-pbbb2\" (UID: \"7180bd41-6c53-4fc7-af1a-0b37fd547edc\") " pod="ai-tenant-e2e-shared-b-f8f81e/auth-test-model-f8f81e-kserve-6d6749b667-pbbb2"
Jul 14 14:34:12.106154 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:12.106134    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7180bd41-6c53-4fc7-af1a-0b37fd547edc-tls-certs\") pod \"auth-test-model-f8f81e-kserve-6d6749b667-pbbb2\" (UID: \"7180bd41-6c53-4fc7-af1a-0b37fd547edc\") " pod="ai-tenant-e2e-shared-b-f8f81e/auth-test-model-f8f81e-kserve-6d6749b667-pbbb2"
Jul 14 14:34:12.112504 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:12.112476    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48bqs\" (UniqueName: \"kubernetes.io/projected/7180bd41-6c53-4fc7-af1a-0b37fd547edc-kube-api-access-48bqs\") pod \"auth-test-model-f8f81e-kserve-6d6749b667-pbbb2\" (UID: \"7180bd41-6c53-4fc7-af1a-0b37fd547edc\") " pod="ai-tenant-e2e-shared-b-f8f81e/auth-test-model-f8f81e-kserve-6d6749b667-pbbb2"
Jul 14 14:34:12.198936 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:12.198840    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f8f81e/auth-test-model-f8f81e-kserve-6d6749b667-pbbb2"
Jul 14 14:34:12.339666 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:12.339635    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f8f81e/auth-test-model-f8f81e-kserve-6d6749b667-pbbb2"]
Jul 14 14:34:12.342381 ip-10-0-128-127 kubenswrapper[2576]: W0714 14:34:12.342344    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7180bd41_6c53_4fc7_af1a_0b37fd547edc.slice/crio-9f027b040f62b96e70b255cd1fa44af00f78db7739d9989b3cf66d9d59cec22e WatchSource:0}: Error finding container 9f027b040f62b96e70b255cd1fa44af00f78db7739d9989b3cf66d9d59cec22e: Status 404 returned error can't find the container with id 9f027b040f62b96e70b255cd1fa44af00f78db7739d9989b3cf66d9d59cec22e
Jul 14 14:34:12.713681 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:12.713653    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sh6rt_5600e3e1-e775-482e-bc72-76d732552705/ovn-acl-logging/0.log"
Jul 14 14:34:12.717175 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:12.717153    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sh6rt_5600e3e1-e775-482e-bc72-76d732552705/ovn-acl-logging/0.log"
Jul 14 14:34:13.173766 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:13.173730    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f8f81e/auth-test-model-f8f81e-kserve-6d6749b667-pbbb2" event={"ID":"7180bd41-6c53-4fc7-af1a-0b37fd547edc","Type":"ContainerStarted","Data":"7d9bb1e4a50c94e3d1b3356a0d168b3acd29c6b7e88ac7401f08dd683beed846"}
Jul 14 14:34:13.174238 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:13.173777    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f8f81e/auth-test-model-f8f81e-kserve-6d6749b667-pbbb2" event={"ID":"7180bd41-6c53-4fc7-af1a-0b37fd547edc","Type":"ContainerStarted","Data":"9f027b040f62b96e70b255cd1fa44af00f78db7739d9989b3cf66d9d59cec22e"}
Jul 14 14:34:13.197540 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:13.197484    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-f8f81e/auth-test-model-f8f81e-kserve-6d6749b667-pbbb2" podStartSLOduration=2.197465993 podStartE2EDuration="2.197465993s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:34:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:34:13.195762002 +0000 UTC m=+2401.046199505" watchObservedRunningTime="2026-07-14 14:34:13.197465993 +0000 UTC m=+2401.047903848"
Jul 14 14:34:15.622911 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:15.622872    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:34:22.199909 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:22.199848    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-f8f81e/auth-test-model-f8f81e-kserve-6d6749b667-pbbb2"
Jul 14 14:34:22.199909 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:22.199919    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-f8f81e/auth-test-model-f8f81e-kserve-6d6749b667-pbbb2"
Jul 14 14:34:22.212714 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:22.212680    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-f8f81e/auth-test-model-f8f81e-kserve-6d6749b667-pbbb2"
Jul 14 14:34:22.227767 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:22.227735    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-f8f81e/auth-test-model-f8f81e-kserve-6d6749b667-pbbb2"
Jul 14 14:34:32.911227 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:32.911186    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:34:50.913014 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:34:50.912919    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:35:00.442532 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:35:00.442486    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:35:06.813494 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:35:06.813452    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:35:23.807552 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:35:23.807513    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:35:42.105733 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:35:42.105685    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:35:59.617360 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:35:59.617316    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:36:18.654618 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:18.654547    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:36:23.362658 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:23.362584    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8442f2/auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd"]
Jul 14 14:36:23.390486 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:23.390453    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:36:24.377290 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:24.377250    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f8f81e/auth-test-model-f8f81e-kserve-6d6749b667-pbbb2"]
Jul 14 14:36:26.204734 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:26.204695    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-8442f2/sub-test-model-8442f2-kserve-859fb7f755-b2ndh"]
Jul 14 14:36:26.208337 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:26.208313    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8442f2/sub-test-model-8442f2-kserve-859fb7f755-b2ndh"
Jul 14 14:36:26.211282 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:26.211258    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-8442f2\"/\"sub-test-model-8442f2-kserve-self-signed-certs\""
Jul 14 14:36:26.218653 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:26.218621    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8442f2/sub-test-model-8442f2-kserve-859fb7f755-b2ndh"]
Jul 14 14:36:26.338438 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:26.338390    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9062dce-1753-4fe0-aa71-57d7468f3ea5-model-cache\") pod \"sub-test-model-8442f2-kserve-859fb7f755-b2ndh\" (UID: \"c9062dce-1753-4fe0-aa71-57d7468f3ea5\") " pod="ai-tenant-e2e-shared-a-8442f2/sub-test-model-8442f2-kserve-859fb7f755-b2ndh"
Jul 14 14:36:26.338676 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:26.338453    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9062dce-1753-4fe0-aa71-57d7468f3ea5-tls-certs\") pod \"sub-test-model-8442f2-kserve-859fb7f755-b2ndh\" (UID: \"c9062dce-1753-4fe0-aa71-57d7468f3ea5\") " pod="ai-tenant-e2e-shared-a-8442f2/sub-test-model-8442f2-kserve-859fb7f755-b2ndh"
Jul 14 14:36:26.338676 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:26.338548    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9062dce-1753-4fe0-aa71-57d7468f3ea5-tmp-dir\") pod \"sub-test-model-8442f2-kserve-859fb7f755-b2ndh\" (UID: \"c9062dce-1753-4fe0-aa71-57d7468f3ea5\") " pod="ai-tenant-e2e-shared-a-8442f2/sub-test-model-8442f2-kserve-859fb7f755-b2ndh"
Jul 14 14:36:26.338676 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:26.338586    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9062dce-1753-4fe0-aa71-57d7468f3ea5-home\") pod \"sub-test-model-8442f2-kserve-859fb7f755-b2ndh\" (UID: \"c9062dce-1753-4fe0-aa71-57d7468f3ea5\") " pod="ai-tenant-e2e-shared-a-8442f2/sub-test-model-8442f2-kserve-859fb7f755-b2ndh"
Jul 14 14:36:26.338676 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:26.338650    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6wzp\" (UniqueName: \"kubernetes.io/projected/c9062dce-1753-4fe0-aa71-57d7468f3ea5-kube-api-access-c6wzp\") pod \"sub-test-model-8442f2-kserve-859fb7f755-b2ndh\" (UID: \"c9062dce-1753-4fe0-aa71-57d7468f3ea5\") " pod="ai-tenant-e2e-shared-a-8442f2/sub-test-model-8442f2-kserve-859fb7f755-b2ndh"
Jul 14 14:36:26.338676 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:26.338671    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9062dce-1753-4fe0-aa71-57d7468f3ea5-dshm\") pod \"sub-test-model-8442f2-kserve-859fb7f755-b2ndh\" (UID: \"c9062dce-1753-4fe0-aa71-57d7468f3ea5\") " pod="ai-tenant-e2e-shared-a-8442f2/sub-test-model-8442f2-kserve-859fb7f755-b2ndh"
Jul 14 14:36:26.439324 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:26.439287    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9062dce-1753-4fe0-aa71-57d7468f3ea5-tls-certs\") pod \"sub-test-model-8442f2-kserve-859fb7f755-b2ndh\" (UID: \"c9062dce-1753-4fe0-aa71-57d7468f3ea5\") " pod="ai-tenant-e2e-shared-a-8442f2/sub-test-model-8442f2-kserve-859fb7f755-b2ndh"
Jul 14 14:36:26.439508 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:26.439349    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9062dce-1753-4fe0-aa71-57d7468f3ea5-tmp-dir\") pod \"sub-test-model-8442f2-kserve-859fb7f755-b2ndh\" (UID: \"c9062dce-1753-4fe0-aa71-57d7468f3ea5\") " pod="ai-tenant-e2e-shared-a-8442f2/sub-test-model-8442f2-kserve-859fb7f755-b2ndh"
Jul 14 14:36:26.439508 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:26.439375    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9062dce-1753-4fe0-aa71-57d7468f3ea5-home\") pod \"sub-test-model-8442f2-kserve-859fb7f755-b2ndh\" (UID: \"c9062dce-1753-4fe0-aa71-57d7468f3ea5\") " pod="ai-tenant-e2e-shared-a-8442f2/sub-test-model-8442f2-kserve-859fb7f755-b2ndh"
Jul 14 14:36:26.439508 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:26.439485    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6wzp\" (UniqueName: \"kubernetes.io/projected/c9062dce-1753-4fe0-aa71-57d7468f3ea5-kube-api-access-c6wzp\") pod \"sub-test-model-8442f2-kserve-859fb7f755-b2ndh\" (UID: \"c9062dce-1753-4fe0-aa71-57d7468f3ea5\") " pod="ai-tenant-e2e-shared-a-8442f2/sub-test-model-8442f2-kserve-859fb7f755-b2ndh"
Jul 14 14:36:26.439715 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:26.439636    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9062dce-1753-4fe0-aa71-57d7468f3ea5-dshm\") pod \"sub-test-model-8442f2-kserve-859fb7f755-b2ndh\" (UID: \"c9062dce-1753-4fe0-aa71-57d7468f3ea5\") " pod="ai-tenant-e2e-shared-a-8442f2/sub-test-model-8442f2-kserve-859fb7f755-b2ndh"
Jul 14 14:36:26.439773 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:26.439733    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9062dce-1753-4fe0-aa71-57d7468f3ea5-model-cache\") pod \"sub-test-model-8442f2-kserve-859fb7f755-b2ndh\" (UID: \"c9062dce-1753-4fe0-aa71-57d7468f3ea5\") " pod="ai-tenant-e2e-shared-a-8442f2/sub-test-model-8442f2-kserve-859fb7f755-b2ndh"
Jul 14 14:36:26.439773 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:26.439745    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9062dce-1753-4fe0-aa71-57d7468f3ea5-home\") pod \"sub-test-model-8442f2-kserve-859fb7f755-b2ndh\" (UID: \"c9062dce-1753-4fe0-aa71-57d7468f3ea5\") " pod="ai-tenant-e2e-shared-a-8442f2/sub-test-model-8442f2-kserve-859fb7f755-b2ndh"
Jul 14 14:36:26.439864 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:26.439812    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9062dce-1753-4fe0-aa71-57d7468f3ea5-tmp-dir\") pod \"sub-test-model-8442f2-kserve-859fb7f755-b2ndh\" (UID: \"c9062dce-1753-4fe0-aa71-57d7468f3ea5\") " pod="ai-tenant-e2e-shared-a-8442f2/sub-test-model-8442f2-kserve-859fb7f755-b2ndh"
Jul 14 14:36:26.440008 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:26.439989    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9062dce-1753-4fe0-aa71-57d7468f3ea5-model-cache\") pod \"sub-test-model-8442f2-kserve-859fb7f755-b2ndh\" (UID: \"c9062dce-1753-4fe0-aa71-57d7468f3ea5\") " pod="ai-tenant-e2e-shared-a-8442f2/sub-test-model-8442f2-kserve-859fb7f755-b2ndh"
Jul 14 14:36:26.441813 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:26.441791    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9062dce-1753-4fe0-aa71-57d7468f3ea5-dshm\") pod \"sub-test-model-8442f2-kserve-859fb7f755-b2ndh\" (UID: \"c9062dce-1753-4fe0-aa71-57d7468f3ea5\") " pod="ai-tenant-e2e-shared-a-8442f2/sub-test-model-8442f2-kserve-859fb7f755-b2ndh"
Jul 14 14:36:26.441930 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:26.441886    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9062dce-1753-4fe0-aa71-57d7468f3ea5-tls-certs\") pod \"sub-test-model-8442f2-kserve-859fb7f755-b2ndh\" (UID: \"c9062dce-1753-4fe0-aa71-57d7468f3ea5\") " pod="ai-tenant-e2e-shared-a-8442f2/sub-test-model-8442f2-kserve-859fb7f755-b2ndh"
Jul 14 14:36:26.449270 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:26.449235    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6wzp\" (UniqueName: \"kubernetes.io/projected/c9062dce-1753-4fe0-aa71-57d7468f3ea5-kube-api-access-c6wzp\") pod \"sub-test-model-8442f2-kserve-859fb7f755-b2ndh\" (UID: \"c9062dce-1753-4fe0-aa71-57d7468f3ea5\") " pod="ai-tenant-e2e-shared-a-8442f2/sub-test-model-8442f2-kserve-859fb7f755-b2ndh"
Jul 14 14:36:26.492520 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:26.492428    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:36:26.518360 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:26.518328    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8442f2/sub-test-model-8442f2-kserve-859fb7f755-b2ndh"
Jul 14 14:36:26.659873 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:26.659836    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8442f2/sub-test-model-8442f2-kserve-859fb7f755-b2ndh"]
Jul 14 14:36:26.663714 ip-10-0-128-127 kubenswrapper[2576]: W0714 14:36:26.663680    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc9062dce_1753_4fe0_aa71_57d7468f3ea5.slice/crio-42df2b962de846346ee96f41957b69b0e716c37b587e3dfbc3629549e5acad81 WatchSource:0}: Error finding container 42df2b962de846346ee96f41957b69b0e716c37b587e3dfbc3629549e5acad81: Status 404 returned error can't find the container with id 42df2b962de846346ee96f41957b69b0e716c37b587e3dfbc3629549e5acad81
Jul 14 14:36:26.769051 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:26.769016    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8442f2/sub-test-model-8442f2-kserve-859fb7f755-b2ndh" event={"ID":"c9062dce-1753-4fe0-aa71-57d7468f3ea5","Type":"ContainerStarted","Data":"7e95d2d7ad73d3bdb62073d30f234d7b93f5d02da381be501f32f05835162c03"}
Jul 14 14:36:26.769240 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:26.769059    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8442f2/sub-test-model-8442f2-kserve-859fb7f755-b2ndh" event={"ID":"c9062dce-1753-4fe0-aa71-57d7468f3ea5","Type":"ContainerStarted","Data":"42df2b962de846346ee96f41957b69b0e716c37b587e3dfbc3629549e5acad81"}
Jul 14 14:36:26.791179 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:26.791119    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-8442f2/sub-test-model-8442f2-kserve-859fb7f755-b2ndh" podStartSLOduration=0.791097969 podStartE2EDuration="791.097969ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:36:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:36:26.78870454 +0000 UTC m=+2534.639142045" watchObservedRunningTime="2026-07-14 14:36:26.791097969 +0000 UTC m=+2534.641535473"
Jul 14 14:36:36.519292 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:36.519250    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-8442f2/sub-test-model-8442f2-kserve-859fb7f755-b2ndh"
Jul 14 14:36:36.519750 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:36.519333    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-8442f2/sub-test-model-8442f2-kserve-859fb7f755-b2ndh"
Jul 14 14:36:36.531977 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:36.531948    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-8442f2/sub-test-model-8442f2-kserve-859fb7f755-b2ndh"
Jul 14 14:36:36.825652 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:36.825552    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-8442f2/sub-test-model-8442f2-kserve-859fb7f755-b2ndh"
Jul 14 14:36:36.958395 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:36.958357    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:36:38.382922 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:38.382877    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-8442f2/auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd" podUID="2b8aba73-34f2-4889-890b-46ded76fabba" containerName="main" containerID="cri-o://4e1eda950af17e36a60b15ad07dd93e219579d014b6fae0f90e81dba5705d1fc" gracePeriod=45
Jul 14 14:36:38.652707 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:38.652680    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8442f2/auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd"
Jul 14 14:36:38.760720 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:38.760673    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b8aba73-34f2-4889-890b-46ded76fabba-dshm\") pod \"2b8aba73-34f2-4889-890b-46ded76fabba\" (UID: \"2b8aba73-34f2-4889-890b-46ded76fabba\") "
Jul 14 14:36:38.760932 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:38.760738    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9rhxz\" (UniqueName: \"kubernetes.io/projected/2b8aba73-34f2-4889-890b-46ded76fabba-kube-api-access-9rhxz\") pod \"2b8aba73-34f2-4889-890b-46ded76fabba\" (UID: \"2b8aba73-34f2-4889-890b-46ded76fabba\") "
Jul 14 14:36:38.760932 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:38.760754    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b8aba73-34f2-4889-890b-46ded76fabba-home\") pod \"2b8aba73-34f2-4889-890b-46ded76fabba\" (UID: \"2b8aba73-34f2-4889-890b-46ded76fabba\") "
Jul 14 14:36:38.760932 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:38.760805    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b8aba73-34f2-4889-890b-46ded76fabba-tls-certs\") pod \"2b8aba73-34f2-4889-890b-46ded76fabba\" (UID: \"2b8aba73-34f2-4889-890b-46ded76fabba\") "
Jul 14 14:36:38.760932 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:38.760840    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b8aba73-34f2-4889-890b-46ded76fabba-tmp-dir\") pod \"2b8aba73-34f2-4889-890b-46ded76fabba\" (UID: \"2b8aba73-34f2-4889-890b-46ded76fabba\") "
Jul 14 14:36:38.760932 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:38.760877    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b8aba73-34f2-4889-890b-46ded76fabba-model-cache\") pod \"2b8aba73-34f2-4889-890b-46ded76fabba\" (UID: \"2b8aba73-34f2-4889-890b-46ded76fabba\") "
Jul 14 14:36:38.761261 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:38.761031    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b8aba73-34f2-4889-890b-46ded76fabba-home" (OuterVolumeSpecName: "home") pod "2b8aba73-34f2-4889-890b-46ded76fabba" (UID: "2b8aba73-34f2-4889-890b-46ded76fabba"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 14:36:38.761261 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:38.761151    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b8aba73-34f2-4889-890b-46ded76fabba-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2b8aba73-34f2-4889-890b-46ded76fabba" (UID: "2b8aba73-34f2-4889-890b-46ded76fabba"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 14:36:38.761261 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:38.761239    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b8aba73-34f2-4889-890b-46ded76fabba-model-cache" (OuterVolumeSpecName: "model-cache") pod "2b8aba73-34f2-4889-890b-46ded76fabba" (UID: "2b8aba73-34f2-4889-890b-46ded76fabba"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 14:36:38.763043 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:38.763018    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b8aba73-34f2-4889-890b-46ded76fabba-dshm" (OuterVolumeSpecName: "dshm") pod "2b8aba73-34f2-4889-890b-46ded76fabba" (UID: "2b8aba73-34f2-4889-890b-46ded76fabba"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 14:36:38.763186 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:38.763068    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b8aba73-34f2-4889-890b-46ded76fabba-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2b8aba73-34f2-4889-890b-46ded76fabba" (UID: "2b8aba73-34f2-4889-890b-46ded76fabba"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 14:36:38.763186 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:38.763148    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b8aba73-34f2-4889-890b-46ded76fabba-kube-api-access-9rhxz" (OuterVolumeSpecName: "kube-api-access-9rhxz") pod "2b8aba73-34f2-4889-890b-46ded76fabba" (UID: "2b8aba73-34f2-4889-890b-46ded76fabba"). InnerVolumeSpecName "kube-api-access-9rhxz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 14:36:38.826476 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:38.826439    2576 generic.go:356] "Generic (PLEG): container finished" podID="2b8aba73-34f2-4889-890b-46ded76fabba" containerID="4e1eda950af17e36a60b15ad07dd93e219579d014b6fae0f90e81dba5705d1fc" exitCode=0
Jul 14 14:36:38.826670 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:38.826502    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8442f2/auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd"
Jul 14 14:36:38.826670 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:38.826522    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8442f2/auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd" event={"ID":"2b8aba73-34f2-4889-890b-46ded76fabba","Type":"ContainerDied","Data":"4e1eda950af17e36a60b15ad07dd93e219579d014b6fae0f90e81dba5705d1fc"}
Jul 14 14:36:38.826670 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:38.826564    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8442f2/auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd" event={"ID":"2b8aba73-34f2-4889-890b-46ded76fabba","Type":"ContainerDied","Data":"80feb6fa94595cdffb5da188859e65f0bc0657f1cf5c8b33cd0f98dfff22d8d2"}
Jul 14 14:36:38.826670 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:38.826601    2576 scope.go:117] "RemoveContainer" containerID="4e1eda950af17e36a60b15ad07dd93e219579d014b6fae0f90e81dba5705d1fc"
Jul 14 14:36:38.836136 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:38.836114    2576 scope.go:117] "RemoveContainer" containerID="4e1eda950af17e36a60b15ad07dd93e219579d014b6fae0f90e81dba5705d1fc"
Jul 14 14:36:38.836468 ip-10-0-128-127 kubenswrapper[2576]: E0714 14:36:38.836449    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e1eda950af17e36a60b15ad07dd93e219579d014b6fae0f90e81dba5705d1fc\": container with ID starting with 4e1eda950af17e36a60b15ad07dd93e219579d014b6fae0f90e81dba5705d1fc not found: ID does not exist" containerID="4e1eda950af17e36a60b15ad07dd93e219579d014b6fae0f90e81dba5705d1fc"
Jul 14 14:36:38.836516 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:38.836477    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e1eda950af17e36a60b15ad07dd93e219579d014b6fae0f90e81dba5705d1fc"} err="failed to get container status \"4e1eda950af17e36a60b15ad07dd93e219579d014b6fae0f90e81dba5705d1fc\": rpc error: code = NotFound desc = could not find container \"4e1eda950af17e36a60b15ad07dd93e219579d014b6fae0f90e81dba5705d1fc\": container with ID starting with 4e1eda950af17e36a60b15ad07dd93e219579d014b6fae0f90e81dba5705d1fc not found: ID does not exist"
Jul 14 14:36:38.862175 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:38.862127    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b8aba73-34f2-4889-890b-46ded76fabba-model-cache\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:36:38.862175 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:38.862161    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b8aba73-34f2-4889-890b-46ded76fabba-dshm\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:36:38.862175 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:38.862175    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9rhxz\" (UniqueName: \"kubernetes.io/projected/2b8aba73-34f2-4889-890b-46ded76fabba-kube-api-access-9rhxz\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:36:38.862462 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:38.862190    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b8aba73-34f2-4889-890b-46ded76fabba-home\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:36:38.862462 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:38.862207    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b8aba73-34f2-4889-890b-46ded76fabba-tls-certs\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:36:38.862462 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:38.862220    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b8aba73-34f2-4889-890b-46ded76fabba-tmp-dir\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:36:38.870692 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:38.870658    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8442f2/auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd"]
Jul 14 14:36:38.878537 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:38.878497    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-8442f2/auth-test-model-8442f2-kserve-bbb8ff7f5-wcgqd"]
Jul 14 14:36:39.397746 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:39.397702    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-f8f81e/auth-test-model-f8f81e-kserve-6d6749b667-pbbb2" podUID="7180bd41-6c53-4fc7-af1a-0b37fd547edc" containerName="main" containerID="cri-o://7d9bb1e4a50c94e3d1b3356a0d168b3acd29c6b7e88ac7401f08dd683beed846" gracePeriod=45
Jul 14 14:36:39.659684 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:39.659611    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f8f81e/auth-test-model-f8f81e-kserve-6d6749b667-pbbb2"
Jul 14 14:36:39.771044 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:39.771006    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7180bd41-6c53-4fc7-af1a-0b37fd547edc-tmp-dir\") pod \"7180bd41-6c53-4fc7-af1a-0b37fd547edc\" (UID: \"7180bd41-6c53-4fc7-af1a-0b37fd547edc\") "
Jul 14 14:36:39.771044 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:39.771056    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7180bd41-6c53-4fc7-af1a-0b37fd547edc-dshm\") pod \"7180bd41-6c53-4fc7-af1a-0b37fd547edc\" (UID: \"7180bd41-6c53-4fc7-af1a-0b37fd547edc\") "
Jul 14 14:36:39.771333 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:39.771088    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7180bd41-6c53-4fc7-af1a-0b37fd547edc-home\") pod \"7180bd41-6c53-4fc7-af1a-0b37fd547edc\" (UID: \"7180bd41-6c53-4fc7-af1a-0b37fd547edc\") "
Jul 14 14:36:39.771333 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:39.771166    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7180bd41-6c53-4fc7-af1a-0b37fd547edc-model-cache\") pod \"7180bd41-6c53-4fc7-af1a-0b37fd547edc\" (UID: \"7180bd41-6c53-4fc7-af1a-0b37fd547edc\") "
Jul 14 14:36:39.771333 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:39.771205    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7180bd41-6c53-4fc7-af1a-0b37fd547edc-tls-certs\") pod \"7180bd41-6c53-4fc7-af1a-0b37fd547edc\" (UID: \"7180bd41-6c53-4fc7-af1a-0b37fd547edc\") "
Jul 14 14:36:39.771333 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:39.771258    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-48bqs\" (UniqueName: \"kubernetes.io/projected/7180bd41-6c53-4fc7-af1a-0b37fd547edc-kube-api-access-48bqs\") pod \"7180bd41-6c53-4fc7-af1a-0b37fd547edc\" (UID: \"7180bd41-6c53-4fc7-af1a-0b37fd547edc\") "
Jul 14 14:36:39.771523 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:39.771355    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7180bd41-6c53-4fc7-af1a-0b37fd547edc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7180bd41-6c53-4fc7-af1a-0b37fd547edc" (UID: "7180bd41-6c53-4fc7-af1a-0b37fd547edc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 14:36:39.771523 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:39.771438    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7180bd41-6c53-4fc7-af1a-0b37fd547edc-home" (OuterVolumeSpecName: "home") pod "7180bd41-6c53-4fc7-af1a-0b37fd547edc" (UID: "7180bd41-6c53-4fc7-af1a-0b37fd547edc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 14:36:39.771523 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:39.771447    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7180bd41-6c53-4fc7-af1a-0b37fd547edc-model-cache" (OuterVolumeSpecName: "model-cache") pod "7180bd41-6c53-4fc7-af1a-0b37fd547edc" (UID: "7180bd41-6c53-4fc7-af1a-0b37fd547edc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 14:36:39.771716 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:39.771662    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7180bd41-6c53-4fc7-af1a-0b37fd547edc-model-cache\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:36:39.771716 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:39.771681    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7180bd41-6c53-4fc7-af1a-0b37fd547edc-tmp-dir\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:36:39.771716 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:39.771695    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7180bd41-6c53-4fc7-af1a-0b37fd547edc-home\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:36:39.773635 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:39.773563    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7180bd41-6c53-4fc7-af1a-0b37fd547edc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7180bd41-6c53-4fc7-af1a-0b37fd547edc" (UID: "7180bd41-6c53-4fc7-af1a-0b37fd547edc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 14:36:39.773783 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:39.773701    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7180bd41-6c53-4fc7-af1a-0b37fd547edc-dshm" (OuterVolumeSpecName: "dshm") pod "7180bd41-6c53-4fc7-af1a-0b37fd547edc" (UID: "7180bd41-6c53-4fc7-af1a-0b37fd547edc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 14:36:39.773783 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:39.773746    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7180bd41-6c53-4fc7-af1a-0b37fd547edc-kube-api-access-48bqs" (OuterVolumeSpecName: "kube-api-access-48bqs") pod "7180bd41-6c53-4fc7-af1a-0b37fd547edc" (UID: "7180bd41-6c53-4fc7-af1a-0b37fd547edc"). InnerVolumeSpecName "kube-api-access-48bqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 14:36:39.833439 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:39.833396    2576 generic.go:356] "Generic (PLEG): container finished" podID="7180bd41-6c53-4fc7-af1a-0b37fd547edc" containerID="7d9bb1e4a50c94e3d1b3356a0d168b3acd29c6b7e88ac7401f08dd683beed846" exitCode=0
Jul 14 14:36:39.833637 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:39.833449    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f8f81e/auth-test-model-f8f81e-kserve-6d6749b667-pbbb2"
Jul 14 14:36:39.833637 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:39.833491    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f8f81e/auth-test-model-f8f81e-kserve-6d6749b667-pbbb2" event={"ID":"7180bd41-6c53-4fc7-af1a-0b37fd547edc","Type":"ContainerDied","Data":"7d9bb1e4a50c94e3d1b3356a0d168b3acd29c6b7e88ac7401f08dd683beed846"}
Jul 14 14:36:39.833637 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:39.833549    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f8f81e/auth-test-model-f8f81e-kserve-6d6749b667-pbbb2" event={"ID":"7180bd41-6c53-4fc7-af1a-0b37fd547edc","Type":"ContainerDied","Data":"9f027b040f62b96e70b255cd1fa44af00f78db7739d9989b3cf66d9d59cec22e"}
Jul 14 14:36:39.833637 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:39.833572    2576 scope.go:117] "RemoveContainer" containerID="7d9bb1e4a50c94e3d1b3356a0d168b3acd29c6b7e88ac7401f08dd683beed846"
Jul 14 14:36:39.842695 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:39.842676    2576 scope.go:117] "RemoveContainer" containerID="7d9bb1e4a50c94e3d1b3356a0d168b3acd29c6b7e88ac7401f08dd683beed846"
Jul 14 14:36:39.842981 ip-10-0-128-127 kubenswrapper[2576]: E0714 14:36:39.842959    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d9bb1e4a50c94e3d1b3356a0d168b3acd29c6b7e88ac7401f08dd683beed846\": container with ID starting with 7d9bb1e4a50c94e3d1b3356a0d168b3acd29c6b7e88ac7401f08dd683beed846 not found: ID does not exist" containerID="7d9bb1e4a50c94e3d1b3356a0d168b3acd29c6b7e88ac7401f08dd683beed846"
Jul 14 14:36:39.843056 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:39.842990    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d9bb1e4a50c94e3d1b3356a0d168b3acd29c6b7e88ac7401f08dd683beed846"} err="failed to get container status \"7d9bb1e4a50c94e3d1b3356a0d168b3acd29c6b7e88ac7401f08dd683beed846\": rpc error: code = NotFound desc = could not find container \"7d9bb1e4a50c94e3d1b3356a0d168b3acd29c6b7e88ac7401f08dd683beed846\": container with ID starting with 7d9bb1e4a50c94e3d1b3356a0d168b3acd29c6b7e88ac7401f08dd683beed846 not found: ID does not exist"
Jul 14 14:36:39.872347 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:39.872314    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7180bd41-6c53-4fc7-af1a-0b37fd547edc-dshm\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:36:39.872347 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:39.872345    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7180bd41-6c53-4fc7-af1a-0b37fd547edc-tls-certs\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:36:39.872347 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:39.872356    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48bqs\" (UniqueName: \"kubernetes.io/projected/7180bd41-6c53-4fc7-af1a-0b37fd547edc-kube-api-access-48bqs\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:36:39.875910 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:39.875880    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f8f81e/auth-test-model-f8f81e-kserve-6d6749b667-pbbb2"]
Jul 14 14:36:39.884485 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:39.884457    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-f8f81e/auth-test-model-f8f81e-kserve-6d6749b667-pbbb2"]
Jul 14 14:36:40.689739 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:40.689700    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b8aba73-34f2-4889-890b-46ded76fabba" path="/var/lib/kubelet/pods/2b8aba73-34f2-4889-890b-46ded76fabba/volumes"
Jul 14 14:36:40.690288 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:40.690268    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7180bd41-6c53-4fc7-af1a-0b37fd547edc" path="/var/lib/kubelet/pods/7180bd41-6c53-4fc7-af1a-0b37fd547edc/volumes"
Jul 14 14:36:42.443689 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:42.443650    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:36:42.506121 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:42.506085    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:36:46.884631 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:46.884579    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-f8f81e/sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm"]
Jul 14 14:36:46.885119 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:46.885103    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7180bd41-6c53-4fc7-af1a-0b37fd547edc" containerName="main"
Jul 14 14:36:46.885164 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:46.885122    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7180bd41-6c53-4fc7-af1a-0b37fd547edc" containerName="main"
Jul 14 14:36:46.885164 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:46.885157    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b8aba73-34f2-4889-890b-46ded76fabba" containerName="main"
Jul 14 14:36:46.885230 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:46.885167    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b8aba73-34f2-4889-890b-46ded76fabba" containerName="main"
Jul 14 14:36:46.885268 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:46.885242    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7180bd41-6c53-4fc7-af1a-0b37fd547edc" containerName="main"
Jul 14 14:36:46.885268 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:46.885259    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b8aba73-34f2-4889-890b-46ded76fabba" containerName="main"
Jul 14 14:36:46.889894 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:46.889871    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f8f81e/sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm"
Jul 14 14:36:46.893544 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:46.893506    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f8f81e\"/\"sub-test-model-f8f81e-kserve-self-signed-certs\""
Jul 14 14:36:46.893664 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:46.893558    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f8f81e\"/\"openshift-service-ca.crt\""
Jul 14 14:36:46.893664 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:46.893614    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f8f81e\"/\"default-dockercfg-4fwrg\""
Jul 14 14:36:46.893798 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:46.893616    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f8f81e\"/\"kube-root-ca.crt\""
Jul 14 14:36:46.898602 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:46.898565    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f8f81e/sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm"]
Jul 14 14:36:46.924625 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:46.924548    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:36:46.935980 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:46.935943    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f9f0923-57d0-4e65-9a89-6cacf48da40e-home\") pod \"sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm\" (UID: \"9f9f0923-57d0-4e65-9a89-6cacf48da40e\") " pod="ai-tenant-e2e-shared-b-f8f81e/sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm"
Jul 14 14:36:46.936129 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:46.935990    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f9f0923-57d0-4e65-9a89-6cacf48da40e-tmp-dir\") pod \"sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm\" (UID: \"9f9f0923-57d0-4e65-9a89-6cacf48da40e\") " pod="ai-tenant-e2e-shared-b-f8f81e/sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm"
Jul 14 14:36:46.936129 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:46.936022    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f9f0923-57d0-4e65-9a89-6cacf48da40e-tls-certs\") pod \"sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm\" (UID: \"9f9f0923-57d0-4e65-9a89-6cacf48da40e\") " pod="ai-tenant-e2e-shared-b-f8f81e/sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm"
Jul 14 14:36:46.936129 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:46.936040    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f9f0923-57d0-4e65-9a89-6cacf48da40e-dshm\") pod \"sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm\" (UID: \"9f9f0923-57d0-4e65-9a89-6cacf48da40e\") " pod="ai-tenant-e2e-shared-b-f8f81e/sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm"
Jul 14 14:36:46.936129 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:46.936083    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f9f0923-57d0-4e65-9a89-6cacf48da40e-model-cache\") pod \"sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm\" (UID: \"9f9f0923-57d0-4e65-9a89-6cacf48da40e\") " pod="ai-tenant-e2e-shared-b-f8f81e/sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm"
Jul 14 14:36:46.936129 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:46.936110    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6b2sb\" (UniqueName: \"kubernetes.io/projected/9f9f0923-57d0-4e65-9a89-6cacf48da40e-kube-api-access-6b2sb\") pod \"sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm\" (UID: \"9f9f0923-57d0-4e65-9a89-6cacf48da40e\") " pod="ai-tenant-e2e-shared-b-f8f81e/sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm"
Jul 14 14:36:47.037517 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:47.037478    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f9f0923-57d0-4e65-9a89-6cacf48da40e-model-cache\") pod \"sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm\" (UID: \"9f9f0923-57d0-4e65-9a89-6cacf48da40e\") " pod="ai-tenant-e2e-shared-b-f8f81e/sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm"
Jul 14 14:36:47.037725 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:47.037528    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6b2sb\" (UniqueName: \"kubernetes.io/projected/9f9f0923-57d0-4e65-9a89-6cacf48da40e-kube-api-access-6b2sb\") pod \"sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm\" (UID: \"9f9f0923-57d0-4e65-9a89-6cacf48da40e\") " pod="ai-tenant-e2e-shared-b-f8f81e/sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm"
Jul 14 14:36:47.037725 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:47.037629    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f9f0923-57d0-4e65-9a89-6cacf48da40e-home\") pod \"sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm\" (UID: \"9f9f0923-57d0-4e65-9a89-6cacf48da40e\") " pod="ai-tenant-e2e-shared-b-f8f81e/sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm"
Jul 14 14:36:47.037725 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:47.037671    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f9f0923-57d0-4e65-9a89-6cacf48da40e-tmp-dir\") pod \"sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm\" (UID: \"9f9f0923-57d0-4e65-9a89-6cacf48da40e\") " pod="ai-tenant-e2e-shared-b-f8f81e/sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm"
Jul 14 14:36:47.037907 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:47.037727    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f9f0923-57d0-4e65-9a89-6cacf48da40e-tls-certs\") pod \"sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm\" (UID: \"9f9f0923-57d0-4e65-9a89-6cacf48da40e\") " pod="ai-tenant-e2e-shared-b-f8f81e/sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm"
Jul 14 14:36:47.037907 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:47.037753    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f9f0923-57d0-4e65-9a89-6cacf48da40e-dshm\") pod \"sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm\" (UID: \"9f9f0923-57d0-4e65-9a89-6cacf48da40e\") " pod="ai-tenant-e2e-shared-b-f8f81e/sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm"
Jul 14 14:36:47.038015 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:47.037939    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f9f0923-57d0-4e65-9a89-6cacf48da40e-model-cache\") pod \"sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm\" (UID: \"9f9f0923-57d0-4e65-9a89-6cacf48da40e\") " pod="ai-tenant-e2e-shared-b-f8f81e/sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm"
Jul 14 14:36:47.038086 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:47.038062    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f9f0923-57d0-4e65-9a89-6cacf48da40e-home\") pod \"sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm\" (UID: \"9f9f0923-57d0-4e65-9a89-6cacf48da40e\") " pod="ai-tenant-e2e-shared-b-f8f81e/sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm"
Jul 14 14:36:47.038137 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:47.038103    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f9f0923-57d0-4e65-9a89-6cacf48da40e-tmp-dir\") pod \"sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm\" (UID: \"9f9f0923-57d0-4e65-9a89-6cacf48da40e\") " pod="ai-tenant-e2e-shared-b-f8f81e/sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm"
Jul 14 14:36:47.040129 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:47.040109    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f9f0923-57d0-4e65-9a89-6cacf48da40e-dshm\") pod \"sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm\" (UID: \"9f9f0923-57d0-4e65-9a89-6cacf48da40e\") " pod="ai-tenant-e2e-shared-b-f8f81e/sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm"
Jul 14 14:36:47.040322 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:47.040303    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f9f0923-57d0-4e65-9a89-6cacf48da40e-tls-certs\") pod \"sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm\" (UID: \"9f9f0923-57d0-4e65-9a89-6cacf48da40e\") " pod="ai-tenant-e2e-shared-b-f8f81e/sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm"
Jul 14 14:36:47.048647 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:47.048620    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6b2sb\" (UniqueName: \"kubernetes.io/projected/9f9f0923-57d0-4e65-9a89-6cacf48da40e-kube-api-access-6b2sb\") pod \"sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm\" (UID: \"9f9f0923-57d0-4e65-9a89-6cacf48da40e\") " pod="ai-tenant-e2e-shared-b-f8f81e/sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm"
Jul 14 14:36:47.201056 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:47.200954    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f8f81e/sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm"
Jul 14 14:36:47.342088 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:47.342051    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f8f81e/sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm"]
Jul 14 14:36:47.345487 ip-10-0-128-127 kubenswrapper[2576]: W0714 14:36:47.345457    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9f9f0923_57d0_4e65_9a89_6cacf48da40e.slice/crio-da98d7275cf6b0b4d1b4fd5ddb18dfc9bed1ecf4573be2a07bd0404baf06c3a2 WatchSource:0}: Error finding container da98d7275cf6b0b4d1b4fd5ddb18dfc9bed1ecf4573be2a07bd0404baf06c3a2: Status 404 returned error can't find the container with id da98d7275cf6b0b4d1b4fd5ddb18dfc9bed1ecf4573be2a07bd0404baf06c3a2
Jul 14 14:36:47.873305 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:47.873262    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f8f81e/sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm" event={"ID":"9f9f0923-57d0-4e65-9a89-6cacf48da40e","Type":"ContainerStarted","Data":"8fa2fa1f85b8523d9df515d6720957d3bfe8ba02537032e58210dcfa957a682b"}
Jul 14 14:36:47.873305 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:47.873311    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f8f81e/sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm" event={"ID":"9f9f0923-57d0-4e65-9a89-6cacf48da40e","Type":"ContainerStarted","Data":"da98d7275cf6b0b4d1b4fd5ddb18dfc9bed1ecf4573be2a07bd0404baf06c3a2"}
Jul 14 14:36:47.908138 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:47.908083    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-f8f81e/sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm" podStartSLOduration=1.908065009 podStartE2EDuration="1.908065009s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:36:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:36:47.907783625 +0000 UTC m=+2555.758221127" watchObservedRunningTime="2026-07-14 14:36:47.908065009 +0000 UTC m=+2555.758502512"
Jul 14 14:36:50.316958 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:50.316920    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:36:57.201829 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:57.201781    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-f8f81e/sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm"
Jul 14 14:36:57.202297 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:57.201846    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-f8f81e/sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm"
Jul 14 14:36:57.215074 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:57.215042    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-f8f81e/sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm"
Jul 14 14:36:57.928092 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:36:57.928062    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-f8f81e/sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm"
Jul 14 14:37:09.216018 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:09.215977    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:37:15.389846 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:15.389805    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:37:15.454921 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:15.454873    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:37:25.110284 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:25.110245    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:37:25.203671 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:25.203625    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:37:29.926524 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:29.926478    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8442f2/sub-test-model-8442f2-kserve-859fb7f755-b2ndh"]
Jul 14 14:37:29.942296 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:29.942254    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:37:30.798124 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:30.798089    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f8f81e/sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm"]
Jul 14 14:37:30.829101 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:30.829027    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:37:42.317717 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:42.317621    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:37:44.947051 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:44.947004    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-8442f2/sub-test-model-8442f2-kserve-859fb7f755-b2ndh" podUID="c9062dce-1753-4fe0-aa71-57d7468f3ea5" containerName="main" containerID="cri-o://7e95d2d7ad73d3bdb62073d30f234d7b93f5d02da381be501f32f05835162c03" gracePeriod=45
Jul 14 14:37:45.135954 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:45.135914    2576 generic.go:356] "Generic (PLEG): container finished" podID="c9062dce-1753-4fe0-aa71-57d7468f3ea5" containerID="7e95d2d7ad73d3bdb62073d30f234d7b93f5d02da381be501f32f05835162c03" exitCode=0
Jul 14 14:37:45.136160 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:45.135981    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8442f2/sub-test-model-8442f2-kserve-859fb7f755-b2ndh" event={"ID":"c9062dce-1753-4fe0-aa71-57d7468f3ea5","Type":"ContainerDied","Data":"7e95d2d7ad73d3bdb62073d30f234d7b93f5d02da381be501f32f05835162c03"}
Jul 14 14:37:45.139509 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:45.139473    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:37:45.213147 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:45.213066    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8442f2/sub-test-model-8442f2-kserve-859fb7f755-b2ndh"
Jul 14 14:37:45.368919 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:45.368880    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9062dce-1753-4fe0-aa71-57d7468f3ea5-tmp-dir\") pod \"c9062dce-1753-4fe0-aa71-57d7468f3ea5\" (UID: \"c9062dce-1753-4fe0-aa71-57d7468f3ea5\") "
Jul 14 14:37:45.369119 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:45.368940    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9062dce-1753-4fe0-aa71-57d7468f3ea5-model-cache\") pod \"c9062dce-1753-4fe0-aa71-57d7468f3ea5\" (UID: \"c9062dce-1753-4fe0-aa71-57d7468f3ea5\") "
Jul 14 14:37:45.369119 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:45.368958    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9062dce-1753-4fe0-aa71-57d7468f3ea5-dshm\") pod \"c9062dce-1753-4fe0-aa71-57d7468f3ea5\" (UID: \"c9062dce-1753-4fe0-aa71-57d7468f3ea5\") "
Jul 14 14:37:45.369119 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:45.368985    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9062dce-1753-4fe0-aa71-57d7468f3ea5-home\") pod \"c9062dce-1753-4fe0-aa71-57d7468f3ea5\" (UID: \"c9062dce-1753-4fe0-aa71-57d7468f3ea5\") "
Jul 14 14:37:45.369119 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:45.369015    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9062dce-1753-4fe0-aa71-57d7468f3ea5-tls-certs\") pod \"c9062dce-1753-4fe0-aa71-57d7468f3ea5\" (UID: \"c9062dce-1753-4fe0-aa71-57d7468f3ea5\") "
Jul 14 14:37:45.369119 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:45.369046    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c6wzp\" (UniqueName: \"kubernetes.io/projected/c9062dce-1753-4fe0-aa71-57d7468f3ea5-kube-api-access-c6wzp\") pod \"c9062dce-1753-4fe0-aa71-57d7468f3ea5\" (UID: \"c9062dce-1753-4fe0-aa71-57d7468f3ea5\") "
Jul 14 14:37:45.369387 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:45.369231    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9062dce-1753-4fe0-aa71-57d7468f3ea5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c9062dce-1753-4fe0-aa71-57d7468f3ea5" (UID: "c9062dce-1753-4fe0-aa71-57d7468f3ea5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 14:37:45.369387 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:45.369303    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9062dce-1753-4fe0-aa71-57d7468f3ea5-home" (OuterVolumeSpecName: "home") pod "c9062dce-1753-4fe0-aa71-57d7468f3ea5" (UID: "c9062dce-1753-4fe0-aa71-57d7468f3ea5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 14:37:45.369387 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:45.369329    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9062dce-1753-4fe0-aa71-57d7468f3ea5-model-cache" (OuterVolumeSpecName: "model-cache") pod "c9062dce-1753-4fe0-aa71-57d7468f3ea5" (UID: "c9062dce-1753-4fe0-aa71-57d7468f3ea5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 14:37:45.369387 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:45.369374    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9062dce-1753-4fe0-aa71-57d7468f3ea5-tmp-dir\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:37:45.371213 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:45.371187    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9062dce-1753-4fe0-aa71-57d7468f3ea5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c9062dce-1753-4fe0-aa71-57d7468f3ea5" (UID: "c9062dce-1753-4fe0-aa71-57d7468f3ea5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 14:37:45.371329 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:45.371228    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9062dce-1753-4fe0-aa71-57d7468f3ea5-dshm" (OuterVolumeSpecName: "dshm") pod "c9062dce-1753-4fe0-aa71-57d7468f3ea5" (UID: "c9062dce-1753-4fe0-aa71-57d7468f3ea5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 14:37:45.371329 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:45.371288    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9062dce-1753-4fe0-aa71-57d7468f3ea5-kube-api-access-c6wzp" (OuterVolumeSpecName: "kube-api-access-c6wzp") pod "c9062dce-1753-4fe0-aa71-57d7468f3ea5" (UID: "c9062dce-1753-4fe0-aa71-57d7468f3ea5"). InnerVolumeSpecName "kube-api-access-c6wzp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 14:37:45.470487 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:45.470391    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9062dce-1753-4fe0-aa71-57d7468f3ea5-tls-certs\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:37:45.470487 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:45.470427    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6wzp\" (UniqueName: \"kubernetes.io/projected/c9062dce-1753-4fe0-aa71-57d7468f3ea5-kube-api-access-c6wzp\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:37:45.470487 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:45.470437    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9062dce-1753-4fe0-aa71-57d7468f3ea5-model-cache\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:37:45.470487 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:45.470448    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9062dce-1753-4fe0-aa71-57d7468f3ea5-dshm\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:37:45.470487 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:45.470455    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9062dce-1753-4fe0-aa71-57d7468f3ea5-home\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:37:45.818334 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:45.818273    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-f8f81e/sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm" podUID="9f9f0923-57d0-4e65-9a89-6cacf48da40e" containerName="main" containerID="cri-o://8fa2fa1f85b8523d9df515d6720957d3bfe8ba02537032e58210dcfa957a682b" gracePeriod=45
Jul 14 14:37:46.075431 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:46.075359    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f8f81e/sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm"
Jul 14 14:37:46.141737 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:46.141701    2576 generic.go:356] "Generic (PLEG): container finished" podID="9f9f0923-57d0-4e65-9a89-6cacf48da40e" containerID="8fa2fa1f85b8523d9df515d6720957d3bfe8ba02537032e58210dcfa957a682b" exitCode=0
Jul 14 14:37:46.141737 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:46.141739    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f8f81e/sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm"
Jul 14 14:37:46.142274 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:46.141794    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f8f81e/sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm" event={"ID":"9f9f0923-57d0-4e65-9a89-6cacf48da40e","Type":"ContainerDied","Data":"8fa2fa1f85b8523d9df515d6720957d3bfe8ba02537032e58210dcfa957a682b"}
Jul 14 14:37:46.142274 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:46.141840    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f8f81e/sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm" event={"ID":"9f9f0923-57d0-4e65-9a89-6cacf48da40e","Type":"ContainerDied","Data":"da98d7275cf6b0b4d1b4fd5ddb18dfc9bed1ecf4573be2a07bd0404baf06c3a2"}
Jul 14 14:37:46.142274 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:46.141856    2576 scope.go:117] "RemoveContainer" containerID="8fa2fa1f85b8523d9df515d6720957d3bfe8ba02537032e58210dcfa957a682b"
Jul 14 14:37:46.143345 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:46.143317    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8442f2/sub-test-model-8442f2-kserve-859fb7f755-b2ndh" event={"ID":"c9062dce-1753-4fe0-aa71-57d7468f3ea5","Type":"ContainerDied","Data":"42df2b962de846346ee96f41957b69b0e716c37b587e3dfbc3629549e5acad81"}
Jul 14 14:37:46.143448 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:46.143349    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8442f2/sub-test-model-8442f2-kserve-859fb7f755-b2ndh"
Jul 14 14:37:46.150395 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:46.150377    2576 scope.go:117] "RemoveContainer" containerID="8fa2fa1f85b8523d9df515d6720957d3bfe8ba02537032e58210dcfa957a682b"
Jul 14 14:37:46.150743 ip-10-0-128-127 kubenswrapper[2576]: E0714 14:37:46.150718    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fa2fa1f85b8523d9df515d6720957d3bfe8ba02537032e58210dcfa957a682b\": container with ID starting with 8fa2fa1f85b8523d9df515d6720957d3bfe8ba02537032e58210dcfa957a682b not found: ID does not exist" containerID="8fa2fa1f85b8523d9df515d6720957d3bfe8ba02537032e58210dcfa957a682b"
Jul 14 14:37:46.150853 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:46.150753    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fa2fa1f85b8523d9df515d6720957d3bfe8ba02537032e58210dcfa957a682b"} err="failed to get container status \"8fa2fa1f85b8523d9df515d6720957d3bfe8ba02537032e58210dcfa957a682b\": rpc error: code = NotFound desc = could not find container \"8fa2fa1f85b8523d9df515d6720957d3bfe8ba02537032e58210dcfa957a682b\": container with ID starting with 8fa2fa1f85b8523d9df515d6720957d3bfe8ba02537032e58210dcfa957a682b not found: ID does not exist"
Jul 14 14:37:46.150853 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:46.150777    2576 scope.go:117] "RemoveContainer" containerID="7e95d2d7ad73d3bdb62073d30f234d7b93f5d02da381be501f32f05835162c03"
Jul 14 14:37:46.166665 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:46.166629    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8442f2/sub-test-model-8442f2-kserve-859fb7f755-b2ndh"]
Jul 14 14:37:46.170764 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:46.170735    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-8442f2/sub-test-model-8442f2-kserve-859fb7f755-b2ndh"]
Jul 14 14:37:46.176755 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:46.176731    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6b2sb\" (UniqueName: \"kubernetes.io/projected/9f9f0923-57d0-4e65-9a89-6cacf48da40e-kube-api-access-6b2sb\") pod \"9f9f0923-57d0-4e65-9a89-6cacf48da40e\" (UID: \"9f9f0923-57d0-4e65-9a89-6cacf48da40e\") "
Jul 14 14:37:46.176867 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:46.176778    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f9f0923-57d0-4e65-9a89-6cacf48da40e-home\") pod \"9f9f0923-57d0-4e65-9a89-6cacf48da40e\" (UID: \"9f9f0923-57d0-4e65-9a89-6cacf48da40e\") "
Jul 14 14:37:46.176867 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:46.176828    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f9f0923-57d0-4e65-9a89-6cacf48da40e-tmp-dir\") pod \"9f9f0923-57d0-4e65-9a89-6cacf48da40e\" (UID: \"9f9f0923-57d0-4e65-9a89-6cacf48da40e\") "
Jul 14 14:37:46.176976 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:46.176874    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f9f0923-57d0-4e65-9a89-6cacf48da40e-dshm\") pod \"9f9f0923-57d0-4e65-9a89-6cacf48da40e\" (UID: \"9f9f0923-57d0-4e65-9a89-6cacf48da40e\") "
Jul 14 14:37:46.176976 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:46.176897    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f9f0923-57d0-4e65-9a89-6cacf48da40e-model-cache\") pod \"9f9f0923-57d0-4e65-9a89-6cacf48da40e\" (UID: \"9f9f0923-57d0-4e65-9a89-6cacf48da40e\") "
Jul 14 14:37:46.176976 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:46.176933    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f9f0923-57d0-4e65-9a89-6cacf48da40e-tls-certs\") pod \"9f9f0923-57d0-4e65-9a89-6cacf48da40e\" (UID: \"9f9f0923-57d0-4e65-9a89-6cacf48da40e\") "
Jul 14 14:37:46.177166 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:46.177117    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f9f0923-57d0-4e65-9a89-6cacf48da40e-home" (OuterVolumeSpecName: "home") pod "9f9f0923-57d0-4e65-9a89-6cacf48da40e" (UID: "9f9f0923-57d0-4e65-9a89-6cacf48da40e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 14:37:46.177297 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:46.177132    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f9f0923-57d0-4e65-9a89-6cacf48da40e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9f9f0923-57d0-4e65-9a89-6cacf48da40e" (UID: "9f9f0923-57d0-4e65-9a89-6cacf48da40e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 14:37:46.177297 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:46.177192    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f9f0923-57d0-4e65-9a89-6cacf48da40e-model-cache" (OuterVolumeSpecName: "model-cache") pod "9f9f0923-57d0-4e65-9a89-6cacf48da40e" (UID: "9f9f0923-57d0-4e65-9a89-6cacf48da40e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 14:37:46.179040 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:46.179016    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f9f0923-57d0-4e65-9a89-6cacf48da40e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9f9f0923-57d0-4e65-9a89-6cacf48da40e" (UID: "9f9f0923-57d0-4e65-9a89-6cacf48da40e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 14:37:46.179151 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:46.179057    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f9f0923-57d0-4e65-9a89-6cacf48da40e-dshm" (OuterVolumeSpecName: "dshm") pod "9f9f0923-57d0-4e65-9a89-6cacf48da40e" (UID: "9f9f0923-57d0-4e65-9a89-6cacf48da40e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 14:37:46.179151 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:46.179069    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f9f0923-57d0-4e65-9a89-6cacf48da40e-kube-api-access-6b2sb" (OuterVolumeSpecName: "kube-api-access-6b2sb") pod "9f9f0923-57d0-4e65-9a89-6cacf48da40e" (UID: "9f9f0923-57d0-4e65-9a89-6cacf48da40e"). InnerVolumeSpecName "kube-api-access-6b2sb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 14:37:46.278616 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:46.278566    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f9f0923-57d0-4e65-9a89-6cacf48da40e-dshm\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:37:46.278798 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:46.278644    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f9f0923-57d0-4e65-9a89-6cacf48da40e-model-cache\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:37:46.278798 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:46.278657    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f9f0923-57d0-4e65-9a89-6cacf48da40e-tls-certs\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:37:46.278798 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:46.278665    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6b2sb\" (UniqueName: \"kubernetes.io/projected/9f9f0923-57d0-4e65-9a89-6cacf48da40e-kube-api-access-6b2sb\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:37:46.278798 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:46.278674    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f9f0923-57d0-4e65-9a89-6cacf48da40e-home\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:37:46.278798 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:46.278704    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f9f0923-57d0-4e65-9a89-6cacf48da40e-tmp-dir\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:37:46.464794 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:46.464758    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f8f81e/sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm"]
Jul 14 14:37:46.471371 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:46.471330    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-f8f81e/sub-test-model-f8f81e-kserve-6df4ffbbdc-dmxpm"]
Jul 14 14:37:46.688562 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:46.688532    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f9f0923-57d0-4e65-9a89-6cacf48da40e" path="/var/lib/kubelet/pods/9f9f0923-57d0-4e65-9a89-6cacf48da40e/volumes"
Jul 14 14:37:46.688957 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:46.688942    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9062dce-1753-4fe0-aa71-57d7468f3ea5" path="/var/lib/kubelet/pods/c9062dce-1753-4fe0-aa71-57d7468f3ea5/volumes"
Jul 14 14:37:55.406985 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:55.406949    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:37:55.699279 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:55.699179    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:37:56.473212 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:37:56.473167    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:38:10.183070 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:10.183033    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-f8f81e/rate-test-model-f8f81e-kserve-59586b96c-66lhh"]
Jul 14 14:38:10.183556 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:10.183373    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f9f0923-57d0-4e65-9a89-6cacf48da40e" containerName="main"
Jul 14 14:38:10.183556 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:10.183385    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f9f0923-57d0-4e65-9a89-6cacf48da40e" containerName="main"
Jul 14 14:38:10.183556 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:10.183405    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9062dce-1753-4fe0-aa71-57d7468f3ea5" containerName="main"
Jul 14 14:38:10.183556 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:10.183412    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9062dce-1753-4fe0-aa71-57d7468f3ea5" containerName="main"
Jul 14 14:38:10.183556 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:10.183467    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9062dce-1753-4fe0-aa71-57d7468f3ea5" containerName="main"
Jul 14 14:38:10.183556 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:10.183480    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f9f0923-57d0-4e65-9a89-6cacf48da40e" containerName="main"
Jul 14 14:38:10.186637 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:10.186617    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f8f81e/rate-test-model-f8f81e-kserve-59586b96c-66lhh"
Jul 14 14:38:10.189258 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:10.189238    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f8f81e\"/\"kube-root-ca.crt\""
Jul 14 14:38:10.190358 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:10.190337    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f8f81e\"/\"rate-test-model-f8f81e-kserve-self-signed-certs\""
Jul 14 14:38:10.190450 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:10.190375    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f8f81e\"/\"openshift-service-ca.crt\""
Jul 14 14:38:10.190521 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:10.190472    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f8f81e\"/\"default-dockercfg-4fwrg\""
Jul 14 14:38:10.197189 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:10.197164    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f8f81e/rate-test-model-f8f81e-kserve-59586b96c-66lhh"]
Jul 14 14:38:10.217692 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:10.217651    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:38:10.271445 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:10.271400    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91f8f107-d43f-4ac6-9ef4-d4e4eb42391d-tls-certs\") pod \"rate-test-model-f8f81e-kserve-59586b96c-66lhh\" (UID: \"91f8f107-d43f-4ac6-9ef4-d4e4eb42391d\") " pod="ai-tenant-e2e-shared-b-f8f81e/rate-test-model-f8f81e-kserve-59586b96c-66lhh"
Jul 14 14:38:10.271445 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:10.271445    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91f8f107-d43f-4ac6-9ef4-d4e4eb42391d-tmp-dir\") pod \"rate-test-model-f8f81e-kserve-59586b96c-66lhh\" (UID: \"91f8f107-d43f-4ac6-9ef4-d4e4eb42391d\") " pod="ai-tenant-e2e-shared-b-f8f81e/rate-test-model-f8f81e-kserve-59586b96c-66lhh"
Jul 14 14:38:10.271738 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:10.271515    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91f8f107-d43f-4ac6-9ef4-d4e4eb42391d-dshm\") pod \"rate-test-model-f8f81e-kserve-59586b96c-66lhh\" (UID: \"91f8f107-d43f-4ac6-9ef4-d4e4eb42391d\") " pod="ai-tenant-e2e-shared-b-f8f81e/rate-test-model-f8f81e-kserve-59586b96c-66lhh"
Jul 14 14:38:10.271738 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:10.271557    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91f8f107-d43f-4ac6-9ef4-d4e4eb42391d-home\") pod \"rate-test-model-f8f81e-kserve-59586b96c-66lhh\" (UID: \"91f8f107-d43f-4ac6-9ef4-d4e4eb42391d\") " pod="ai-tenant-e2e-shared-b-f8f81e/rate-test-model-f8f81e-kserve-59586b96c-66lhh"
Jul 14 14:38:10.271738 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:10.271661    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91f8f107-d43f-4ac6-9ef4-d4e4eb42391d-model-cache\") pod \"rate-test-model-f8f81e-kserve-59586b96c-66lhh\" (UID: \"91f8f107-d43f-4ac6-9ef4-d4e4eb42391d\") " pod="ai-tenant-e2e-shared-b-f8f81e/rate-test-model-f8f81e-kserve-59586b96c-66lhh"
Jul 14 14:38:10.271738 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:10.271700    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg877\" (UniqueName: \"kubernetes.io/projected/91f8f107-d43f-4ac6-9ef4-d4e4eb42391d-kube-api-access-mg877\") pod \"rate-test-model-f8f81e-kserve-59586b96c-66lhh\" (UID: \"91f8f107-d43f-4ac6-9ef4-d4e4eb42391d\") " pod="ai-tenant-e2e-shared-b-f8f81e/rate-test-model-f8f81e-kserve-59586b96c-66lhh"
Jul 14 14:38:10.372541 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:10.372477    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91f8f107-d43f-4ac6-9ef4-d4e4eb42391d-tls-certs\") pod \"rate-test-model-f8f81e-kserve-59586b96c-66lhh\" (UID: \"91f8f107-d43f-4ac6-9ef4-d4e4eb42391d\") " pod="ai-tenant-e2e-shared-b-f8f81e/rate-test-model-f8f81e-kserve-59586b96c-66lhh"
Jul 14 14:38:10.372541 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:10.372546    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91f8f107-d43f-4ac6-9ef4-d4e4eb42391d-tmp-dir\") pod \"rate-test-model-f8f81e-kserve-59586b96c-66lhh\" (UID: \"91f8f107-d43f-4ac6-9ef4-d4e4eb42391d\") " pod="ai-tenant-e2e-shared-b-f8f81e/rate-test-model-f8f81e-kserve-59586b96c-66lhh"
Jul 14 14:38:10.372848 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:10.372577    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91f8f107-d43f-4ac6-9ef4-d4e4eb42391d-dshm\") pod \"rate-test-model-f8f81e-kserve-59586b96c-66lhh\" (UID: \"91f8f107-d43f-4ac6-9ef4-d4e4eb42391d\") " pod="ai-tenant-e2e-shared-b-f8f81e/rate-test-model-f8f81e-kserve-59586b96c-66lhh"
Jul 14 14:38:10.372848 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:10.372617    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91f8f107-d43f-4ac6-9ef4-d4e4eb42391d-home\") pod \"rate-test-model-f8f81e-kserve-59586b96c-66lhh\" (UID: \"91f8f107-d43f-4ac6-9ef4-d4e4eb42391d\") " pod="ai-tenant-e2e-shared-b-f8f81e/rate-test-model-f8f81e-kserve-59586b96c-66lhh"
Jul 14 14:38:10.372848 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:10.372645    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91f8f107-d43f-4ac6-9ef4-d4e4eb42391d-model-cache\") pod \"rate-test-model-f8f81e-kserve-59586b96c-66lhh\" (UID: \"91f8f107-d43f-4ac6-9ef4-d4e4eb42391d\") " pod="ai-tenant-e2e-shared-b-f8f81e/rate-test-model-f8f81e-kserve-59586b96c-66lhh"
Jul 14 14:38:10.372848 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:10.372689    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg877\" (UniqueName: \"kubernetes.io/projected/91f8f107-d43f-4ac6-9ef4-d4e4eb42391d-kube-api-access-mg877\") pod \"rate-test-model-f8f81e-kserve-59586b96c-66lhh\" (UID: \"91f8f107-d43f-4ac6-9ef4-d4e4eb42391d\") " pod="ai-tenant-e2e-shared-b-f8f81e/rate-test-model-f8f81e-kserve-59586b96c-66lhh"
Jul 14 14:38:10.373071 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:10.373045    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91f8f107-d43f-4ac6-9ef4-d4e4eb42391d-tmp-dir\") pod \"rate-test-model-f8f81e-kserve-59586b96c-66lhh\" (UID: \"91f8f107-d43f-4ac6-9ef4-d4e4eb42391d\") " pod="ai-tenant-e2e-shared-b-f8f81e/rate-test-model-f8f81e-kserve-59586b96c-66lhh"
Jul 14 14:38:10.373135 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:10.373118    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91f8f107-d43f-4ac6-9ef4-d4e4eb42391d-home\") pod \"rate-test-model-f8f81e-kserve-59586b96c-66lhh\" (UID: \"91f8f107-d43f-4ac6-9ef4-d4e4eb42391d\") " pod="ai-tenant-e2e-shared-b-f8f81e/rate-test-model-f8f81e-kserve-59586b96c-66lhh"
Jul 14 14:38:10.373186 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:10.373139    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91f8f107-d43f-4ac6-9ef4-d4e4eb42391d-model-cache\") pod \"rate-test-model-f8f81e-kserve-59586b96c-66lhh\" (UID: \"91f8f107-d43f-4ac6-9ef4-d4e4eb42391d\") " pod="ai-tenant-e2e-shared-b-f8f81e/rate-test-model-f8f81e-kserve-59586b96c-66lhh"
Jul 14 14:38:10.374976 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:10.374953    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91f8f107-d43f-4ac6-9ef4-d4e4eb42391d-dshm\") pod \"rate-test-model-f8f81e-kserve-59586b96c-66lhh\" (UID: \"91f8f107-d43f-4ac6-9ef4-d4e4eb42391d\") " pod="ai-tenant-e2e-shared-b-f8f81e/rate-test-model-f8f81e-kserve-59586b96c-66lhh"
Jul 14 14:38:10.375136 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:10.375117    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91f8f107-d43f-4ac6-9ef4-d4e4eb42391d-tls-certs\") pod \"rate-test-model-f8f81e-kserve-59586b96c-66lhh\" (UID: \"91f8f107-d43f-4ac6-9ef4-d4e4eb42391d\") " pod="ai-tenant-e2e-shared-b-f8f81e/rate-test-model-f8f81e-kserve-59586b96c-66lhh"
Jul 14 14:38:10.382224 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:10.382193    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg877\" (UniqueName: \"kubernetes.io/projected/91f8f107-d43f-4ac6-9ef4-d4e4eb42391d-kube-api-access-mg877\") pod \"rate-test-model-f8f81e-kserve-59586b96c-66lhh\" (UID: \"91f8f107-d43f-4ac6-9ef4-d4e4eb42391d\") " pod="ai-tenant-e2e-shared-b-f8f81e/rate-test-model-f8f81e-kserve-59586b96c-66lhh"
Jul 14 14:38:10.497296 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:10.497176    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f8f81e/rate-test-model-f8f81e-kserve-59586b96c-66lhh"
Jul 14 14:38:10.638837 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:10.638800    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f8f81e/rate-test-model-f8f81e-kserve-59586b96c-66lhh"]
Jul 14 14:38:10.642571 ip-10-0-128-127 kubenswrapper[2576]: W0714 14:38:10.642531    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod91f8f107_d43f_4ac6_9ef4_d4e4eb42391d.slice/crio-f878214534ebb051c34b8ba5661c7001758a45fc4f2be6807fb51c7d763176d4 WatchSource:0}: Error finding container f878214534ebb051c34b8ba5661c7001758a45fc4f2be6807fb51c7d763176d4: Status 404 returned error can't find the container with id f878214534ebb051c34b8ba5661c7001758a45fc4f2be6807fb51c7d763176d4
Jul 14 14:38:11.265355 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:11.265315    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f8f81e/rate-test-model-f8f81e-kserve-59586b96c-66lhh" event={"ID":"91f8f107-d43f-4ac6-9ef4-d4e4eb42391d","Type":"ContainerStarted","Data":"7ac23f38f6c4273c71e68629d3fc80bf62fb441171c4d7b09e76a45b5ab94dca"}
Jul 14 14:38:11.265355 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:11.265359    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f8f81e/rate-test-model-f8f81e-kserve-59586b96c-66lhh" event={"ID":"91f8f107-d43f-4ac6-9ef4-d4e4eb42391d","Type":"ContainerStarted","Data":"f878214534ebb051c34b8ba5661c7001758a45fc4f2be6807fb51c7d763176d4"}
Jul 14 14:38:11.284772 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:11.284718    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-f8f81e/rate-test-model-f8f81e-kserve-59586b96c-66lhh" podStartSLOduration=1.284702646 podStartE2EDuration="1.284702646s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:38:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:38:11.283195205 +0000 UTC m=+2639.133632708" watchObservedRunningTime="2026-07-14 14:38:11.284702646 +0000 UTC m=+2639.135140149"
Jul 14 14:38:14.929683 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:14.929647    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:38:20.498081 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:20.498030    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-f8f81e/rate-test-model-f8f81e-kserve-59586b96c-66lhh"
Jul 14 14:38:20.498633 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:20.498104    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-f8f81e/rate-test-model-f8f81e-kserve-59586b96c-66lhh"
Jul 14 14:38:20.510775 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:20.510744    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-f8f81e/rate-test-model-f8f81e-kserve-59586b96c-66lhh"
Jul 14 14:38:21.319237 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:21.319200    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-f8f81e/rate-test-model-f8f81e-kserve-59586b96c-66lhh"
Jul 14 14:38:30.165261 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:30.165214    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:38:30.266404 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:30.266366    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:38:39.213551 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:39.213510    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:38:49.980502 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:49.980460    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:38:50.100188 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:50.100154    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:38:57.309854 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:57.309809    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:38:57.977328 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:57.977284    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:38:58.077438 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:38:58.077399    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:39:02.230998 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:02.230959    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:39:03.264102 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:03.264064    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f8f81e/rate-test-model-f8f81e-kserve-59586b96c-66lhh"]
Jul 14 14:39:03.297434 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:03.297391    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:39:12.739215 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:12.739174    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sh6rt_5600e3e1-e775-482e-bc72-76d732552705/ovn-acl-logging/0.log"
Jul 14 14:39:12.744017 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:12.743993    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sh6rt_5600e3e1-e775-482e-bc72-76d732552705/ovn-acl-logging/0.log"
Jul 14 14:39:15.220228 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:15.220192    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:39:18.288478 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:18.288405    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-f8f81e/rate-test-model-f8f81e-kserve-59586b96c-66lhh" podUID="91f8f107-d43f-4ac6-9ef4-d4e4eb42391d" containerName="main" containerID="cri-o://7ac23f38f6c4273c71e68629d3fc80bf62fb441171c4d7b09e76a45b5ab94dca" gracePeriod=45
Jul 14 14:39:18.531088 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:18.531063    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f8f81e/rate-test-model-f8f81e-kserve-59586b96c-66lhh"
Jul 14 14:39:18.565458 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:18.565368    2576 generic.go:356] "Generic (PLEG): container finished" podID="91f8f107-d43f-4ac6-9ef4-d4e4eb42391d" containerID="7ac23f38f6c4273c71e68629d3fc80bf62fb441171c4d7b09e76a45b5ab94dca" exitCode=0
Jul 14 14:39:18.565458 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:18.565419    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f8f81e/rate-test-model-f8f81e-kserve-59586b96c-66lhh"
Jul 14 14:39:18.565458 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:18.565451    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f8f81e/rate-test-model-f8f81e-kserve-59586b96c-66lhh" event={"ID":"91f8f107-d43f-4ac6-9ef4-d4e4eb42391d","Type":"ContainerDied","Data":"7ac23f38f6c4273c71e68629d3fc80bf62fb441171c4d7b09e76a45b5ab94dca"}
Jul 14 14:39:18.565709 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:18.565490    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f8f81e/rate-test-model-f8f81e-kserve-59586b96c-66lhh" event={"ID":"91f8f107-d43f-4ac6-9ef4-d4e4eb42391d","Type":"ContainerDied","Data":"f878214534ebb051c34b8ba5661c7001758a45fc4f2be6807fb51c7d763176d4"}
Jul 14 14:39:18.565709 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:18.565506    2576 scope.go:117] "RemoveContainer" containerID="7ac23f38f6c4273c71e68629d3fc80bf62fb441171c4d7b09e76a45b5ab94dca"
Jul 14 14:39:18.574936 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:18.574913    2576 scope.go:117] "RemoveContainer" containerID="7ac23f38f6c4273c71e68629d3fc80bf62fb441171c4d7b09e76a45b5ab94dca"
Jul 14 14:39:18.575307 ip-10-0-128-127 kubenswrapper[2576]: E0714 14:39:18.575282    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ac23f38f6c4273c71e68629d3fc80bf62fb441171c4d7b09e76a45b5ab94dca\": container with ID starting with 7ac23f38f6c4273c71e68629d3fc80bf62fb441171c4d7b09e76a45b5ab94dca not found: ID does not exist" containerID="7ac23f38f6c4273c71e68629d3fc80bf62fb441171c4d7b09e76a45b5ab94dca"
Jul 14 14:39:18.575362 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:18.575318    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ac23f38f6c4273c71e68629d3fc80bf62fb441171c4d7b09e76a45b5ab94dca"} err="failed to get container status \"7ac23f38f6c4273c71e68629d3fc80bf62fb441171c4d7b09e76a45b5ab94dca\": rpc error: code = NotFound desc = could not find container \"7ac23f38f6c4273c71e68629d3fc80bf62fb441171c4d7b09e76a45b5ab94dca\": container with ID starting with 7ac23f38f6c4273c71e68629d3fc80bf62fb441171c4d7b09e76a45b5ab94dca not found: ID does not exist"
Jul 14 14:39:18.702688 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:18.702647    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91f8f107-d43f-4ac6-9ef4-d4e4eb42391d-home\") pod \"91f8f107-d43f-4ac6-9ef4-d4e4eb42391d\" (UID: \"91f8f107-d43f-4ac6-9ef4-d4e4eb42391d\") "
Jul 14 14:39:18.702896 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:18.702716    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91f8f107-d43f-4ac6-9ef4-d4e4eb42391d-dshm\") pod \"91f8f107-d43f-4ac6-9ef4-d4e4eb42391d\" (UID: \"91f8f107-d43f-4ac6-9ef4-d4e4eb42391d\") "
Jul 14 14:39:18.702896 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:18.702767    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91f8f107-d43f-4ac6-9ef4-d4e4eb42391d-tls-certs\") pod \"91f8f107-d43f-4ac6-9ef4-d4e4eb42391d\" (UID: \"91f8f107-d43f-4ac6-9ef4-d4e4eb42391d\") "
Jul 14 14:39:18.702896 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:18.702785    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91f8f107-d43f-4ac6-9ef4-d4e4eb42391d-model-cache\") pod \"91f8f107-d43f-4ac6-9ef4-d4e4eb42391d\" (UID: \"91f8f107-d43f-4ac6-9ef4-d4e4eb42391d\") "
Jul 14 14:39:18.702896 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:18.702804    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91f8f107-d43f-4ac6-9ef4-d4e4eb42391d-tmp-dir\") pod \"91f8f107-d43f-4ac6-9ef4-d4e4eb42391d\" (UID: \"91f8f107-d43f-4ac6-9ef4-d4e4eb42391d\") "
Jul 14 14:39:18.702896 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:18.702847    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mg877\" (UniqueName: \"kubernetes.io/projected/91f8f107-d43f-4ac6-9ef4-d4e4eb42391d-kube-api-access-mg877\") pod \"91f8f107-d43f-4ac6-9ef4-d4e4eb42391d\" (UID: \"91f8f107-d43f-4ac6-9ef4-d4e4eb42391d\") "
Jul 14 14:39:18.703166 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:18.702924    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91f8f107-d43f-4ac6-9ef4-d4e4eb42391d-home" (OuterVolumeSpecName: "home") pod "91f8f107-d43f-4ac6-9ef4-d4e4eb42391d" (UID: "91f8f107-d43f-4ac6-9ef4-d4e4eb42391d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 14:39:18.703166 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:18.703113    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91f8f107-d43f-4ac6-9ef4-d4e4eb42391d-home\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:39:18.703166 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:18.703089    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91f8f107-d43f-4ac6-9ef4-d4e4eb42391d-model-cache" (OuterVolumeSpecName: "model-cache") pod "91f8f107-d43f-4ac6-9ef4-d4e4eb42391d" (UID: "91f8f107-d43f-4ac6-9ef4-d4e4eb42391d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 14:39:18.703326 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:18.703161    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91f8f107-d43f-4ac6-9ef4-d4e4eb42391d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "91f8f107-d43f-4ac6-9ef4-d4e4eb42391d" (UID: "91f8f107-d43f-4ac6-9ef4-d4e4eb42391d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 14:39:18.705007 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:18.704977    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91f8f107-d43f-4ac6-9ef4-d4e4eb42391d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "91f8f107-d43f-4ac6-9ef4-d4e4eb42391d" (UID: "91f8f107-d43f-4ac6-9ef4-d4e4eb42391d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 14:39:18.705117 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:18.705012    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91f8f107-d43f-4ac6-9ef4-d4e4eb42391d-dshm" (OuterVolumeSpecName: "dshm") pod "91f8f107-d43f-4ac6-9ef4-d4e4eb42391d" (UID: "91f8f107-d43f-4ac6-9ef4-d4e4eb42391d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 14:39:18.705117 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:18.705019    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91f8f107-d43f-4ac6-9ef4-d4e4eb42391d-kube-api-access-mg877" (OuterVolumeSpecName: "kube-api-access-mg877") pod "91f8f107-d43f-4ac6-9ef4-d4e4eb42391d" (UID: "91f8f107-d43f-4ac6-9ef4-d4e4eb42391d"). InnerVolumeSpecName "kube-api-access-mg877". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 14:39:18.803921 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:18.803881    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91f8f107-d43f-4ac6-9ef4-d4e4eb42391d-tls-certs\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:39:18.803921 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:18.803916    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91f8f107-d43f-4ac6-9ef4-d4e4eb42391d-model-cache\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:39:18.803921 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:18.803924    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91f8f107-d43f-4ac6-9ef4-d4e4eb42391d-tmp-dir\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:39:18.803921 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:18.803933    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mg877\" (UniqueName: \"kubernetes.io/projected/91f8f107-d43f-4ac6-9ef4-d4e4eb42391d-kube-api-access-mg877\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:39:18.804209 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:18.803943    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91f8f107-d43f-4ac6-9ef4-d4e4eb42391d-dshm\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:39:18.889924 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:18.889887    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f8f81e/rate-test-model-f8f81e-kserve-59586b96c-66lhh"]
Jul 14 14:39:18.895896 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:18.895865    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-f8f81e/rate-test-model-f8f81e-kserve-59586b96c-66lhh"]
Jul 14 14:39:20.689775 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:20.689737    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91f8f107-d43f-4ac6-9ef4-d4e4eb42391d" path="/var/lib/kubelet/pods/91f8f107-d43f-4ac6-9ef4-d4e4eb42391d/volumes"
Jul 14 14:39:23.521987 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:23.521950    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:39:23.645090 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:23.645044    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:39:33.409769 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:33.409718    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8442f2-8448d45f8-xx4s8"]
Jul 14 14:39:33.410201 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:33.410042    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8442f2-8448d45f8-xx4s8" podUID="eb9b8831-8748-47c2-b77c-67a9e8e960d3" containerName="maas-api" containerID="cri-o://404d28779907b8867a4237e570ca5d9fe1bb7faa9a13ec60f5f94895696a1b63" gracePeriod=30
Jul 14 14:39:33.435226 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:33.435190    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:39:33.632232 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:33.632193    2576 generic.go:356] "Generic (PLEG): container finished" podID="eb9b8831-8748-47c2-b77c-67a9e8e960d3" containerID="404d28779907b8867a4237e570ca5d9fe1bb7faa9a13ec60f5f94895696a1b63" exitCode=0
Jul 14 14:39:33.632412 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:33.632257    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8442f2-8448d45f8-xx4s8" event={"ID":"eb9b8831-8748-47c2-b77c-67a9e8e960d3","Type":"ContainerDied","Data":"404d28779907b8867a4237e570ca5d9fe1bb7faa9a13ec60f5f94895696a1b63"}
Jul 14 14:39:33.664498 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:33.664468    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8442f2-8448d45f8-xx4s8"
Jul 14 14:39:33.735697 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:33.735651    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7wrl6\" (UniqueName: \"kubernetes.io/projected/eb9b8831-8748-47c2-b77c-67a9e8e960d3-kube-api-access-7wrl6\") pod \"eb9b8831-8748-47c2-b77c-67a9e8e960d3\" (UID: \"eb9b8831-8748-47c2-b77c-67a9e8e960d3\") "
Jul 14 14:39:33.735913 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:33.735710    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb9b8831-8748-47c2-b77c-67a9e8e960d3-maas-api-tls\") pod \"eb9b8831-8748-47c2-b77c-67a9e8e960d3\" (UID: \"eb9b8831-8748-47c2-b77c-67a9e8e960d3\") "
Jul 14 14:39:33.735913 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:33.735802    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb9b8831-8748-47c2-b77c-67a9e8e960d3-openshift-service-ca\") pod \"eb9b8831-8748-47c2-b77c-67a9e8e960d3\" (UID: \"eb9b8831-8748-47c2-b77c-67a9e8e960d3\") "
Jul 14 14:39:33.736243 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:33.736196    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb9b8831-8748-47c2-b77c-67a9e8e960d3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "eb9b8831-8748-47c2-b77c-67a9e8e960d3" (UID: "eb9b8831-8748-47c2-b77c-67a9e8e960d3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 14:39:33.737907 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:33.737878    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb9b8831-8748-47c2-b77c-67a9e8e960d3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "eb9b8831-8748-47c2-b77c-67a9e8e960d3" (UID: "eb9b8831-8748-47c2-b77c-67a9e8e960d3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 14:39:33.738001 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:33.737906    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb9b8831-8748-47c2-b77c-67a9e8e960d3-kube-api-access-7wrl6" (OuterVolumeSpecName: "kube-api-access-7wrl6") pod "eb9b8831-8748-47c2-b77c-67a9e8e960d3" (UID: "eb9b8831-8748-47c2-b77c-67a9e8e960d3"). InnerVolumeSpecName "kube-api-access-7wrl6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 14:39:33.837367 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:33.837276    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb9b8831-8748-47c2-b77c-67a9e8e960d3-maas-api-tls\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:39:33.837367 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:33.837308    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb9b8831-8748-47c2-b77c-67a9e8e960d3-openshift-service-ca\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:39:33.837367 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:33.837318    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7wrl6\" (UniqueName: \"kubernetes.io/projected/eb9b8831-8748-47c2-b77c-67a9e8e960d3-kube-api-access-7wrl6\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:39:34.638894 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:34.638861    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8442f2-8448d45f8-xx4s8"
Jul 14 14:39:34.638894 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:34.638877    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8442f2-8448d45f8-xx4s8" event={"ID":"eb9b8831-8748-47c2-b77c-67a9e8e960d3","Type":"ContainerDied","Data":"8b137dcc6ad597e2caac90d183bf03f6790763d0508197099c68ddba8cdb1c96"}
Jul 14 14:39:34.639404 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:34.638943    2576 scope.go:117] "RemoveContainer" containerID="404d28779907b8867a4237e570ca5d9fe1bb7faa9a13ec60f5f94895696a1b63"
Jul 14 14:39:34.662550 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:34.662517    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8442f2-8448d45f8-xx4s8"]
Jul 14 14:39:34.667006 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:34.666974    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8442f2-8448d45f8-xx4s8"]
Jul 14 14:39:34.689041 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:34.689009    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb9b8831-8748-47c2-b77c-67a9e8e960d3" path="/var/lib/kubelet/pods/eb9b8831-8748-47c2-b77c-67a9e8e960d3/volumes"
Jul 14 14:39:36.813068 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:36.813031    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:39:38.403742 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:38.403709    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:39:44.496407 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:44.496367    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:39:44.597856 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:44.597808    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:39:48.853617 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:48.853560    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:39:49.121111 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:49.121007    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:39:49.786417 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:49.786381    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:39:49.886746 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:49.886696    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:39:59.717446 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:59.717403    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f8f81e-5f6f7bd897-j7v5c"]
Jul 14 14:39:59.718089 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:59.717734    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f8f81e-5f6f7bd897-j7v5c" podUID="53f9753d-c9a0-4e0a-9e5d-e5d268094b1c" containerName="maas-api" containerID="cri-o://f4a2735e5d5d60c5a3843cde728db6267c5005963bb5f9e42e6859c8ad0f6d2e" gracePeriod=30
Jul 14 14:39:59.975698 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:39:59.975630    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f8f81e-5f6f7bd897-j7v5c"
Jul 14 14:40:00.069625 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:00.069559    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53f9753d-c9a0-4e0a-9e5d-e5d268094b1c-openshift-service-ca\") pod \"53f9753d-c9a0-4e0a-9e5d-e5d268094b1c\" (UID: \"53f9753d-c9a0-4e0a-9e5d-e5d268094b1c\") "
Jul 14 14:40:00.069625 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:00.069633    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-29qsd\" (UniqueName: \"kubernetes.io/projected/53f9753d-c9a0-4e0a-9e5d-e5d268094b1c-kube-api-access-29qsd\") pod \"53f9753d-c9a0-4e0a-9e5d-e5d268094b1c\" (UID: \"53f9753d-c9a0-4e0a-9e5d-e5d268094b1c\") "
Jul 14 14:40:00.069871 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:00.069737    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53f9753d-c9a0-4e0a-9e5d-e5d268094b1c-maas-api-tls\") pod \"53f9753d-c9a0-4e0a-9e5d-e5d268094b1c\" (UID: \"53f9753d-c9a0-4e0a-9e5d-e5d268094b1c\") "
Jul 14 14:40:00.070021 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:00.069989    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53f9753d-c9a0-4e0a-9e5d-e5d268094b1c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "53f9753d-c9a0-4e0a-9e5d-e5d268094b1c" (UID: "53f9753d-c9a0-4e0a-9e5d-e5d268094b1c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 14:40:00.071857 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:00.071825    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53f9753d-c9a0-4e0a-9e5d-e5d268094b1c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "53f9753d-c9a0-4e0a-9e5d-e5d268094b1c" (UID: "53f9753d-c9a0-4e0a-9e5d-e5d268094b1c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 14:40:00.071955 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:00.071867    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53f9753d-c9a0-4e0a-9e5d-e5d268094b1c-kube-api-access-29qsd" (OuterVolumeSpecName: "kube-api-access-29qsd") pod "53f9753d-c9a0-4e0a-9e5d-e5d268094b1c" (UID: "53f9753d-c9a0-4e0a-9e5d-e5d268094b1c"). InnerVolumeSpecName "kube-api-access-29qsd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 14:40:00.170712 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:00.170670    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53f9753d-c9a0-4e0a-9e5d-e5d268094b1c-maas-api-tls\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:40:00.170712 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:00.170705    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53f9753d-c9a0-4e0a-9e5d-e5d268094b1c-openshift-service-ca\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:40:00.170712 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:00.170716    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-29qsd\" (UniqueName: \"kubernetes.io/projected/53f9753d-c9a0-4e0a-9e5d-e5d268094b1c-kube-api-access-29qsd\") on node \"ip-10-0-128-127.ec2.internal\" DevicePath \"\""
Jul 14 14:40:00.215855 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:00.215822    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:40:00.753679 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:00.753647    2576 generic.go:356] "Generic (PLEG): container finished" podID="53f9753d-c9a0-4e0a-9e5d-e5d268094b1c" containerID="f4a2735e5d5d60c5a3843cde728db6267c5005963bb5f9e42e6859c8ad0f6d2e" exitCode=0
Jul 14 14:40:00.754098 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:00.753694    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f8f81e-5f6f7bd897-j7v5c"
Jul 14 14:40:00.754098 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:00.753736    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f8f81e-5f6f7bd897-j7v5c" event={"ID":"53f9753d-c9a0-4e0a-9e5d-e5d268094b1c","Type":"ContainerDied","Data":"f4a2735e5d5d60c5a3843cde728db6267c5005963bb5f9e42e6859c8ad0f6d2e"}
Jul 14 14:40:00.754098 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:00.753790    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f8f81e-5f6f7bd897-j7v5c" event={"ID":"53f9753d-c9a0-4e0a-9e5d-e5d268094b1c","Type":"ContainerDied","Data":"2fc9865395f438dd37fe8ede6a91f4b0ff55c4931a62c1014be4e2d199969ffc"}
Jul 14 14:40:00.754098 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:00.753813    2576 scope.go:117] "RemoveContainer" containerID="f4a2735e5d5d60c5a3843cde728db6267c5005963bb5f9e42e6859c8ad0f6d2e"
Jul 14 14:40:00.762338 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:00.762315    2576 scope.go:117] "RemoveContainer" containerID="f4a2735e5d5d60c5a3843cde728db6267c5005963bb5f9e42e6859c8ad0f6d2e"
Jul 14 14:40:00.762699 ip-10-0-128-127 kubenswrapper[2576]: E0714 14:40:00.762672    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4a2735e5d5d60c5a3843cde728db6267c5005963bb5f9e42e6859c8ad0f6d2e\": container with ID starting with f4a2735e5d5d60c5a3843cde728db6267c5005963bb5f9e42e6859c8ad0f6d2e not found: ID does not exist" containerID="f4a2735e5d5d60c5a3843cde728db6267c5005963bb5f9e42e6859c8ad0f6d2e"
Jul 14 14:40:00.762809 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:00.762710    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4a2735e5d5d60c5a3843cde728db6267c5005963bb5f9e42e6859c8ad0f6d2e"} err="failed to get container status \"f4a2735e5d5d60c5a3843cde728db6267c5005963bb5f9e42e6859c8ad0f6d2e\": rpc error: code = NotFound desc = could not find container \"f4a2735e5d5d60c5a3843cde728db6267c5005963bb5f9e42e6859c8ad0f6d2e\": container with ID starting with f4a2735e5d5d60c5a3843cde728db6267c5005963bb5f9e42e6859c8ad0f6d2e not found: ID does not exist"
Jul 14 14:40:00.771395 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:00.771367    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f8f81e-5f6f7bd897-j7v5c"]
Jul 14 14:40:00.775376 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:00.775348    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f8f81e-5f6f7bd897-j7v5c"]
Jul 14 14:40:02.690200 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:02.690165    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53f9753d-c9a0-4e0a-9e5d-e5d268094b1c" path="/var/lib/kubelet/pods/53f9753d-c9a0-4e0a-9e5d-e5d268094b1c/volumes"
Jul 14 14:40:02.924489 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:02.924444    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dbjgd"]
Jul 14 14:40:04.385902 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:04.385858    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:40:11.712554 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:11.712511    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:40:11.820655 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:11.820615    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:40:15.266810 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:15.266765    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:40:15.511326 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:15.511285    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:40:15.941783 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:15.941747    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7b78d8d56f-z795f_1bc2547d-3d25-4140-b48b-5176097ca87a/authorino/0.log"
Jul 14 14:40:21.839902 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:21.839866    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-kg9mv_cd810af6-a109-428b-acfb-ecbcdf9634b5/manager/0.log"
Jul 14 14:40:22.702702 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:22.702669    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f858c8968-gfxlg_0e733524-eb39-472d-9f61-739555243347/copy-manifests/0.log"
Jul 14 14:40:22.722299 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:22.722241    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f858c8968-gfxlg_0e733524-eb39-472d-9f61-739555243347/manager/0.log"
Jul 14 14:40:26.541979 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:26.541950    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7b78d8d56f-z795f_1bc2547d-3d25-4140-b48b-5176097ca87a/authorino/0.log"
Jul 14 14:40:26.948153 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:26.948066    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-7lcrd_ada14027-8b6d-4738-b0fa-cfd2bb01480e/kuadrant-console-plugin/0.log"
Jul 14 14:40:27.386440 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:27.386411    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-dbjgd_41e392e0-c389-4901-9eae-78919fadc2a5/limitador/0.log"
Jul 14 14:40:28.325672 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:28.325643    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64945c8566-vbbg6_13fadc50-918b-4e8d-a4d4-b0381ee96030/kube-auth-proxy/0.log"
Jul 14 14:40:29.371949 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:29.371903    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:40:29.888815 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:29.888789    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-nvk4k_8934bfa2-4d59-4a28-ab17-b0feeaad4cda/main/0.log"
Jul 14 14:40:34.248202 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:34.248116    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-flfn5/must-gather-gwfrk"]
Jul 14 14:40:34.248730 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:34.248546    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53f9753d-c9a0-4e0a-9e5d-e5d268094b1c" containerName="maas-api"
Jul 14 14:40:34.248730 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:34.248566    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="53f9753d-c9a0-4e0a-9e5d-e5d268094b1c" containerName="maas-api"
Jul 14 14:40:34.248730 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:34.248585    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91f8f107-d43f-4ac6-9ef4-d4e4eb42391d" containerName="main"
Jul 14 14:40:34.248730 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:34.248606    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="91f8f107-d43f-4ac6-9ef4-d4e4eb42391d" containerName="main"
Jul 14 14:40:34.248730 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:34.248544    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:40:34.248730 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:34.248636    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb9b8831-8748-47c2-b77c-67a9e8e960d3" containerName="maas-api"
Jul 14 14:40:34.248730 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:34.248685    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb9b8831-8748-47c2-b77c-67a9e8e960d3" containerName="maas-api"
Jul 14 14:40:34.248994 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:34.248785    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="91f8f107-d43f-4ac6-9ef4-d4e4eb42391d" containerName="main"
Jul 14 14:40:34.248994 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:34.248796    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb9b8831-8748-47c2-b77c-67a9e8e960d3" containerName="maas-api"
Jul 14 14:40:34.248994 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:34.248801    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="53f9753d-c9a0-4e0a-9e5d-e5d268094b1c" containerName="maas-api"
Jul 14 14:40:34.251987 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:34.251970    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-flfn5/must-gather-gwfrk"
Jul 14 14:40:34.254616 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:34.254562    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-flfn5\"/\"kube-root-ca.crt\""
Jul 14 14:40:34.255297 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:34.255278    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-flfn5\"/\"openshift-service-ca.crt\""
Jul 14 14:40:34.255414 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:34.255298    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-flfn5\"/\"default-dockercfg-6s2zw\""
Jul 14 14:40:34.260649 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:34.260619    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-flfn5/must-gather-gwfrk"]
Jul 14 14:40:34.359578 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:34.359543    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b1a2b3c6-e98e-495b-bca0-84f5885ba156-must-gather-output\") pod \"must-gather-gwfrk\" (UID: \"b1a2b3c6-e98e-495b-bca0-84f5885ba156\") " pod="openshift-must-gather-flfn5/must-gather-gwfrk"
Jul 14 14:40:34.359810 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:34.359625    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p59vr\" (UniqueName: \"kubernetes.io/projected/b1a2b3c6-e98e-495b-bca0-84f5885ba156-kube-api-access-p59vr\") pod \"must-gather-gwfrk\" (UID: \"b1a2b3c6-e98e-495b-bca0-84f5885ba156\") " pod="openshift-must-gather-flfn5/must-gather-gwfrk"
Jul 14 14:40:34.359810 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:34.359662    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log" limit=1024
Jul 14 14:40:34.461038 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:34.460998    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b1a2b3c6-e98e-495b-bca0-84f5885ba156-must-gather-output\") pod \"must-gather-gwfrk\" (UID: \"b1a2b3c6-e98e-495b-bca0-84f5885ba156\") " pod="openshift-must-gather-flfn5/must-gather-gwfrk"
Jul 14 14:40:34.461038 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:34.461039    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p59vr\" (UniqueName: \"kubernetes.io/projected/b1a2b3c6-e98e-495b-bca0-84f5885ba156-kube-api-access-p59vr\") pod \"must-gather-gwfrk\" (UID: \"b1a2b3c6-e98e-495b-bca0-84f5885ba156\") " pod="openshift-must-gather-flfn5/must-gather-gwfrk"
Jul 14 14:40:34.461422 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:34.461401    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b1a2b3c6-e98e-495b-bca0-84f5885ba156-must-gather-output\") pod \"must-gather-gwfrk\" (UID: \"b1a2b3c6-e98e-495b-bca0-84f5885ba156\") " pod="openshift-must-gather-flfn5/must-gather-gwfrk"
Jul 14 14:40:34.474190 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:34.474154    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p59vr\" (UniqueName: \"kubernetes.io/projected/b1a2b3c6-e98e-495b-bca0-84f5885ba156-kube-api-access-p59vr\") pod \"must-gather-gwfrk\" (UID: \"b1a2b3c6-e98e-495b-bca0-84f5885ba156\") " pod="openshift-must-gather-flfn5/must-gather-gwfrk"
Jul 14 14:40:34.562214 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:34.562180    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-flfn5/must-gather-gwfrk"
Jul 14 14:40:34.699678 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:34.699617    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-flfn5/must-gather-gwfrk"]
Jul 14 14:40:34.702456 ip-10-0-128-127 kubenswrapper[2576]: W0714 14:40:34.702425    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb1a2b3c6_e98e_495b_bca0_84f5885ba156.slice/crio-6c9c37f3fe39b92f6b1987e38279cf1fb5e024c1b82e19458dfcd20380f99af3 WatchSource:0}: Error finding container 6c9c37f3fe39b92f6b1987e38279cf1fb5e024c1b82e19458dfcd20380f99af3: Status 404 returned error can't find the container with id 6c9c37f3fe39b92f6b1987e38279cf1fb5e024c1b82e19458dfcd20380f99af3
Jul 14 14:40:34.704399 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:34.704379    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 14:40:34.910980 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:34.910884    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-flfn5/must-gather-gwfrk" event={"ID":"b1a2b3c6-e98e-495b-bca0-84f5885ba156","Type":"ContainerStarted","Data":"6c9c37f3fe39b92f6b1987e38279cf1fb5e024c1b82e19458dfcd20380f99af3"}
Jul 14 14:40:35.919993 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:35.919071    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-flfn5/must-gather-gwfrk" event={"ID":"b1a2b3c6-e98e-495b-bca0-84f5885ba156","Type":"ContainerStarted","Data":"652cf74bb46c503d1c88f7456bce9e6ea63a7cd7ab48df1205e73219f3bcb1ca"}
Jul 14 14:40:35.919993 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:35.919128    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-flfn5/must-gather-gwfrk" event={"ID":"b1a2b3c6-e98e-495b-bca0-84f5885ba156","Type":"ContainerStarted","Data":"09d867e81490f40c7f758e6bb301673039557b88bdf45f8def4281acf34a0dc7"}
Jul 14 14:40:35.939897 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:35.939832    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-flfn5/must-gather-gwfrk" podStartSLOduration=1.126095731 podStartE2EDuration="1.939806399s" totalImagesPullingTime="813.710668ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:40:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:40:35.9367166 +0000 UTC m=+2783.787154106" watchObservedRunningTime="2026-07-14 14:40:35.939806399 +0000 UTC m=+2783.790243903"
Jul 14 14:40:37.288973 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:37.288937    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-5m6xn_02b53875-f2af-42e5-a311-74b8301e8b62/global-pull-secret-syncer/0.log"
Jul 14 14:40:37.398230 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:37.398191    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-65w26_b23d0d9c-a401-4327-9167-db15cbf7095e/konnectivity-agent/0.log"
Jul 14 14:40:37.487520 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:37.487487    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-127.ec2.internal_8c0fbe03dcb71398a547458685400965/haproxy/0.log"
Jul 14 14:40:41.972398 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:41.971825    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7b78d8d56f-z795f_1bc2547d-3d25-4140-b48b-5176097ca87a/authorino/0.log"
Jul 14 14:40:42.096188 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:42.096091    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-7lcrd_ada14027-8b6d-4738-b0fa-cfd2bb01480e/kuadrant-console-plugin/0.log"
Jul 14 14:40:42.311089 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:42.311055    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-dbjgd_41e392e0-c389-4901-9eae-78919fadc2a5/limitador/0.log"
Jul 14 14:40:44.393893 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:44.393855    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-6jlxv_470ac83d-59aa-470a-8107-d15cf5e77a92/kube-state-metrics/0.log"
Jul 14 14:40:44.417394 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:44.417360    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-6jlxv_470ac83d-59aa-470a-8107-d15cf5e77a92/kube-rbac-proxy-main/0.log"
Jul 14 14:40:44.444616 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:44.443957    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-6jlxv_470ac83d-59aa-470a-8107-d15cf5e77a92/kube-rbac-proxy-self/0.log"
Jul 14 14:40:44.630001 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:44.629967    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-59gw6_e04c79d0-6500-4b8a-925e-d9aed17efbf6/node-exporter/0.log"
Jul 14 14:40:44.664936 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:44.664846    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-59gw6_e04c79d0-6500-4b8a-925e-d9aed17efbf6/kube-rbac-proxy/0.log"
Jul 14 14:40:44.696204 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:44.696117    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-59gw6_e04c79d0-6500-4b8a-925e-d9aed17efbf6/init-textfile/0.log"
Jul 14 14:40:44.947487 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:44.947384    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1bf6c916-ce8f-4617-8e78-6b6fce3b4657/prometheus/0.log"
Jul 14 14:40:44.975347 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:44.975323    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1bf6c916-ce8f-4617-8e78-6b6fce3b4657/config-reloader/0.log"
Jul 14 14:40:45.005762 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:45.005729    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1bf6c916-ce8f-4617-8e78-6b6fce3b4657/thanos-sidecar/0.log"
Jul 14 14:40:45.039737 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:45.039711    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1bf6c916-ce8f-4617-8e78-6b6fce3b4657/kube-rbac-proxy-web/0.log"
Jul 14 14:40:45.078946 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:45.078921    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1bf6c916-ce8f-4617-8e78-6b6fce3b4657/kube-rbac-proxy/0.log"
Jul 14 14:40:45.118466 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:45.118437    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1bf6c916-ce8f-4617-8e78-6b6fce3b4657/kube-rbac-proxy-thanos/0.log"
Jul 14 14:40:45.150542 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:45.150511    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1bf6c916-ce8f-4617-8e78-6b6fce3b4657/init-config-reloader/0.log"
Jul 14 14:40:45.366646 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:45.366610    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-676c66bd44-g5zp9_f170ae29-26b7-4bbe-8de9-787f8de9617f/thanos-query/0.log"
Jul 14 14:40:45.393337 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:45.393302    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-676c66bd44-g5zp9_f170ae29-26b7-4bbe-8de9-787f8de9617f/kube-rbac-proxy-web/0.log"
Jul 14 14:40:45.420388 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:45.420326    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-676c66bd44-g5zp9_f170ae29-26b7-4bbe-8de9-787f8de9617f/kube-rbac-proxy/0.log"
Jul 14 14:40:45.448133 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:45.448112    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-676c66bd44-g5zp9_f170ae29-26b7-4bbe-8de9-787f8de9617f/prom-label-proxy/0.log"
Jul 14 14:40:45.479485 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:45.479303    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-676c66bd44-g5zp9_f170ae29-26b7-4bbe-8de9-787f8de9617f/kube-rbac-proxy-rules/0.log"
Jul 14 14:40:45.509785 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:45.509697    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-676c66bd44-g5zp9_f170ae29-26b7-4bbe-8de9-787f8de9617f/kube-rbac-proxy-metrics/0.log"
Jul 14 14:40:47.618257 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:47.618219    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-flfn5/perf-node-gather-daemonset-zn4xs"]
Jul 14 14:40:47.623813 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:47.623785    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zn4xs"
Jul 14 14:40:47.637954 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:47.636465    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-flfn5/perf-node-gather-daemonset-zn4xs"]
Jul 14 14:40:47.718998 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:47.718958    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/db5b22e2-fd85-4834-ab2b-e6da02be1e3e-sys\") pod \"perf-node-gather-daemonset-zn4xs\" (UID: \"db5b22e2-fd85-4834-ab2b-e6da02be1e3e\") " pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zn4xs"
Jul 14 14:40:47.718998 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:47.718998    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/db5b22e2-fd85-4834-ab2b-e6da02be1e3e-lib-modules\") pod \"perf-node-gather-daemonset-zn4xs\" (UID: \"db5b22e2-fd85-4834-ab2b-e6da02be1e3e\") " pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zn4xs"
Jul 14 14:40:47.719274 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:47.719043    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/db5b22e2-fd85-4834-ab2b-e6da02be1e3e-podres\") pod \"perf-node-gather-daemonset-zn4xs\" (UID: \"db5b22e2-fd85-4834-ab2b-e6da02be1e3e\") " pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zn4xs"
Jul 14 14:40:47.719274 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:47.719076    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zrkp\" (UniqueName: \"kubernetes.io/projected/db5b22e2-fd85-4834-ab2b-e6da02be1e3e-kube-api-access-5zrkp\") pod \"perf-node-gather-daemonset-zn4xs\" (UID: \"db5b22e2-fd85-4834-ab2b-e6da02be1e3e\") " pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zn4xs"
Jul 14 14:40:47.719274 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:47.719107    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/db5b22e2-fd85-4834-ab2b-e6da02be1e3e-proc\") pod \"perf-node-gather-daemonset-zn4xs\" (UID: \"db5b22e2-fd85-4834-ab2b-e6da02be1e3e\") " pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zn4xs"
Jul 14 14:40:47.820480 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:47.820433    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/db5b22e2-fd85-4834-ab2b-e6da02be1e3e-sys\") pod \"perf-node-gather-daemonset-zn4xs\" (UID: \"db5b22e2-fd85-4834-ab2b-e6da02be1e3e\") " pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zn4xs"
Jul 14 14:40:47.820480 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:47.820483    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/db5b22e2-fd85-4834-ab2b-e6da02be1e3e-lib-modules\") pod \"perf-node-gather-daemonset-zn4xs\" (UID: \"db5b22e2-fd85-4834-ab2b-e6da02be1e3e\") " pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zn4xs"
Jul 14 14:40:47.820707 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:47.820659    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/db5b22e2-fd85-4834-ab2b-e6da02be1e3e-lib-modules\") pod \"perf-node-gather-daemonset-zn4xs\" (UID: \"db5b22e2-fd85-4834-ab2b-e6da02be1e3e\") " pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zn4xs"
Jul 14 14:40:47.820707 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:47.820658    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/db5b22e2-fd85-4834-ab2b-e6da02be1e3e-podres\") pod \"perf-node-gather-daemonset-zn4xs\" (UID: \"db5b22e2-fd85-4834-ab2b-e6da02be1e3e\") " pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zn4xs"
Jul 14 14:40:47.820776 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:47.820732    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zrkp\" (UniqueName: \"kubernetes.io/projected/db5b22e2-fd85-4834-ab2b-e6da02be1e3e-kube-api-access-5zrkp\") pod \"perf-node-gather-daemonset-zn4xs\" (UID: \"db5b22e2-fd85-4834-ab2b-e6da02be1e3e\") " pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zn4xs"
Jul 14 14:40:47.820776 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:47.820745    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/db5b22e2-fd85-4834-ab2b-e6da02be1e3e-podres\") pod \"perf-node-gather-daemonset-zn4xs\" (UID: \"db5b22e2-fd85-4834-ab2b-e6da02be1e3e\") " pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zn4xs"
Jul 14 14:40:47.820844 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:47.820780    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/db5b22e2-fd85-4834-ab2b-e6da02be1e3e-proc\") pod \"perf-node-gather-daemonset-zn4xs\" (UID: \"db5b22e2-fd85-4834-ab2b-e6da02be1e3e\") " pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zn4xs"
Jul 14 14:40:47.820910 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:47.820895    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/db5b22e2-fd85-4834-ab2b-e6da02be1e3e-proc\") pod \"perf-node-gather-daemonset-zn4xs\" (UID: \"db5b22e2-fd85-4834-ab2b-e6da02be1e3e\") " pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zn4xs"
Jul 14 14:40:47.821046 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:47.821024    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/db5b22e2-fd85-4834-ab2b-e6da02be1e3e-sys\") pod \"perf-node-gather-daemonset-zn4xs\" (UID: \"db5b22e2-fd85-4834-ab2b-e6da02be1e3e\") " pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zn4xs"
Jul 14 14:40:47.831851 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:47.831811    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zrkp\" (UniqueName: \"kubernetes.io/projected/db5b22e2-fd85-4834-ab2b-e6da02be1e3e-kube-api-access-5zrkp\") pod \"perf-node-gather-daemonset-zn4xs\" (UID: \"db5b22e2-fd85-4834-ab2b-e6da02be1e3e\") " pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zn4xs"
Jul 14 14:40:47.939846 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:47.939733    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zn4xs"
Jul 14 14:40:48.139839 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:48.139798    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-flfn5/perf-node-gather-daemonset-zn4xs"]
Jul 14 14:40:48.148552 ip-10-0-128-127 kubenswrapper[2576]: W0714 14:40:48.148513    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poddb5b22e2_fd85_4834_ab2b_e6da02be1e3e.slice/crio-ca9b8c76bb9e12eedb00faff2f9a4f5ffb086e33bd77736240a473e48c093472 WatchSource:0}: Error finding container ca9b8c76bb9e12eedb00faff2f9a4f5ffb086e33bd77736240a473e48c093472: Status 404 returned error can't find the container with id ca9b8c76bb9e12eedb00faff2f9a4f5ffb086e33bd77736240a473e48c093472
Jul 14 14:40:49.013575 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:49.013441    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zn4xs" event={"ID":"db5b22e2-fd85-4834-ab2b-e6da02be1e3e","Type":"ContainerStarted","Data":"d84e9f52ef55b0c30b2d4c4b75ea9365cc71060e5edee1d8fe06cf41a2ec865d"}
Jul 14 14:40:49.013575 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:49.013510    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zn4xs" event={"ID":"db5b22e2-fd85-4834-ab2b-e6da02be1e3e","Type":"ContainerStarted","Data":"ca9b8c76bb9e12eedb00faff2f9a4f5ffb086e33bd77736240a473e48c093472"}
Jul 14 14:40:49.013575 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:49.013538    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zn4xs"
Jul 14 14:40:49.032734 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:49.032670    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zn4xs" podStartSLOduration=2.032646164 podStartE2EDuration="2.032646164s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 14:40:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:40:49.032315853 +0000 UTC m=+2796.882753360" watchObservedRunningTime="2026-07-14 14:40:49.032646164 +0000 UTC m=+2796.883083668"
Jul 14 14:40:50.285657 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:50.285625    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hhmmt_f6561da1-26ed-4edb-b397-4ce7a197fc07/dns/0.log"
Jul 14 14:40:50.313876 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:50.313851    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hhmmt_f6561da1-26ed-4edb-b397-4ce7a197fc07/kube-rbac-proxy/0.log"
Jul 14 14:40:50.469470 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:50.469444    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jfpgf_5a5c1fc8-edd8-4cb9-8afb-81970bbd10c6/dns-node-resolver/0.log"
Jul 14 14:40:51.030828 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:51.030792    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6d9fd7cf6c-pk2t8_886aaf44-5bd7-4262-bda7-f622787aeb6c/registry/0.log"
Jul 14 14:40:51.057531 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:51.057502    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5pjcr_8a9c4e5c-fb07-4089-9d9c-f01896cd6e25/node-ca/0.log"
Jul 14 14:40:52.267349 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:52.267304    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64945c8566-vbbg6_13fadc50-918b-4e8d-a4d4-b0381ee96030/kube-auth-proxy/0.log"
Jul 14 14:40:53.153518 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:53.153479    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vvtxj_f2371b29-9e56-469a-ac86-7ead875c7127/serve-healthcheck-canary/0.log"
Jul 14 14:40:53.780087 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:53.780059    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-q6rx4_62788698-f167-42d8-b1e9-cc61cca44012/kube-rbac-proxy/0.log"
Jul 14 14:40:53.802718 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:53.802688    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-q6rx4_62788698-f167-42d8-b1e9-cc61cca44012/exporter/0.log"
Jul 14 14:40:53.824864 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:53.824826    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-q6rx4_62788698-f167-42d8-b1e9-cc61cca44012/extractor/0.log"
Jul 14 14:40:55.030388 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:55.030353    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zn4xs"
Jul 14 14:40:56.373885 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:56.373854    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-kg9mv_cd810af6-a109-428b-acfb-ecbcdf9634b5/manager/0.log"
Jul 14 14:40:56.810117 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:56.810078    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f858c8968-gfxlg_0e733524-eb39-472d-9f61-739555243347/manager/0.log"
Jul 14 14:40:56.830082 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:56.830055    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f858c8968-gfxlg_0e733524-eb39-472d-9f61-739555243347/copy-manifests/0.log"
Jul 14 14:40:58.223462 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:58.223362    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-d76b86dd9-7gp6n_201cbcaf-0b08-445d-a706-2b5f7da85021/manager/0.log"
Jul 14 14:40:58.279471 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:58.279429    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-88lc4_5e76c961-6669-4694-8bd1-9e1d85dc0027/openshift-lws-operator/0.log"
Jul 14 14:40:58.558391 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:40:58.558354    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64945c8566-vbbg6_13fadc50-918b-4e8d-a4d4-b0381ee96030/kube-auth-proxy/0.log"
Jul 14 14:41:00.113723 ip-10-0-128-127 kubenswrapper[2576]: I0714 14:41:00.113684    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64945c8566-vbbg6_13fadc50-918b-4e8d-a4d4-b0381ee96030/kube-auth-proxy/0.log"