Jul 20 15:24:46.367731 ip-10-0-130-19 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 20 15:24:46.367743 ip-10-0-130-19 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 20 15:24:46.367752 ip-10-0-130-19 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 20 15:24:46.368140 ip-10-0-130-19 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 20 15:24:57.476207 ip-10-0-130-19 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 20 15:24:57.476222 ip-10-0-130-19 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 243d20b637dc4e84a85e41ae28d72d9f --
Jul 20 15:26:56.878580 ip-10-0-130-19 systemd[1]: Starting Kubernetes Kubelet...
Jul 20 15:26:57.255496 ip-10-0-130-19 kubenswrapper[2559]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 20 15:26:57.255496 ip-10-0-130-19 kubenswrapper[2559]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 20 15:26:57.255496 ip-10-0-130-19 kubenswrapper[2559]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 20 15:26:57.255496 ip-10-0-130-19 kubenswrapper[2559]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 20 15:26:57.255496 ip-10-0-130-19 kubenswrapper[2559]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 20 15:26:57.256156 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.255555    2559 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 20 15:26:57.260985 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.260968    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 15:26:57.260985 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.260984    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 15:26:57.261048 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.260988    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 15:26:57.261048 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.260992    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 15:26:57.261048 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.260995    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 15:26:57.261048 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.260998    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 15:26:57.261048 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261001    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 15:26:57.261048 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261004    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 15:26:57.261048 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261009    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 15:26:57.261048 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261013    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 15:26:57.261048 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261024    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 15:26:57.261048 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261027    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 15:26:57.261048 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261030    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 15:26:57.261048 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261033    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 15:26:57.261048 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261035    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 15:26:57.261048 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261038    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 15:26:57.261048 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261041    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 15:26:57.261048 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261044    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 15:26:57.261048 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261046    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 15:26:57.261048 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261050    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 15:26:57.261048 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261054    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 15:26:57.261510 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261058    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 15:26:57.261510 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261061    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 15:26:57.261510 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261066    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 15:26:57.261510 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261068    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 15:26:57.261510 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261071    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 15:26:57.261510 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261074    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 15:26:57.261510 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261076    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 15:26:57.261510 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261079    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 15:26:57.261510 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261081    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 15:26:57.261510 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261084    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 15:26:57.261510 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261086    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 15:26:57.261510 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261089    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 15:26:57.261510 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261091    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 15:26:57.261510 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261094    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 15:26:57.261510 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261096    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 15:26:57.261510 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261099    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 15:26:57.261510 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261101    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 15:26:57.261510 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261104    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 15:26:57.261510 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261106    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 15:26:57.261510 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261109    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 15:26:57.261977 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261112    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 15:26:57.261977 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261114    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 15:26:57.261977 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261117    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 15:26:57.261977 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261120    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 15:26:57.261977 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261122    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 15:26:57.261977 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261124    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 15:26:57.261977 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261127    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 15:26:57.261977 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261129    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 15:26:57.261977 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261132    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 15:26:57.261977 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261134    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 15:26:57.261977 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261136    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 15:26:57.261977 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261140    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 15:26:57.261977 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261142    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 15:26:57.261977 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261147    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 15:26:57.261977 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261150    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 15:26:57.261977 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261153    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 15:26:57.261977 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261156    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 15:26:57.261977 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261159    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 15:26:57.261977 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261162    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 15:26:57.262431 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261164    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 15:26:57.262431 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261167    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 15:26:57.262431 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261169    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 15:26:57.262431 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261172    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 15:26:57.262431 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261174    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 15:26:57.262431 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261177    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 15:26:57.262431 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261179    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 15:26:57.262431 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261182    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 15:26:57.262431 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261184    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 15:26:57.262431 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261187    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 15:26:57.262431 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261189    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 15:26:57.262431 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261192    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 15:26:57.262431 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261195    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 15:26:57.262431 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261197    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 15:26:57.262431 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261200    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 15:26:57.262431 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261204    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 15:26:57.262431 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261207    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 15:26:57.262431 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261209    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 15:26:57.262431 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261211    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 15:26:57.262431 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261214    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 15:26:57.262895 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261217    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 15:26:57.262895 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261219    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 15:26:57.262895 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261222    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 15:26:57.262895 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261224    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 15:26:57.262895 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261227    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 15:26:57.262895 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261230    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 15:26:57.262895 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261236    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 15:26:57.262895 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261239    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 15:26:57.262895 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261241    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 15:26:57.262895 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261244    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 15:26:57.262895 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261247    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 15:26:57.262895 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261250    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 15:26:57.262895 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261252    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 15:26:57.262895 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261255    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 15:26:57.262895 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261258    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 15:26:57.262895 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261260    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 15:26:57.262895 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261263    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 15:26:57.262895 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261265    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 15:26:57.262895 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.261267    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 15:26:57.262895 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.262889    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 15:26:57.263407 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.262896    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 15:26:57.263407 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.262900    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 15:26:57.263407 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.262903    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 15:26:57.263407 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.262906    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 15:26:57.263407 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.262911    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 15:26:57.263407 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.262913    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 15:26:57.263407 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.262916    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 15:26:57.263407 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.262919    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 15:26:57.263407 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.262921    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 15:26:57.263407 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.262924    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 15:26:57.263407 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.262927    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 15:26:57.263407 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.262929    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 15:26:57.263407 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.262932    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 15:26:57.263407 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.262934    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 15:26:57.263407 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.262937    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 15:26:57.263407 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.262939    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 15:26:57.263407 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.262942    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 15:26:57.263407 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.262944    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 15:26:57.263407 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.262947    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 15:26:57.263407 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.262949    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 15:26:57.263893 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.262952    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 15:26:57.263893 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.262954    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 15:26:57.263893 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.262957    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 15:26:57.263893 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.262959    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 15:26:57.263893 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.262962    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 15:26:57.263893 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.262964    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 15:26:57.263893 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.262966    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 15:26:57.263893 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.262969    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 15:26:57.263893 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.262971    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 15:26:57.263893 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.262974    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 15:26:57.263893 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.262976    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 15:26:57.263893 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.262979    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 15:26:57.263893 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.262981    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 15:26:57.263893 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.262985    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 15:26:57.263893 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.262988    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 15:26:57.263893 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.262991    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 15:26:57.263893 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.262993    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 15:26:57.263893 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.262996    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 15:26:57.263893 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.262999    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 15:26:57.263893 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263001    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 15:26:57.264369 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263003    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 15:26:57.264369 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263006    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 15:26:57.264369 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263008    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 15:26:57.264369 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263011    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 15:26:57.264369 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263013    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 15:26:57.264369 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263016    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 15:26:57.264369 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263019    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 15:26:57.264369 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263021    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 15:26:57.264369 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263024    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 15:26:57.264369 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263026    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 15:26:57.264369 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263029    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 15:26:57.264369 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263031    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 15:26:57.264369 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263034    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 15:26:57.264369 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263036    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 15:26:57.264369 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263039    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 15:26:57.264369 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263041    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 15:26:57.264369 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263045    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 15:26:57.264369 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263047    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 15:26:57.264369 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263050    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 15:26:57.264369 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263052    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 15:26:57.264835 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263055    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 15:26:57.264835 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263058    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 15:26:57.264835 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263060    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 15:26:57.264835 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263063    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 15:26:57.264835 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263065    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 15:26:57.264835 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263068    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 15:26:57.264835 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263072    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 15:26:57.264835 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263076    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 15:26:57.264835 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263079    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 15:26:57.264835 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263081    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 15:26:57.264835 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263084    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 15:26:57.264835 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263087    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 15:26:57.264835 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263089    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 15:26:57.264835 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263092    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 15:26:57.264835 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263094    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 15:26:57.264835 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263097    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 15:26:57.264835 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263099    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 15:26:57.264835 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263102    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 15:26:57.264835 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263106    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 15:26:57.265297 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263109    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 15:26:57.265297 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263113    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 15:26:57.265297 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263116    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 15:26:57.265297 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263119    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 15:26:57.265297 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263122    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 15:26:57.265297 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263125    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 15:26:57.265297 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263129    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 15:26:57.265297 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263132    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 15:26:57.265297 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263134    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 15:26:57.265297 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263137    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 15:26:57.265297 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263140    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 15:26:57.265297 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263143    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 15:26:57.265297 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263145    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 15:26:57.265297 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263148    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 15:26:57.265297 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263151    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 15:26:57.265297 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263153    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 15:26:57.265297 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263156    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 15:26:57.265297 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263159    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 15:26:57.265297 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263161    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 15:26:57.265744 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263589    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 15:26:57.265744 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263595    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 15:26:57.265744 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263598    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 15:26:57.265744 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263600    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 15:26:57.265744 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263604    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 15:26:57.265744 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263607    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 15:26:57.265744 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263609    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 15:26:57.265744 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263612    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 15:26:57.265744 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263615    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 15:26:57.265744 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263619    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 15:26:57.265744 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263623    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 15:26:57.265744 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263625    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 15:26:57.265744 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263628    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 15:26:57.265744 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263631    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 15:26:57.265744 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263634    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 15:26:57.265744 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263637    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 15:26:57.265744 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263639    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 15:26:57.265744 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263642    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 15:26:57.265744 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263645    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 15:26:57.265744 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263648    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 15:26:57.266210 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263650    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 15:26:57.266210 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263653    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 15:26:57.266210 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263656    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 15:26:57.266210 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263658    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 15:26:57.266210 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263661    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 15:26:57.266210 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263666    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 15:26:57.266210 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263669    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 15:26:57.266210 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263672    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 15:26:57.266210 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263675    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 15:26:57.266210 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263678    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 15:26:57.266210 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263681    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 15:26:57.266210 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263684    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 15:26:57.266210 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263686    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 15:26:57.266210 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263689    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 15:26:57.266210 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263692    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 15:26:57.266210 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263694    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 15:26:57.266210 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263697    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 15:26:57.266210 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263700    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 15:26:57.266210 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263702    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 15:26:57.266210 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263708    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 15:26:57.266695 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263710    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 15:26:57.266695 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263713    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 15:26:57.266695 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263715    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 15:26:57.266695 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263718    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 15:26:57.266695 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263720    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 15:26:57.266695 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263723    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 15:26:57.266695 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263725    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 15:26:57.266695 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263728    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 15:26:57.266695 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263730    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 15:26:57.266695 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263733    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 15:26:57.266695 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263735    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 15:26:57.266695 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263738    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 15:26:57.266695 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263740    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 15:26:57.266695 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263742    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 15:26:57.266695 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263753    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 15:26:57.266695 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263756    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 15:26:57.266695 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263759    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 15:26:57.266695 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263762    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 15:26:57.266695 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263765    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 15:26:57.266695 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263768    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 15:26:57.267179 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263770    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 15:26:57.267179 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263773    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 15:26:57.267179 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263775    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 15:26:57.267179 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263779    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 15:26:57.267179 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263782    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 15:26:57.267179 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263785    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 15:26:57.267179 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263788    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 15:26:57.267179 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263791    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 15:26:57.267179 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263793    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 15:26:57.267179 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263796    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 15:26:57.267179 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263799    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 15:26:57.267179 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263803    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 15:26:57.267179 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263806    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 15:26:57.267179 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263808    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 15:26:57.267179 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263811    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 15:26:57.267179 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263813    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 15:26:57.267179 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263816    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 15:26:57.267179 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263818    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 15:26:57.267179 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263821    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 15:26:57.267179 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263823    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 15:26:57.267660 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263826    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 15:26:57.267660 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263828    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 15:26:57.267660 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263831    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 15:26:57.267660 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263833    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 15:26:57.267660 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263836    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 15:26:57.267660 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263840    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 15:26:57.267660 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263843    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 15:26:57.267660 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263845    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 15:26:57.267660 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263848    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 15:26:57.267660 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263850    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 15:26:57.267660 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263853    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 15:26:57.267660 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263855    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 15:26:57.267660 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263858    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 15:26:57.267660 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263860    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 15:26:57.267660 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263862    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 15:26:57.267660 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263865    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 15:26:57.267660 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263868    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 15:26:57.267660 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263870    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 15:26:57.267660 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.263873    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 15:26:57.267660 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.263968    2559 flags.go:64] FLAG: --address="0.0.0.0"
Jul 20 15:26:57.268179 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.263979    2559 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 20 15:26:57.268179 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.263988    2559 flags.go:64] FLAG: --anonymous-auth="true"
Jul 20 15:26:57.268179 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.263994    2559 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 20 15:26:57.268179 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264001    2559 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 20 15:26:57.268179 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264005    2559 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 20 15:26:57.268179 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264011    2559 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 20 15:26:57.268179 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264016    2559 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 20 15:26:57.268179 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264019    2559 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 20 15:26:57.268179 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264022    2559 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 20 15:26:57.268179 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264025    2559 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 20 15:26:57.268179 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264029    2559 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 20 15:26:57.268179 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264032    2559 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 20 15:26:57.268179 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264035    2559 flags.go:64] FLAG: --cgroup-root=""
Jul 20 15:26:57.268179 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264037    2559 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 20 15:26:57.268179 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264040    2559 flags.go:64] FLAG: --client-ca-file=""
Jul 20 15:26:57.268179 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264043    2559 flags.go:64] FLAG: --cloud-provider="external"
Jul 20 15:26:57.268179 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264046    2559 flags.go:64] FLAG: --cluster-dns="[]"
Jul 20 15:26:57.268179 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264054    2559 flags.go:64] FLAG: --cluster-domain=""
Jul 20 15:26:57.268179 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264057    2559 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 20 15:26:57.268179 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264060    2559 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 20 15:26:57.268179 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264063    2559 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 20 15:26:57.268179 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264066    2559 flags.go:64] FLAG: --container-log-max-files="5"
Jul 20 15:26:57.268179 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264070    2559 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 20 15:26:57.268179 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264073    2559 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 20 15:26:57.268853 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264076    2559 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 20 15:26:57.268853 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264079    2559 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 20 15:26:57.268853 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264082    2559 flags.go:64] FLAG: --contention-profiling="false"
Jul 20 15:26:57.268853 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264085    2559 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 20 15:26:57.268853 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264088    2559 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 20 15:26:57.268853 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264091    2559 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 20 15:26:57.268853 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264094    2559 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 20 15:26:57.268853 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264098    2559 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 20 15:26:57.268853 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264101    2559 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 20 15:26:57.268853 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264104    2559 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 20 15:26:57.268853 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264106    2559 flags.go:64] FLAG: --enable-load-reader="false"
Jul 20 15:26:57.268853 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264111    2559 flags.go:64] FLAG: --enable-server="true"
Jul 20 15:26:57.268853 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264114    2559 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 20 15:26:57.268853 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264118    2559 flags.go:64] FLAG: --event-burst="100"
Jul 20 15:26:57.268853 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264121    2559 flags.go:64] FLAG: --event-qps="50"
Jul 20 15:26:57.268853 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264124    2559 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 20 15:26:57.268853 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264127    2559 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 20 15:26:57.268853 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264130    2559 flags.go:64] FLAG: --eviction-hard=""
Jul 20 15:26:57.268853 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264133    2559 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 20 15:26:57.268853 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264136    2559 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 20 15:26:57.268853 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264139    2559 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 20 15:26:57.268853 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264142    2559 flags.go:64] FLAG: --eviction-soft=""
Jul 20 15:26:57.268853 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264145    2559 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 20 15:26:57.268853 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264147    2559 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 20 15:26:57.268853 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264150    2559 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 20 15:26:57.269469 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264154    2559 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 20 15:26:57.269469 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264157    2559 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 20 15:26:57.269469 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264160    2559 flags.go:64] FLAG: --fail-swap-on="true"
Jul 20 15:26:57.269469 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264163    2559 flags.go:64] FLAG: --feature-gates=""
Jul 20 15:26:57.269469 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264167    2559 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 20 15:26:57.269469 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264169    2559 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 20 15:26:57.269469 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264172    2559 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 20 15:26:57.269469 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264175    2559 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 20 15:26:57.269469 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264178    2559 flags.go:64] FLAG: --healthz-port="10248"
Jul 20 15:26:57.269469 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264181    2559 flags.go:64] FLAG: --help="false"
Jul 20 15:26:57.269469 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264184    2559 flags.go:64] FLAG: --hostname-override="ip-10-0-130-19.ec2.internal"
Jul 20 15:26:57.269469 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264187    2559 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 20 15:26:57.269469 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264190    2559 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 20 15:26:57.269469 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264193    2559 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 20 15:26:57.269469 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264196    2559 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 20 15:26:57.269469 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264200    2559 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 20 15:26:57.269469 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264203    2559 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 20 15:26:57.269469 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264205    2559 flags.go:64] FLAG: --image-service-endpoint=""
Jul 20 15:26:57.269469 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264209    2559 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 20 15:26:57.269469 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264212    2559 flags.go:64] FLAG: --kube-api-burst="100"
Jul 20 15:26:57.269469 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264216    2559 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 20 15:26:57.269469 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264219    2559 flags.go:64] FLAG: --kube-api-qps="50"
Jul 20 15:26:57.269469 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264222    2559 flags.go:64] FLAG: --kube-reserved=""
Jul 20 15:26:57.269469 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264224    2559 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 20 15:26:57.270045 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264227    2559 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 20 15:26:57.270045 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264230    2559 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 20 15:26:57.270045 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264233    2559 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 20 15:26:57.270045 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264235    2559 flags.go:64] FLAG: --lock-file=""
Jul 20 15:26:57.270045 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264238    2559 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 20 15:26:57.270045 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264241    2559 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 20 15:26:57.270045 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264244    2559 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 20 15:26:57.270045 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264249    2559 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 20 15:26:57.270045 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264253    2559 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 20 15:26:57.270045 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264256    2559 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 20 15:26:57.270045 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264259    2559 flags.go:64] FLAG: --logging-format="text"
Jul 20 15:26:57.270045 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264261    2559 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 20 15:26:57.270045 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264265    2559 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 20 15:26:57.270045 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264268    2559 flags.go:64] FLAG: --manifest-url=""
Jul 20 15:26:57.270045 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264271    2559 flags.go:64] FLAG: --manifest-url-header=""
Jul 20 15:26:57.270045 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264275    2559 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 20 15:26:57.270045 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264278    2559 flags.go:64] FLAG: --max-open-files="1000000"
Jul 20 15:26:57.270045 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264282    2559 flags.go:64] FLAG: --max-pods="110"
Jul 20 15:26:57.270045 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264298    2559 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 20 15:26:57.270045 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264301    2559 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 20 15:26:57.270045 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264304    2559 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 20 15:26:57.270045 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264307    2559 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 20 15:26:57.270045 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264310    2559 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 20 15:26:57.270045 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264313    2559 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 20 15:26:57.270045 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264316    2559 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 20 15:26:57.270923 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264324    2559 flags.go:64] FLAG: --node-status-max-images="50"
Jul 20 15:26:57.270923 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264328    2559 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 20 15:26:57.270923 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264331    2559 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 20 15:26:57.270923 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264334    2559 flags.go:64] FLAG: --pod-cidr=""
Jul 20 15:26:57.270923 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264337    2559 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 20 15:26:57.270923 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264342    2559 flags.go:64] FLAG: --pod-manifest-path=""
Jul 20 15:26:57.270923 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264345    2559 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 20 15:26:57.270923 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264348    2559 flags.go:64] FLAG: --pods-per-core="0"
Jul 20 15:26:57.270923 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264351    2559 flags.go:64] FLAG: --port="10250"
Jul 20 15:26:57.270923 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264354    2559 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 20 15:26:57.270923 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264357    2559 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0479f479826a852cf"
Jul 20 15:26:57.270923 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264360    2559 flags.go:64] FLAG: --qos-reserved=""
Jul 20 15:26:57.270923 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264363    2559 flags.go:64] FLAG: --read-only-port="10255"
Jul 20 15:26:57.270923 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264366    2559 flags.go:64] FLAG: --register-node="true"
Jul 20 15:26:57.270923 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264369    2559 flags.go:64] FLAG: --register-with-taints=""
Jul 20 15:26:57.270923 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264372    2559 flags.go:64] FLAG: --registry-burst="10"
Jul 20 15:26:57.270923 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264376    2559 flags.go:64] FLAG: --registry-qps="5"
Jul 20 15:26:57.270923 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264379    2559 flags.go:64] FLAG: --reserved-cpus=""
Jul 20 15:26:57.270923 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264382    2559 flags.go:64] FLAG: --reserved-memory=""
Jul 20 15:26:57.270923 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264386    2559 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 20 15:26:57.270923 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264389    2559 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 20 15:26:57.270923 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264392    2559 flags.go:64] FLAG: --rotate-certificates="false"
Jul 20 15:26:57.270923 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264395    2559 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 20 15:26:57.270923 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264398    2559 flags.go:64] FLAG: --runonce="false"
Jul 20 15:26:57.270923 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264401    2559 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 20 15:26:57.272007 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264404    2559 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 20 15:26:57.272007 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264407    2559 flags.go:64] FLAG: --seccomp-default="false"
Jul 20 15:26:57.272007 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264409    2559 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 20 15:26:57.272007 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264412    2559 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 20 15:26:57.272007 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264415    2559 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 20 15:26:57.272007 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264418    2559 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 20 15:26:57.272007 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264421    2559 flags.go:64] FLAG: --storage-driver-password="root"
Jul 20 15:26:57.272007 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264424    2559 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 20 15:26:57.272007 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264428    2559 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 20 15:26:57.272007 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264430    2559 flags.go:64] FLAG: --storage-driver-user="root"
Jul 20 15:26:57.272007 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264433    2559 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 20 15:26:57.272007 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264436    2559 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 20 15:26:57.272007 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264439    2559 flags.go:64] FLAG: --system-cgroups=""
Jul 20 15:26:57.272007 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264442    2559 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 20 15:26:57.272007 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264447    2559 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 20 15:26:57.272007 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264450    2559 flags.go:64] FLAG: --tls-cert-file=""
Jul 20 15:26:57.272007 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264452    2559 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 20 15:26:57.272007 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264457    2559 flags.go:64] FLAG: --tls-min-version=""
Jul 20 15:26:57.272007 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264459    2559 flags.go:64] FLAG: --tls-private-key-file=""
Jul 20 15:26:57.272007 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264462    2559 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 20 15:26:57.272007 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264465    2559 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 20 15:26:57.272007 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264468    2559 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 20 15:26:57.272007 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264470    2559 flags.go:64] FLAG: --v="2"
Jul 20 15:26:57.272007 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264476    2559 flags.go:64] FLAG: --version="false"
Jul 20 15:26:57.272007 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264480    2559 flags.go:64] FLAG: --vmodule=""
Jul 20 15:26:57.273110 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264484    2559 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 20 15:26:57.273110 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264487    2559 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 20 15:26:57.273110 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264604    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 15:26:57.273110 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264607    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 15:26:57.273110 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264611    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 15:26:57.273110 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264614    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 15:26:57.273110 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264617    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 15:26:57.273110 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264619    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 15:26:57.273110 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264622    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 15:26:57.273110 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264625    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 15:26:57.273110 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264629    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 15:26:57.273110 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264632    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 15:26:57.273110 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264635    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 15:26:57.273110 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264638    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 15:26:57.273110 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264640    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 15:26:57.273110 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264643    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 15:26:57.273110 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264646    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 15:26:57.273110 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264648    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 15:26:57.273110 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264650    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 15:26:57.273981 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264653    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 15:26:57.273981 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264656    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 15:26:57.273981 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264659    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 15:26:57.273981 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264661    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 15:26:57.273981 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264664    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 15:26:57.273981 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264666    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 15:26:57.273981 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264669    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 15:26:57.273981 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264671    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 15:26:57.273981 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264674    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 15:26:57.273981 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264676    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 15:26:57.273981 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264679    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 15:26:57.273981 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264683    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 15:26:57.273981 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264686    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 15:26:57.273981 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264688    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 15:26:57.273981 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264690    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 15:26:57.273981 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264693    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 15:26:57.273981 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264695    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 15:26:57.273981 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264698    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 15:26:57.273981 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264700    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 15:26:57.273981 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264703    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 15:26:57.274582 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264705    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 15:26:57.274582 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264708    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 15:26:57.274582 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264711    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 15:26:57.274582 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264714    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 15:26:57.274582 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264716    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 15:26:57.274582 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264718    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 15:26:57.274582 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264721    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 15:26:57.274582 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264724    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 15:26:57.274582 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264726    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 15:26:57.274582 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264730    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 15:26:57.274582 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264733    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 15:26:57.274582 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264736    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 15:26:57.274582 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264739    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 15:26:57.274582 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264742    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 15:26:57.274582 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264746    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 15:26:57.274582 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264748    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 15:26:57.274582 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264752    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 15:26:57.274582 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264754    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 15:26:57.274582 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264757    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 15:26:57.274582 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264760    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 15:26:57.275228 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264762    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 15:26:57.275228 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264764    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 15:26:57.275228 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264767    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 15:26:57.275228 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264771    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 15:26:57.275228 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264774    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 15:26:57.275228 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264777    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 15:26:57.275228 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264779    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 15:26:57.275228 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264782    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 15:26:57.275228 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264784    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 15:26:57.275228 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264787    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 15:26:57.275228 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264789    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 15:26:57.275228 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264792    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 15:26:57.275228 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264795    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 15:26:57.275228 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264798    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 15:26:57.275228 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264802    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 15:26:57.275228 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264804    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 15:26:57.275228 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264807    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 15:26:57.275228 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264809    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 15:26:57.275228 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264812    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 15:26:57.275722 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264815    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 15:26:57.275722 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264817    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 15:26:57.275722 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264819    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 15:26:57.275722 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264822    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 15:26:57.275722 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264824    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 15:26:57.275722 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264827    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 15:26:57.275722 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264829    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 15:26:57.275722 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264831    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 15:26:57.275722 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264834    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 15:26:57.275722 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264837    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 15:26:57.275722 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264840    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 15:26:57.275722 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264842    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 15:26:57.275722 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264845    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 15:26:57.275722 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264847    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 15:26:57.275722 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264850    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 15:26:57.275722 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264852    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 15:26:57.275722 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264855    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 15:26:57.275722 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264859    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 15:26:57.275722 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264861    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 15:26:57.275722 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264864    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 15:26:57.276377 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264867    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 15:26:57.276377 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264869    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 15:26:57.276377 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.264872    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 15:26:57.276377 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.264880    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 20 15:26:57.276377 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.272645    2559 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 20 15:26:57.276377 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.272663    2559 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 20 15:26:57.276377 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.272687    2559 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 20 15:26:57.276377 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.272697    2559 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 20 15:26:57.276377 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272781    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 15:26:57.276377 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272789    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 15:26:57.276377 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272795    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 15:26:57.276377 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272800    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 15:26:57.276377 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272805    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 15:26:57.276377 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272810    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 15:26:57.276377 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272814    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 15:26:57.276983 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272818    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 15:26:57.276983 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272825    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 15:26:57.276983 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272831    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 15:26:57.276983 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272836    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 15:26:57.276983 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272841    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 15:26:57.276983 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272846    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 15:26:57.276983 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272851    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 15:26:57.276983 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272856    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 15:26:57.276983 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272860    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 15:26:57.276983 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272865    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 15:26:57.276983 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272869    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 15:26:57.276983 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272873    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 15:26:57.276983 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272877    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 15:26:57.276983 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272881    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 15:26:57.276983 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272885    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 15:26:57.276983 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272892    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 15:26:57.276983 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272897    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 15:26:57.276983 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272902    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 15:26:57.277689 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272907    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 15:26:57.277689 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272912    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 15:26:57.277689 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272917    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 15:26:57.277689 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272922    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 15:26:57.277689 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272928    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 15:26:57.277689 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272933    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 15:26:57.277689 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272938    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 15:26:57.277689 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272943    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 15:26:57.277689 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272947    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 15:26:57.277689 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272952    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 15:26:57.277689 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272957    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 15:26:57.277689 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272963    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 15:26:57.277689 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272969    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 15:26:57.277689 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272974    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 15:26:57.277689 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272978    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 15:26:57.277689 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272982    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 15:26:57.277689 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272986    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 15:26:57.277689 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272991    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 15:26:57.277689 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.272996    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 15:26:57.277689 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273000    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 15:26:57.278213 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273004    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 15:26:57.278213 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273008    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 15:26:57.278213 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273012    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 15:26:57.278213 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273016    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 15:26:57.278213 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273021    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 15:26:57.278213 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273024    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 15:26:57.278213 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273028    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 15:26:57.278213 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273033    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 15:26:57.278213 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273037    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 15:26:57.278213 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273042    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 15:26:57.278213 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273046    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 15:26:57.278213 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273050    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 15:26:57.278213 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273054    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 15:26:57.278213 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273058    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 15:26:57.278213 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273063    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 15:26:57.278213 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273066    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 15:26:57.278213 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273071    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 15:26:57.278213 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273077    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 15:26:57.278213 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273081    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 15:26:57.278213 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273085    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 15:26:57.278823 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273089    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 15:26:57.278823 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273093    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 15:26:57.278823 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273097    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 15:26:57.278823 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273101    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 15:26:57.278823 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273106    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 15:26:57.278823 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273110    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 15:26:57.278823 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273114    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 15:26:57.278823 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273117    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 15:26:57.278823 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273122    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 15:26:57.278823 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273125    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 15:26:57.278823 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273130    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 15:26:57.278823 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273134    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 15:26:57.278823 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273138    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 15:26:57.278823 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273143    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 15:26:57.278823 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273147    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 15:26:57.278823 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273152    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 15:26:57.278823 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273155    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 15:26:57.278823 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273160    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 15:26:57.278823 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273164    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 15:26:57.278823 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273168    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 15:26:57.279325 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273172    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 15:26:57.279325 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273176    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 15:26:57.279325 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273181    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 15:26:57.279325 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273185    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 15:26:57.279325 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273189    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 15:26:57.279325 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273193    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 15:26:57.279325 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273198    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 15:26:57.279325 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273202    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 15:26:57.279325 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273207    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 15:26:57.279325 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273212    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 15:26:57.279325 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273216    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 15:26:57.279325 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273220    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 15:26:57.279325 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273224    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 15:26:57.279325 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273228    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 15:26:57.279325 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.273237    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 20 15:26:57.279708 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273430    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 15:26:57.279708 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273441    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 15:26:57.279708 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273446    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 15:26:57.279708 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273451    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 15:26:57.279708 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273456    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 15:26:57.279708 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273460    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 15:26:57.279708 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273465    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 15:26:57.279708 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273469    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 15:26:57.279708 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273473    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 15:26:57.279708 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273477    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 15:26:57.279708 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273483    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 15:26:57.279708 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273487    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 15:26:57.279708 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273492    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 15:26:57.279708 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273496    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 15:26:57.279708 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273500    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 15:26:57.279708 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273504    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 15:26:57.279708 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273509    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 15:26:57.279708 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273513    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 15:26:57.279708 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273517    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 15:26:57.279708 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273521    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 15:26:57.280188 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273525    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 15:26:57.280188 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273529    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 15:26:57.280188 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273533    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 15:26:57.280188 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273538    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 15:26:57.280188 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273542    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 15:26:57.280188 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273546    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 15:26:57.280188 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273551    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 15:26:57.280188 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273555    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 15:26:57.280188 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273560    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 15:26:57.280188 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273564    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 15:26:57.280188 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273568    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 15:26:57.280188 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273572    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 15:26:57.280188 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273576    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 15:26:57.280188 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273580    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 15:26:57.280188 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273585    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 15:26:57.280188 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273589    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 15:26:57.280188 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273593    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 15:26:57.280188 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273597    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 15:26:57.280188 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273603    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 15:26:57.280188 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273609    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 15:26:57.280706 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273614    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 15:26:57.280706 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273619    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 15:26:57.280706 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273623    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 15:26:57.280706 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273628    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 15:26:57.280706 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273632    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 15:26:57.280706 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273636    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 15:26:57.280706 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273640    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 15:26:57.280706 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273644    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 15:26:57.280706 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273649    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 15:26:57.280706 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273653    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 15:26:57.280706 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273657    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 15:26:57.280706 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273663    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 15:26:57.280706 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273669    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 15:26:57.280706 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273673    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 15:26:57.280706 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273678    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 15:26:57.280706 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273682    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 15:26:57.280706 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273686    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 15:26:57.280706 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273691    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 15:26:57.280706 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273695    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 15:26:57.280706 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273699    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 15:26:57.281173 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273703    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 15:26:57.281173 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273710    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 15:26:57.281173 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273715    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 15:26:57.281173 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273720    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 15:26:57.281173 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273724    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 15:26:57.281173 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273728    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 15:26:57.281173 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273732    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 15:26:57.281173 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273736    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 15:26:57.281173 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273740    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 15:26:57.281173 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273745    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 15:26:57.281173 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273749    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 15:26:57.281173 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273753    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 15:26:57.281173 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273757    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 15:26:57.281173 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273761    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 15:26:57.281173 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273765    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 15:26:57.281173 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273769    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 15:26:57.281173 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273773    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 15:26:57.281173 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273777    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 15:26:57.281173 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273782    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 15:26:57.281636 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273787    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 15:26:57.281636 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273791    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 15:26:57.281636 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273796    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 15:26:57.281636 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273800    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 15:26:57.281636 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273804    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 15:26:57.281636 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273808    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 15:26:57.281636 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273812    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 15:26:57.281636 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273816    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 15:26:57.281636 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273820    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 15:26:57.281636 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273824    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 15:26:57.281636 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273828    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 15:26:57.281636 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273833    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 15:26:57.281636 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273837    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 15:26:57.281636 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273841    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 15:26:57.281636 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273845    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 15:26:57.281636 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273850    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 15:26:57.281636 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273854    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 15:26:57.281636 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273858    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 15:26:57.281636 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273862    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 15:26:57.282070 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:57.273866    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 15:26:57.282070 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.273874    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 20 15:26:57.282070 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.274037    2559 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 20 15:26:57.282070 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.279319    2559 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 20 15:26:57.282070 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.281300    2559 server.go:1019] "Starting client certificate rotation"
Jul 20 15:26:57.282070 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.281397    2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 20 15:26:57.282230 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.282130    2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 20 15:26:57.283756 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.283735    2559 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 20 15:26:57.284852 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.284836    2559 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 20 15:26:57.293086 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.293070    2559 log.go:25] "Validated CRI v1 runtime API"
Jul 20 15:26:57.297527 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.297513    2559 log.go:25] "Validated CRI v1 image API"
Jul 20 15:26:57.298352 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.298335    2559 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 20 15:26:57.301894 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.301876    2559 fs.go:135] Filesystem UUIDs: map[683ee934-37c4-4987-a6fe-1a3055efc39f:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 879a3f37-0980-4696-b56e-f83c51fb4344:/dev/nvme0n1p3]
Jul 20 15:26:57.301954 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.301893    2559 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 20 15:26:57.306973 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.306853    2559 manager.go:216] Machine: {Timestamp:2026-07-20 15:26:57.305586661 +0000 UTC m=+0.315208314 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099317 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20953e5b295c2fbff249ff003e7fa8 SystemUUID:ec20953e-5b29-5c2f-bff2-49ff003e7fa8 BootID:243d20b6-37dc-4e84-a85e-41ae28d72d9f Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c5:23:b1:70:35 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c5:23:b1:70:35 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:52:a1:2d:90:2c:bb Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 20 15:26:57.306973 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.306973    2559 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 20 15:26:57.307084 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.307052    2559 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 20 15:26:57.309509 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.309485    2559 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 20 15:26:57.309658 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.309511    2559 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-19.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 20 15:26:57.310993 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.310983    2559 topology_manager.go:138] "Creating topology manager with none policy"
Jul 20 15:26:57.311032 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.310995    2559 container_manager_linux.go:309] "Creating device plugin manager"
Jul 20 15:26:57.311032 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.311002    2559 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 20 15:26:57.311610 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.311594    2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 20 15:26:57.311684 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.311674    2559 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 20 15:26:57.313352 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.313342    2559 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 20 15:26:57.317899 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.317888    2559 state_mem.go:36] "Initialized new in-memory state store"
Jul 20 15:26:57.318009 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.318000    2559 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 20 15:26:57.318038 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.318027    2559 kubelet.go:486] "Attempting to sync node with API server"
Jul 20 15:26:57.318074 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.318043    2559 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 20 15:26:57.318074 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.318060    2559 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 20 15:26:57.318074 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.318069    2559 kubelet.go:398] "Adding apiserver pod source"
Jul 20 15:26:57.318146 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.318127    2559 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 20 15:26:57.318839 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.318823    2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 20 15:26:57.318907 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.318853    2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 20 15:26:57.323083 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.323063    2559 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 20 15:26:57.325467 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.325450    2559 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 20 15:26:57.325561 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.325482    2559 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 20 15:26:57.326920 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.326904    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 20 15:26:57.326920 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.326920    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 20 15:26:57.326999 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.326926    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 20 15:26:57.326999 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.326931    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 20 15:26:57.326999 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.326940    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 20 15:26:57.326999 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.326945    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 20 15:26:57.326999 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.326954    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 20 15:26:57.326999 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.326959    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 20 15:26:57.326999 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.326966    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 20 15:26:57.326999 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.326971    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 20 15:26:57.326999 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.326980    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 20 15:26:57.326999 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.326985    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 20 15:26:57.328864 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.328853    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 20 15:26:57.328864 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.328864    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 20 15:26:57.331909 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.331891    2559 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-19.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 20 15:26:57.332126 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:26:57.332092    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-19.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 20 15:26:57.332126 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:26:57.332105    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 20 15:26:57.332237 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.332141    2559 server.go:1268] "Started kubelet"
Jul 20 15:26:57.332281 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.332233    2559 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 20 15:26:57.332734 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.332667    2559 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 20 15:26:57.332818 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.332756    2559 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 20 15:26:57.332958 ip-10-0-130-19 systemd[1]: Started Kubernetes Kubelet.
Jul 20 15:26:57.334036 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.334009    2559 server.go:310] "Adding debug handlers to kubelet server"
Jul 20 15:26:57.334902 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.334887    2559 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 20 15:26:57.337111 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.337091    2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rh2mm"
Jul 20 15:26:57.337888 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:26:57.337874    2559 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 20 15:26:57.342870 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:26:57.342050    2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-19.ec2.internal.18c408f8df0275b9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-19.ec2.internal,UID:ip-10-0-130-19.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-19.ec2.internal,},FirstTimestamp:2026-07-20 15:26:57.331951033 +0000 UTC m=+0.341572744,LastTimestamp:2026-07-20 15:26:57.331951033 +0000 UTC m=+0.341572744,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-19.ec2.internal,}"
Jul 20 15:26:57.342970 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.342951    2559 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 20 15:26:57.343053 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.342954    2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 20 15:26:57.343053 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:26:57.343034    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-19.ec2.internal\" not found"
Jul 20 15:26:57.344652 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.343892    2559 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 20 15:26:57.344751 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.344738    2559 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 20 15:26:57.345251 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.345232    2559 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 20 15:26:57.345518 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.345506    2559 reconstruct.go:97] "Volume reconstruction finished"
Jul 20 15:26:57.345597 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.345588    2559 reconciler.go:29] "Reconciler: start to sync state"
Jul 20 15:26:57.345908 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.345890    2559 factory.go:55] Registering systemd factory
Jul 20 15:26:57.345908 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.345910    2559 factory.go:223] Registration of the systemd container factory successfully
Jul 20 15:26:57.346161 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.346147    2559 factory.go:153] Registering CRI-O factory
Jul 20 15:26:57.346235 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.346176    2559 factory.go:223] Registration of the crio container factory successfully
Jul 20 15:26:57.346308 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.346240    2559 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 20 15:26:57.346308 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.346273    2559 factory.go:103] Registering Raw factory
Jul 20 15:26:57.346396 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.346328    2559 manager.go:1196] Started watching for new ooms in manager
Jul 20 15:26:57.346807 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.346785    2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rh2mm"
Jul 20 15:26:57.346894 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.346880    2559 manager.go:349] Starting recovery of all containers
Jul 20 15:26:57.347686 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:26:57.347662    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 20 15:26:57.347832 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:26:57.347793    2559 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-19.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 20 15:26:57.359070 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.359054    2559 manager.go:354] Recovery completed
Jul 20 15:26:57.362795 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.362783    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 15:26:57.364494 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.364480    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-19.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 15:26:57.364542 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.364507    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-19.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 15:26:57.364542 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.364519    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-19.ec2.internal" event="NodeHasSufficientPID"
Jul 20 15:26:57.364987 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.364974    2559 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 20 15:26:57.364987 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.364985    2559 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 20 15:26:57.365056 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.364998    2559 state_mem.go:36] "Initialized new in-memory state store"
Jul 20 15:26:57.365056 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.364978    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 15:26:57.365619 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.365605    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-19.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 15:26:57.365696 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.365635    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-19.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 15:26:57.365696 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.365649    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-19.ec2.internal" event="NodeHasSufficientPID"
Jul 20 15:26:57.366881 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.366866    2559 policy_none.go:49] "None policy: Start"
Jul 20 15:26:57.366934 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.366884    2559 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 20 15:26:57.366934 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.366894    2559 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 20 15:26:57.367015 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:26:57.366896    2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-19.ec2.internal.18c408f8e0f309f4  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-19.ec2.internal,UID:ip-10-0-130-19.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-19.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-19.ec2.internal,},FirstTimestamp:2026-07-20 15:26:57.364494836 +0000 UTC m=+0.374116486,LastTimestamp:2026-07-20 15:26:57.364494836 +0000 UTC m=+0.374116486,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-19.ec2.internal,}"
Jul 20 15:26:57.368704 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.368693    2559 policy_none.go:47] "Start"
Jul 20 15:26:57.406817 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.406795    2559 manager.go:337] "Starting Device Plugin manager"
Jul 20 15:26:57.406925 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:26:57.406830    2559 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 20 15:26:57.406925 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.406843    2559 server.go:87] "Starting device plugin registration server"
Jul 20 15:26:57.407100 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.407087    2559 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 20 15:26:57.407315 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.407103    2559 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 20 15:26:57.407315 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.407194    2559 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 20 15:26:57.407315 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.407303    2559 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 20 15:26:57.407315 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.407312    2559 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 20 15:26:57.407798 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:26:57.407779    2559 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 20 15:26:57.407852 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:26:57.407824    2559 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-19.ec2.internal\" not found"
Jul 20 15:26:57.507044 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.506965    2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 20 15:26:57.508892 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.507321    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 15:26:57.508892 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.508046    2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 20 15:26:57.508892 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.508076    2559 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 20 15:26:57.508892 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.508096    2559 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 20 15:26:57.508892 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:26:57.508128    2559 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 20 15:26:57.508892 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.508264    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-19.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 15:26:57.508892 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.508307    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-19.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 15:26:57.508892 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.508318    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-19.ec2.internal" event="NodeHasSufficientPID"
Jul 20 15:26:57.508892 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.508341    2559 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-19.ec2.internal"
Jul 20 15:26:57.509767 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.509754    2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 15:26:57.513166 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.513149    2559 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-19.ec2.internal"
Jul 20 15:26:57.513240 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:26:57.513171    2559 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-19.ec2.internal\": node \"ip-10-0-130-19.ec2.internal\" not found"
Jul 20 15:26:57.577871 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:26:57.577844    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-19.ec2.internal\" not found"
Jul 20 15:26:57.608949 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.608910    2559 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-19.ec2.internal"]
Jul 20 15:26:57.609027 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.608993    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 15:26:57.609880 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.609865    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-19.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 15:26:57.609971 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.609889    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-19.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 15:26:57.609971 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.609905    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-19.ec2.internal" event="NodeHasSufficientPID"
Jul 20 15:26:57.612054 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.612041    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 15:26:57.612157 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.612145    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.ec2.internal"
Jul 20 15:26:57.612193 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.612176    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 15:26:57.612695 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.612680    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-19.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 15:26:57.612695 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.612698    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-19.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 15:26:57.612803 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.612718    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-19.ec2.internal" event="NodeHasSufficientPID"
Jul 20 15:26:57.612803 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.612726    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-19.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 15:26:57.612803 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.612745    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-19.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 15:26:57.612803 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.612758    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-19.ec2.internal" event="NodeHasSufficientPID"
Jul 20 15:26:57.614610 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.614594    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-19.ec2.internal"
Jul 20 15:26:57.614690 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.614618    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 15:26:57.615213 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.615199    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-19.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 15:26:57.615274 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.615224    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-19.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 15:26:57.615274 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.615235    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-19.ec2.internal" event="NodeHasSufficientPID"
Jul 20 15:26:57.638810 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:26:57.638791    2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-19.ec2.internal\" not found" node="ip-10-0-130-19.ec2.internal"
Jul 20 15:26:57.642716 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:26:57.642702    2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-19.ec2.internal\" not found" node="ip-10-0-130-19.ec2.internal"
Jul 20 15:26:57.647088 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.647073    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9120b8f40b5d9a133728742e2ee8b96d-config\") pod \"kube-apiserver-proxy-ip-10-0-130-19.ec2.internal\" (UID: \"9120b8f40b5d9a133728742e2ee8b96d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-19.ec2.internal"
Jul 20 15:26:57.647165 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.647094    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/efe68e62a819b5c98670e409a64453e3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-19.ec2.internal\" (UID: \"efe68e62a819b5c98670e409a64453e3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.ec2.internal"
Jul 20 15:26:57.647165 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.647108    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/efe68e62a819b5c98670e409a64453e3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-19.ec2.internal\" (UID: \"efe68e62a819b5c98670e409a64453e3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.ec2.internal"
Jul 20 15:26:57.678436 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:26:57.678409    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-19.ec2.internal\" not found"
Jul 20 15:26:57.747469 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.747446    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/efe68e62a819b5c98670e409a64453e3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-19.ec2.internal\" (UID: \"efe68e62a819b5c98670e409a64453e3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.ec2.internal"
Jul 20 15:26:57.747628 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.747477    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9120b8f40b5d9a133728742e2ee8b96d-config\") pod \"kube-apiserver-proxy-ip-10-0-130-19.ec2.internal\" (UID: \"9120b8f40b5d9a133728742e2ee8b96d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-19.ec2.internal"
Jul 20 15:26:57.747628 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.747504    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/efe68e62a819b5c98670e409a64453e3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-19.ec2.internal\" (UID: \"efe68e62a819b5c98670e409a64453e3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.ec2.internal"
Jul 20 15:26:57.747628 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.747531    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/efe68e62a819b5c98670e409a64453e3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-19.ec2.internal\" (UID: \"efe68e62a819b5c98670e409a64453e3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.ec2.internal"
Jul 20 15:26:57.747628 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.747532    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/efe68e62a819b5c98670e409a64453e3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-19.ec2.internal\" (UID: \"efe68e62a819b5c98670e409a64453e3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.ec2.internal"
Jul 20 15:26:57.747628 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.747534    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9120b8f40b5d9a133728742e2ee8b96d-config\") pod \"kube-apiserver-proxy-ip-10-0-130-19.ec2.internal\" (UID: \"9120b8f40b5d9a133728742e2ee8b96d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-19.ec2.internal"
Jul 20 15:26:57.779259 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:26:57.779208    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-19.ec2.internal\" not found"
Jul 20 15:26:57.880259 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:26:57.880229    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-19.ec2.internal\" not found"
Jul 20 15:26:57.942511 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.942356    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.ec2.internal"
Jul 20 15:26:57.945956 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:57.945940    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-19.ec2.internal"
Jul 20 15:26:57.981320 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:26:57.981302    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-19.ec2.internal\" not found"
Jul 20 15:26:58.081755 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:26:58.081695    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-19.ec2.internal\" not found"
Jul 20 15:26:58.160053 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:58.160034    2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 15:26:58.182494 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:26:58.182474    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-19.ec2.internal\" not found"
Jul 20 15:26:58.280008 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:58.279986    2559 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 20 15:26:58.280390 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:58.280090    2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 20 15:26:58.280390 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:58.280096    2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 20 15:26:58.283104 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:26:58.283088    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-19.ec2.internal\" not found"
Jul 20 15:26:58.352511 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:58.352453    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-19 15:21:57 +0000 UTC" deadline="2028-01-17 00:25:14.167292196 +0000 UTC"
Jul 20 15:26:58.352511 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:58.352477    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13088h58m15.814817681s"
Jul 20 15:26:58.383582 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:26:58.383560    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-19.ec2.internal\" not found"
Jul 20 15:26:58.412492 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:58.412469    2559 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 15:26:58.476219 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:58.475460    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefe68e62a819b5c98670e409a64453e3.slice/crio-e2f4ddd8e68545e15992513a1a9c2c5d0a1f26a375956b4d0a52d1db320ace52 WatchSource:0}: Error finding container e2f4ddd8e68545e15992513a1a9c2c5d0a1f26a375956b4d0a52d1db320ace52: Status 404 returned error can't find the container with id e2f4ddd8e68545e15992513a1a9c2c5d0a1f26a375956b4d0a52d1db320ace52
Jul 20 15:26:58.476219 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:26:58.475707    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9120b8f40b5d9a133728742e2ee8b96d.slice/crio-e53ad88c8c7d49966b5c2b3fe3486efc6c93b095166210f9ca1ed16e25900cbc WatchSource:0}: Error finding container e53ad88c8c7d49966b5c2b3fe3486efc6c93b095166210f9ca1ed16e25900cbc: Status 404 returned error can't find the container with id e53ad88c8c7d49966b5c2b3fe3486efc6c93b095166210f9ca1ed16e25900cbc
Jul 20 15:26:58.480345 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:58.480330    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 15:26:58.483807 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:26:58.483791    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-19.ec2.internal\" not found"
Jul 20 15:26:58.511594 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:58.511559    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-19.ec2.internal" event={"ID":"9120b8f40b5d9a133728742e2ee8b96d","Type":"ContainerStarted","Data":"e53ad88c8c7d49966b5c2b3fe3486efc6c93b095166210f9ca1ed16e25900cbc"}
Jul 20 15:26:58.512387 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:58.512369    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.ec2.internal" event={"ID":"efe68e62a819b5c98670e409a64453e3","Type":"ContainerStarted","Data":"e2f4ddd8e68545e15992513a1a9c2c5d0a1f26a375956b4d0a52d1db320ace52"}
Jul 20 15:26:58.584836 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:26:58.584812    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-19.ec2.internal\" not found"
Jul 20 15:26:58.685172 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:26:58.685120    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-19.ec2.internal\" not found"
Jul 20 15:26:58.758000 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:58.757977    2559 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 15:26:58.843956 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:58.843928    2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.ec2.internal"
Jul 20 15:26:58.858800 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:58.858779    2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 20 15:26:58.859670 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:58.859658    2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-19.ec2.internal"
Jul 20 15:26:58.867316 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:58.867277    2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 20 15:26:59.180649 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.180566    2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 15:26:59.319554 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.319346    2559 apiserver.go:52] "Watching apiserver"
Jul 20 15:26:59.329299 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.329268    2559 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 20 15:26:59.330408 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.330384    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-7hgpz","kube-system/global-pull-secret-syncer-t54tw","kube-system/konnectivity-agent-2m28f","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zqfwk","openshift-dns/node-resolver-j45gb","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.ec2.internal","openshift-multus/multus-additional-cni-plugins-gpzpc","openshift-multus/network-metrics-daemon-4kzlg","openshift-network-diagnostics/network-check-target-n7pqm","kube-system/kube-apiserver-proxy-ip-10-0-130-19.ec2.internal","openshift-cluster-node-tuning-operator/tuned-9nxcn","openshift-image-registry/node-ca-pqqrs","openshift-multus/multus-tx286","openshift-network-operator/iptables-alerter-t2ckp"]
Jul 20 15:26:59.332894 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.332873    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gpzpc"
Jul 20 15:26:59.335805 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.335783    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bhg65\""
Jul 20 15:26:59.336550 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.336532    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 20 15:26:59.337026 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.337006    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.339427 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.339408    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2m28f"
Jul 20 15:26:59.341721 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.341355    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zqfwk"
Jul 20 15:26:59.342085 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.342060    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 20 15:26:59.343818 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.343801    2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 20 15:26:59.344730 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.344176    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 20 15:26:59.344730 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.344387    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-j45gb"
Jul 20 15:26:59.344730 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.344580    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 20 15:26:59.344730 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.344614    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 20 15:26:59.345055 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.344741    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 20 15:26:59.345055 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.344750    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 20 15:26:59.345895 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.345406    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 20 15:26:59.345895 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.345448    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-qv5vg\""
Jul 20 15:26:59.345895 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.345517    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-sqlwx\""
Jul 20 15:26:59.345895 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.345603    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 20 15:26:59.345895 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.345605    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 20 15:26:59.345895 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.345757    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-mhxxr\""
Jul 20 15:26:59.345895 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.345803    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 20 15:26:59.346231 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.345929    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-t2ckp"
Jul 20 15:26:59.347658 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.347643    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7pqm"
Jul 20 15:26:59.347747 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:26:59.347718    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7pqm" podUID="0293426e-c324-435c-873e-468da9602ff6"
Jul 20 15:26:59.350237 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.349874    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.352251 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.352082    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t54tw"
Jul 20 15:26:59.352251 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:26:59.352136    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t54tw" podUID="23b1e60d-e25e-48f8-9ab8-c361f433594c"
Jul 20 15:26:59.354482 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.354464    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kzlg"
Jul 20 15:26:59.354565 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:26:59.354522    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4kzlg" podUID="7b5f0a63-5c26-49e1-a580-8b19d8a6e289"
Jul 20 15:26:59.355400 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.355381    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-hostroot\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.355480 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.355411    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f60e8181-bece-4bff-b8a3-4ad06b349d6c-konnectivity-ca\") pod \"konnectivity-agent-2m28f\" (UID: \"f60e8181-bece-4bff-b8a3-4ad06b349d6c\") " pod="kube-system/konnectivity-agent-2m28f"
Jul 20 15:26:59.355480 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.355435    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7db5w\" (UniqueName: \"kubernetes.io/projected/dc8c759a-ed37-4c8d-bb52-86dfd95ea94b-kube-api-access-7db5w\") pod \"iptables-alerter-t2ckp\" (UID: \"dc8c759a-ed37-4c8d-bb52-86dfd95ea94b\") " pod="openshift-network-operator/iptables-alerter-t2ckp"
Jul 20 15:26:59.355480 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.355458    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5edc4d01-8bc5-45b4-9ae3-ca95d913d778-system-cni-dir\") pod \"multus-additional-cni-plugins-gpzpc\" (UID: \"5edc4d01-8bc5-45b4-9ae3-ca95d913d778\") " pod="openshift-multus/multus-additional-cni-plugins-gpzpc"
Jul 20 15:26:59.355613 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.355479    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-multus-cni-dir\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.355613 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.355500    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-os-release\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.355613 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.355521    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-multus-conf-dir\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.355613 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.355565    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-host-run-multus-certs\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.355613 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.355606    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f5208bb2-476e-43d5-b0aa-86c6383f6ec7-device-dir\") pod \"aws-ebs-csi-driver-node-zqfwk\" (UID: \"f5208bb2-476e-43d5-b0aa-86c6383f6ec7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zqfwk"
Jul 20 15:26:59.355830 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.355642    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dc8c759a-ed37-4c8d-bb52-86dfd95ea94b-iptables-alerter-script\") pod \"iptables-alerter-t2ckp\" (UID: \"dc8c759a-ed37-4c8d-bb52-86dfd95ea94b\") " pod="openshift-network-operator/iptables-alerter-t2ckp"
Jul 20 15:26:59.355830 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.355687    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-multus-socket-dir-parent\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.355830 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.355709    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-host-run-netns\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.355830 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.355732    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f60e8181-bece-4bff-b8a3-4ad06b349d6c-agent-certs\") pod \"konnectivity-agent-2m28f\" (UID: \"f60e8181-bece-4bff-b8a3-4ad06b349d6c\") " pod="kube-system/konnectivity-agent-2m28f"
Jul 20 15:26:59.355830 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.355752    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f5208bb2-476e-43d5-b0aa-86c6383f6ec7-socket-dir\") pod \"aws-ebs-csi-driver-node-zqfwk\" (UID: \"f5208bb2-476e-43d5-b0aa-86c6383f6ec7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zqfwk"
Jul 20 15:26:59.355830 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.355773    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rk5cc\" (UniqueName: \"kubernetes.io/projected/f5208bb2-476e-43d5-b0aa-86c6383f6ec7-kube-api-access-rk5cc\") pod \"aws-ebs-csi-driver-node-zqfwk\" (UID: \"f5208bb2-476e-43d5-b0aa-86c6383f6ec7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zqfwk"
Jul 20 15:26:59.355830 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.355793    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dc8c759a-ed37-4c8d-bb52-86dfd95ea94b-host-slash\") pod \"iptables-alerter-t2ckp\" (UID: \"dc8c759a-ed37-4c8d-bb52-86dfd95ea94b\") " pod="openshift-network-operator/iptables-alerter-t2ckp"
Jul 20 15:26:59.355830 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.355820    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-system-cni-dir\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.356144 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.355852    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-host-var-lib-kubelet\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.356144 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.355887    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ada5dd42-e780-4cc0-a845-ea417f1eac27-hosts-file\") pod \"node-resolver-j45gb\" (UID: \"ada5dd42-e780-4cc0-a845-ea417f1eac27\") " pod="openshift-dns/node-resolver-j45gb"
Jul 20 15:26:59.356144 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.355907    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5edc4d01-8bc5-45b4-9ae3-ca95d913d778-cni-binary-copy\") pod \"multus-additional-cni-plugins-gpzpc\" (UID: \"5edc4d01-8bc5-45b4-9ae3-ca95d913d778\") " pod="openshift-multus/multus-additional-cni-plugins-gpzpc"
Jul 20 15:26:59.356144 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.355928    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-host-var-lib-cni-multus\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.356144 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.355948    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-etc-kubernetes\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.356144 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.355969    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5edc4d01-8bc5-45b4-9ae3-ca95d913d778-cnibin\") pod \"multus-additional-cni-plugins-gpzpc\" (UID: \"5edc4d01-8bc5-45b4-9ae3-ca95d913d778\") " pod="openshift-multus/multus-additional-cni-plugins-gpzpc"
Jul 20 15:26:59.356144 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.356019    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gw69\" (UniqueName: \"kubernetes.io/projected/5edc4d01-8bc5-45b4-9ae3-ca95d913d778-kube-api-access-2gw69\") pod \"multus-additional-cni-plugins-gpzpc\" (UID: \"5edc4d01-8bc5-45b4-9ae3-ca95d913d778\") " pod="openshift-multus/multus-additional-cni-plugins-gpzpc"
Jul 20 15:26:59.356144 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.356049    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-host-var-lib-cni-bin\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.356144 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.356074    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f5208bb2-476e-43d5-b0aa-86c6383f6ec7-registration-dir\") pod \"aws-ebs-csi-driver-node-zqfwk\" (UID: \"f5208bb2-476e-43d5-b0aa-86c6383f6ec7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zqfwk"
Jul 20 15:26:59.356144 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.356099    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5edc4d01-8bc5-45b4-9ae3-ca95d913d778-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gpzpc\" (UID: \"5edc4d01-8bc5-45b4-9ae3-ca95d913d778\") " pod="openshift-multus/multus-additional-cni-plugins-gpzpc"
Jul 20 15:26:59.356144 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.356127    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5edc4d01-8bc5-45b4-9ae3-ca95d913d778-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gpzpc\" (UID: \"5edc4d01-8bc5-45b4-9ae3-ca95d913d778\") " pod="openshift-multus/multus-additional-cni-plugins-gpzpc"
Jul 20 15:26:59.356695 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.356149    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-multus-daemon-config\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.356695 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.356186    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bttc\" (UniqueName: \"kubernetes.io/projected/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-kube-api-access-6bttc\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.356695 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.356220    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f5208bb2-476e-43d5-b0aa-86c6383f6ec7-sys-fs\") pod \"aws-ebs-csi-driver-node-zqfwk\" (UID: \"f5208bb2-476e-43d5-b0aa-86c6383f6ec7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zqfwk"
Jul 20 15:26:59.356695 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.356250    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5edc4d01-8bc5-45b4-9ae3-ca95d913d778-os-release\") pod \"multus-additional-cni-plugins-gpzpc\" (UID: \"5edc4d01-8bc5-45b4-9ae3-ca95d913d778\") " pod="openshift-multus/multus-additional-cni-plugins-gpzpc"
Jul 20 15:26:59.356695 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.356279    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-cnibin\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.356695 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.356326    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-host-run-k8s-cni-cncf-io\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.356695 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.356347    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f5208bb2-476e-43d5-b0aa-86c6383f6ec7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zqfwk\" (UID: \"f5208bb2-476e-43d5-b0aa-86c6383f6ec7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zqfwk"
Jul 20 15:26:59.356695 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.356368    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f5208bb2-476e-43d5-b0aa-86c6383f6ec7-etc-selinux\") pod \"aws-ebs-csi-driver-node-zqfwk\" (UID: \"f5208bb2-476e-43d5-b0aa-86c6383f6ec7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zqfwk"
Jul 20 15:26:59.356695 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.356388    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ada5dd42-e780-4cc0-a845-ea417f1eac27-tmp-dir\") pod \"node-resolver-j45gb\" (UID: \"ada5dd42-e780-4cc0-a845-ea417f1eac27\") " pod="openshift-dns/node-resolver-j45gb"
Jul 20 15:26:59.356695 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.356409    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cg8h\" (UniqueName: \"kubernetes.io/projected/ada5dd42-e780-4cc0-a845-ea417f1eac27-kube-api-access-6cg8h\") pod \"node-resolver-j45gb\" (UID: \"ada5dd42-e780-4cc0-a845-ea417f1eac27\") " pod="openshift-dns/node-resolver-j45gb"
Jul 20 15:26:59.356695 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.356431    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5edc4d01-8bc5-45b4-9ae3-ca95d913d778-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gpzpc\" (UID: \"5edc4d01-8bc5-45b4-9ae3-ca95d913d778\") " pod="openshift-multus/multus-additional-cni-plugins-gpzpc"
Jul 20 15:26:59.356695 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.356451    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-cni-binary-copy\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.361422 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.357724    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 20 15:26:59.361422 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.357780    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 20 15:26:59.361422 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.357946    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 20 15:26:59.361422 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.357973    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 20 15:26:59.361422 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.358790    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 20 15:26:59.361422 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.359160    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.361422 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.359586    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pqqrs"
Jul 20 15:26:59.361422 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.359796    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 20 15:26:59.361422 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.360247    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 20 15:26:59.361422 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.360615    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9wsjg\""
Jul 20 15:26:59.361422 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.360675    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 20 15:26:59.361422 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.360968    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 20 15:26:59.361422 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.361167    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-dc2xv\""
Jul 20 15:26:59.361422 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.361326    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 20 15:26:59.361422 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.361365    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 20 15:26:59.362350 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.361496    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-569z8\""
Jul 20 15:26:59.364034 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.363356    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 20 15:26:59.364034 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.363463    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 20 15:26:59.366340 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.366161    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9phc4\""
Jul 20 15:26:59.366868 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.366614    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 20 15:26:59.367619 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.367072    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 20 15:26:59.367619 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.367320    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 20 15:26:59.367619 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.367489    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-h9hh5\""
Jul 20 15:26:59.374785 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.374694    2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 20 15:26:59.414365 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.414341    2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-wwb8k"
Jul 20 15:26:59.426355 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.426333    2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-wwb8k"
Jul 20 15:26:59.444908 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.444844    2559 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 20 15:26:59.456929 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.456905    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f60e8181-bece-4bff-b8a3-4ad06b349d6c-agent-certs\") pod \"konnectivity-agent-2m28f\" (UID: \"f60e8181-bece-4bff-b8a3-4ad06b349d6c\") " pod="kube-system/konnectivity-agent-2m28f"
Jul 20 15:26:59.457055 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.456939    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q78mt\" (UniqueName: \"kubernetes.io/projected/7b5f0a63-5c26-49e1-a580-8b19d8a6e289-kube-api-access-q78mt\") pod \"network-metrics-daemon-4kzlg\" (UID: \"7b5f0a63-5c26-49e1-a580-8b19d8a6e289\") " pod="openshift-multus/network-metrics-daemon-4kzlg"
Jul 20 15:26:59.457055 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.456959    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-systemd-units\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.457055 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.456981    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-host-var-lib-kubelet\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.457055 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457001    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ada5dd42-e780-4cc0-a845-ea417f1eac27-hosts-file\") pod \"node-resolver-j45gb\" (UID: \"ada5dd42-e780-4cc0-a845-ea417f1eac27\") " pod="openshift-dns/node-resolver-j45gb"
Jul 20 15:26:59.457055 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457016    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b5f0a63-5c26-49e1-a580-8b19d8a6e289-metrics-certs\") pod \"network-metrics-daemon-4kzlg\" (UID: \"7b5f0a63-5c26-49e1-a580-8b19d8a6e289\") " pod="openshift-multus/network-metrics-daemon-4kzlg"
Jul 20 15:26:59.457055 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457032    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5edc4d01-8bc5-45b4-9ae3-ca95d913d778-cni-binary-copy\") pod \"multus-additional-cni-plugins-gpzpc\" (UID: \"5edc4d01-8bc5-45b4-9ae3-ca95d913d778\") " pod="openshift-multus/multus-additional-cni-plugins-gpzpc"
Jul 20 15:26:59.457055 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457056    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-tmp\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.457337 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457064    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-host-var-lib-kubelet\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.457337 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457071    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-host-kubelet\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.457337 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457121    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-var-lib-openvswitch\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.457337 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457150    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-etc-kubernetes\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.457337 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457174    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gw69\" (UniqueName: \"kubernetes.io/projected/5edc4d01-8bc5-45b4-9ae3-ca95d913d778-kube-api-access-2gw69\") pod \"multus-additional-cni-plugins-gpzpc\" (UID: \"5edc4d01-8bc5-45b4-9ae3-ca95d913d778\") " pod="openshift-multus/multus-additional-cni-plugins-gpzpc"
Jul 20 15:26:59.457337 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457221    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ada5dd42-e780-4cc0-a845-ea417f1eac27-hosts-file\") pod \"node-resolver-j45gb\" (UID: \"ada5dd42-e780-4cc0-a845-ea417f1eac27\") " pod="openshift-dns/node-resolver-j45gb"
Jul 20 15:26:59.457337 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457296    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-etc-kubernetes\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.457337 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457330    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-host-var-lib-cni-bin\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.457569 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457318    2559 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 20 15:26:59.457569 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457358    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f5208bb2-476e-43d5-b0aa-86c6383f6ec7-registration-dir\") pod \"aws-ebs-csi-driver-node-zqfwk\" (UID: \"f5208bb2-476e-43d5-b0aa-86c6383f6ec7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zqfwk"
Jul 20 15:26:59.457569 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457381    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-etc-systemd\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.457569 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457399    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-host-slash\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.457569 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457417    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-host-cni-netd\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.457569 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457439    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-etc-sysctl-conf\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.457569 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457461    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5edc4d01-8bc5-45b4-9ae3-ca95d913d778-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gpzpc\" (UID: \"5edc4d01-8bc5-45b4-9ae3-ca95d913d778\") " pod="openshift-multus/multus-additional-cni-plugins-gpzpc"
Jul 20 15:26:59.457569 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457500    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bttc\" (UniqueName: \"kubernetes.io/projected/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-kube-api-access-6bttc\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.457569 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457532    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f5208bb2-476e-43d5-b0aa-86c6383f6ec7-sys-fs\") pod \"aws-ebs-csi-driver-node-zqfwk\" (UID: \"f5208bb2-476e-43d5-b0aa-86c6383f6ec7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zqfwk"
Jul 20 15:26:59.457569 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457558    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-etc-modprobe-d\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.457955 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457584    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-run-openvswitch\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.457955 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457626    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5edc4d01-8bc5-45b4-9ae3-ca95d913d778-os-release\") pod \"multus-additional-cni-plugins-gpzpc\" (UID: \"5edc4d01-8bc5-45b4-9ae3-ca95d913d778\") " pod="openshift-multus/multus-additional-cni-plugins-gpzpc"
Jul 20 15:26:59.457955 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457667    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-host-run-k8s-cni-cncf-io\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.457955 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457691    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f5208bb2-476e-43d5-b0aa-86c6383f6ec7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zqfwk\" (UID: \"f5208bb2-476e-43d5-b0aa-86c6383f6ec7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zqfwk"
Jul 20 15:26:59.457955 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457733    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5edc4d01-8bc5-45b4-9ae3-ca95d913d778-cni-binary-copy\") pod \"multus-additional-cni-plugins-gpzpc\" (UID: \"5edc4d01-8bc5-45b4-9ae3-ca95d913d778\") " pod="openshift-multus/multus-additional-cni-plugins-gpzpc"
Jul 20 15:26:59.457955 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457750    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ada5dd42-e780-4cc0-a845-ea417f1eac27-tmp-dir\") pod \"node-resolver-j45gb\" (UID: \"ada5dd42-e780-4cc0-a845-ea417f1eac27\") " pod="openshift-dns/node-resolver-j45gb"
Jul 20 15:26:59.457955 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457774    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f5208bb2-476e-43d5-b0aa-86c6383f6ec7-sys-fs\") pod \"aws-ebs-csi-driver-node-zqfwk\" (UID: \"f5208bb2-476e-43d5-b0aa-86c6383f6ec7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zqfwk"
Jul 20 15:26:59.457955 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457780    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cg8h\" (UniqueName: \"kubernetes.io/projected/ada5dd42-e780-4cc0-a845-ea417f1eac27-kube-api-access-6cg8h\") pod \"node-resolver-j45gb\" (UID: \"ada5dd42-e780-4cc0-a845-ea417f1eac27\") " pod="openshift-dns/node-resolver-j45gb"
Jul 20 15:26:59.457955 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457822    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f5208bb2-476e-43d5-b0aa-86c6383f6ec7-registration-dir\") pod \"aws-ebs-csi-driver-node-zqfwk\" (UID: \"f5208bb2-476e-43d5-b0aa-86c6383f6ec7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zqfwk"
Jul 20 15:26:59.457955 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457814    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/23b1e60d-e25e-48f8-9ab8-c361f433594c-kubelet-config\") pod \"global-pull-secret-syncer-t54tw\" (UID: \"23b1e60d-e25e-48f8-9ab8-c361f433594c\") " pod="kube-system/global-pull-secret-syncer-t54tw"
Jul 20 15:26:59.457955 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457857    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-host-cni-bin\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.457955 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457879    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/451bb108-e7e1-4ed7-9f73-f40794b199ee-ovnkube-script-lib\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.457955 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457894    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5edc4d01-8bc5-45b4-9ae3-ca95d913d778-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gpzpc\" (UID: \"5edc4d01-8bc5-45b4-9ae3-ca95d913d778\") " pod="openshift-multus/multus-additional-cni-plugins-gpzpc"
Jul 20 15:26:59.457955 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457906    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7db5w\" (UniqueName: \"kubernetes.io/projected/dc8c759a-ed37-4c8d-bb52-86dfd95ea94b-kube-api-access-7db5w\") pod \"iptables-alerter-t2ckp\" (UID: \"dc8c759a-ed37-4c8d-bb52-86dfd95ea94b\") " pod="openshift-network-operator/iptables-alerter-t2ckp"
Jul 20 15:26:59.457955 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457944    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-etc-tuned\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.457955 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457956    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f5208bb2-476e-43d5-b0aa-86c6383f6ec7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zqfwk\" (UID: \"f5208bb2-476e-43d5-b0aa-86c6383f6ec7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zqfwk"
Jul 20 15:26:59.458637 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457969    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b87d2983-fef9-4867-89c6-5f3fe76bd062-serviceca\") pod \"node-ca-pqqrs\" (UID: \"b87d2983-fef9-4867-89c6-5f3fe76bd062\") " pod="openshift-image-registry/node-ca-pqqrs"
Jul 20 15:26:59.458637 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457973    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5edc4d01-8bc5-45b4-9ae3-ca95d913d778-os-release\") pod \"multus-additional-cni-plugins-gpzpc\" (UID: \"5edc4d01-8bc5-45b4-9ae3-ca95d913d778\") " pod="openshift-multus/multus-additional-cni-plugins-gpzpc"
Jul 20 15:26:59.458637 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.457991    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-node-log\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.458637 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458012    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-host-run-k8s-cni-cncf-io\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.458637 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458018    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5edc4d01-8bc5-45b4-9ae3-ca95d913d778-system-cni-dir\") pod \"multus-additional-cni-plugins-gpzpc\" (UID: \"5edc4d01-8bc5-45b4-9ae3-ca95d913d778\") " pod="openshift-multus/multus-additional-cni-plugins-gpzpc"
Jul 20 15:26:59.458637 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458052    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5edc4d01-8bc5-45b4-9ae3-ca95d913d778-system-cni-dir\") pod \"multus-additional-cni-plugins-gpzpc\" (UID: \"5edc4d01-8bc5-45b4-9ae3-ca95d913d778\") " pod="openshift-multus/multus-additional-cni-plugins-gpzpc"
Jul 20 15:26:59.458637 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458071    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-multus-cni-dir\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.458637 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458091    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-host-var-lib-cni-bin\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.458637 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458101    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f5208bb2-476e-43d5-b0aa-86c6383f6ec7-device-dir\") pod \"aws-ebs-csi-driver-node-zqfwk\" (UID: \"f5208bb2-476e-43d5-b0aa-86c6383f6ec7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zqfwk"
Jul 20 15:26:59.458637 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458135    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dc8c759a-ed37-4c8d-bb52-86dfd95ea94b-iptables-alerter-script\") pod \"iptables-alerter-t2ckp\" (UID: \"dc8c759a-ed37-4c8d-bb52-86dfd95ea94b\") " pod="openshift-network-operator/iptables-alerter-t2ckp"
Jul 20 15:26:59.458637 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458161    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-host\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.458637 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458137    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f5208bb2-476e-43d5-b0aa-86c6383f6ec7-device-dir\") pod \"aws-ebs-csi-driver-node-zqfwk\" (UID: \"f5208bb2-476e-43d5-b0aa-86c6383f6ec7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zqfwk"
Jul 20 15:26:59.458637 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458186    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-host-run-netns\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.458637 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458216    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ada5dd42-e780-4cc0-a845-ea417f1eac27-tmp-dir\") pod \"node-resolver-j45gb\" (UID: \"ada5dd42-e780-4cc0-a845-ea417f1eac27\") " pod="openshift-dns/node-resolver-j45gb"
Jul 20 15:26:59.458637 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458220    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f5208bb2-476e-43d5-b0aa-86c6383f6ec7-socket-dir\") pod \"aws-ebs-csi-driver-node-zqfwk\" (UID: \"f5208bb2-476e-43d5-b0aa-86c6383f6ec7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zqfwk"
Jul 20 15:26:59.458637 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458270    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk5cc\" (UniqueName: \"kubernetes.io/projected/f5208bb2-476e-43d5-b0aa-86c6383f6ec7-kube-api-access-rk5cc\") pod \"aws-ebs-csi-driver-node-zqfwk\" (UID: \"f5208bb2-476e-43d5-b0aa-86c6383f6ec7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zqfwk"
Jul 20 15:26:59.458637 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458245    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-host-run-netns\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.459392 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458274    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-multus-cni-dir\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.459392 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458310    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dc8c759a-ed37-4c8d-bb52-86dfd95ea94b-host-slash\") pod \"iptables-alerter-t2ckp\" (UID: \"dc8c759a-ed37-4c8d-bb52-86dfd95ea94b\") " pod="openshift-network-operator/iptables-alerter-t2ckp"
Jul 20 15:26:59.459392 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458334    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv85s\" (UniqueName: \"kubernetes.io/projected/0293426e-c324-435c-873e-468da9602ff6-kube-api-access-jv85s\") pod \"network-check-target-n7pqm\" (UID: \"0293426e-c324-435c-873e-468da9602ff6\") " pod="openshift-network-diagnostics/network-check-target-n7pqm"
Jul 20 15:26:59.459392 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458347    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f5208bb2-476e-43d5-b0aa-86c6383f6ec7-socket-dir\") pod \"aws-ebs-csi-driver-node-zqfwk\" (UID: \"f5208bb2-476e-43d5-b0aa-86c6383f6ec7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zqfwk"
Jul 20 15:26:59.459392 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458356    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-host-run-netns\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.459392 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458389    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-system-cni-dir\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.459392 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458402    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dc8c759a-ed37-4c8d-bb52-86dfd95ea94b-host-slash\") pod \"iptables-alerter-t2ckp\" (UID: \"dc8c759a-ed37-4c8d-bb52-86dfd95ea94b\") " pod="openshift-network-operator/iptables-alerter-t2ckp"
Jul 20 15:26:59.459392 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458437    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/23b1e60d-e25e-48f8-9ab8-c361f433594c-original-pull-secret\") pod \"global-pull-secret-syncer-t54tw\" (UID: \"23b1e60d-e25e-48f8-9ab8-c361f433594c\") " pod="kube-system/global-pull-secret-syncer-t54tw"
Jul 20 15:26:59.459392 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458455    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-system-cni-dir\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.459392 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458463    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-etc-kubernetes\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.459392 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458488    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-etc-sysctl-d\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.459392 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458508    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vd57p\" (UniqueName: \"kubernetes.io/projected/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-kube-api-access-vd57p\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.459392 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458529    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-run-systemd\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.459392 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458574    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/451bb108-e7e1-4ed7-9f73-f40794b199ee-ovn-node-metrics-cert\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.459392 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458598    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-host-var-lib-cni-multus\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.459392 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458637    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-etc-sysconfig\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.459392 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458640    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dc8c759a-ed37-4c8d-bb52-86dfd95ea94b-iptables-alerter-script\") pod \"iptables-alerter-t2ckp\" (UID: \"dc8c759a-ed37-4c8d-bb52-86dfd95ea94b\") " pod="openshift-network-operator/iptables-alerter-t2ckp"
Jul 20 15:26:59.460122 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458651    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-host-var-lib-cni-multus\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.460122 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458659    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-sys\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.460122 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458708    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-etc-openvswitch\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.460122 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458732    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/451bb108-e7e1-4ed7-9f73-f40794b199ee-ovnkube-config\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.460122 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458758    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5edc4d01-8bc5-45b4-9ae3-ca95d913d778-cnibin\") pod \"multus-additional-cni-plugins-gpzpc\" (UID: \"5edc4d01-8bc5-45b4-9ae3-ca95d913d778\") " pod="openshift-multus/multus-additional-cni-plugins-gpzpc"
Jul 20 15:26:59.460122 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458780    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-var-lib-kubelet\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.460122 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458826    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5edc4d01-8bc5-45b4-9ae3-ca95d913d778-cnibin\") pod \"multus-additional-cni-plugins-gpzpc\" (UID: \"5edc4d01-8bc5-45b4-9ae3-ca95d913d778\") " pod="openshift-multus/multus-additional-cni-plugins-gpzpc"
Jul 20 15:26:59.460122 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458836    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.460122 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458873    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5edc4d01-8bc5-45b4-9ae3-ca95d913d778-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gpzpc\" (UID: \"5edc4d01-8bc5-45b4-9ae3-ca95d913d778\") " pod="openshift-multus/multus-additional-cni-plugins-gpzpc"
Jul 20 15:26:59.460122 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458902    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-multus-daemon-config\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.460122 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458958    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/23b1e60d-e25e-48f8-9ab8-c361f433594c-dbus\") pod \"global-pull-secret-syncer-t54tw\" (UID: \"23b1e60d-e25e-48f8-9ab8-c361f433594c\") " pod="kube-system/global-pull-secret-syncer-t54tw"
Jul 20 15:26:59.460122 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.458988    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45pj2\" (UniqueName: \"kubernetes.io/projected/b87d2983-fef9-4867-89c6-5f3fe76bd062-kube-api-access-45pj2\") pod \"node-ca-pqqrs\" (UID: \"b87d2983-fef9-4867-89c6-5f3fe76bd062\") " pod="openshift-image-registry/node-ca-pqqrs"
Jul 20 15:26:59.460122 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.459014    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-cnibin\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.460122 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.459038    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f5208bb2-476e-43d5-b0aa-86c6383f6ec7-etc-selinux\") pod \"aws-ebs-csi-driver-node-zqfwk\" (UID: \"f5208bb2-476e-43d5-b0aa-86c6383f6ec7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zqfwk"
Jul 20 15:26:59.460122 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.459043    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5edc4d01-8bc5-45b4-9ae3-ca95d913d778-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gpzpc\" (UID: \"5edc4d01-8bc5-45b4-9ae3-ca95d913d778\") " pod="openshift-multus/multus-additional-cni-plugins-gpzpc"
Jul 20 15:26:59.460122 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.459063    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-run\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.460122 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.459084    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b87d2983-fef9-4867-89c6-5f3fe76bd062-host\") pod \"node-ca-pqqrs\" (UID: \"b87d2983-fef9-4867-89c6-5f3fe76bd062\") " pod="openshift-image-registry/node-ca-pqqrs"
Jul 20 15:26:59.460909 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.459113    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-cnibin\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.460909 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.459125    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-log-socket\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.460909 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.459148    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f5208bb2-476e-43d5-b0aa-86c6383f6ec7-etc-selinux\") pod \"aws-ebs-csi-driver-node-zqfwk\" (UID: \"f5208bb2-476e-43d5-b0aa-86c6383f6ec7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zqfwk"
Jul 20 15:26:59.460909 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.459165    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5edc4d01-8bc5-45b4-9ae3-ca95d913d778-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gpzpc\" (UID: \"5edc4d01-8bc5-45b4-9ae3-ca95d913d778\") " pod="openshift-multus/multus-additional-cni-plugins-gpzpc"
Jul 20 15:26:59.460909 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.459203    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-cni-binary-copy\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.460909 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.459224    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-hostroot\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.460909 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.459265    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f60e8181-bece-4bff-b8a3-4ad06b349d6c-konnectivity-ca\") pod \"konnectivity-agent-2m28f\" (UID: \"f60e8181-bece-4bff-b8a3-4ad06b349d6c\") " pod="kube-system/konnectivity-agent-2m28f"
Jul 20 15:26:59.460909 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.459281    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-hostroot\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.460909 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.459311    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-lib-modules\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.460909 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.459335    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-run-ovn\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.460909 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.459369    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/451bb108-e7e1-4ed7-9f73-f40794b199ee-env-overrides\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.460909 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.459390    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxgbf\" (UniqueName: \"kubernetes.io/projected/451bb108-e7e1-4ed7-9f73-f40794b199ee-kube-api-access-qxgbf\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.460909 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.459415    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-multus-daemon-config\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.460909 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.459419    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-os-release\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.460909 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.459461    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-multus-conf-dir\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.460909 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.459490    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-os-release\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.460909 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.459499    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-multus-conf-dir\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.460909 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.459524    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-host-run-multus-certs\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.461771 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.459559    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-host-run-ovn-kubernetes\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.461771 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.459583    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-multus-socket-dir-parent\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.461771 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.459615    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-host-run-multus-certs\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.461771 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.459683    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5edc4d01-8bc5-45b4-9ae3-ca95d913d778-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gpzpc\" (UID: \"5edc4d01-8bc5-45b4-9ae3-ca95d913d778\") " pod="openshift-multus/multus-additional-cni-plugins-gpzpc"
Jul 20 15:26:59.461771 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.459699    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-multus-socket-dir-parent\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.461771 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.459770    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f60e8181-bece-4bff-b8a3-4ad06b349d6c-konnectivity-ca\") pod \"konnectivity-agent-2m28f\" (UID: \"f60e8181-bece-4bff-b8a3-4ad06b349d6c\") " pod="kube-system/konnectivity-agent-2m28f"
Jul 20 15:26:59.461771 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.459880    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-cni-binary-copy\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.462185 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.462166    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f60e8181-bece-4bff-b8a3-4ad06b349d6c-agent-certs\") pod \"konnectivity-agent-2m28f\" (UID: \"f60e8181-bece-4bff-b8a3-4ad06b349d6c\") " pod="kube-system/konnectivity-agent-2m28f"
Jul 20 15:26:59.466860 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.466837    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7db5w\" (UniqueName: \"kubernetes.io/projected/dc8c759a-ed37-4c8d-bb52-86dfd95ea94b-kube-api-access-7db5w\") pod \"iptables-alerter-t2ckp\" (UID: \"dc8c759a-ed37-4c8d-bb52-86dfd95ea94b\") " pod="openshift-network-operator/iptables-alerter-t2ckp"
Jul 20 15:26:59.467490 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.467449    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cg8h\" (UniqueName: \"kubernetes.io/projected/ada5dd42-e780-4cc0-a845-ea417f1eac27-kube-api-access-6cg8h\") pod \"node-resolver-j45gb\" (UID: \"ada5dd42-e780-4cc0-a845-ea417f1eac27\") " pod="openshift-dns/node-resolver-j45gb"
Jul 20 15:26:59.467717 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.467670    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gw69\" (UniqueName: \"kubernetes.io/projected/5edc4d01-8bc5-45b4-9ae3-ca95d913d778-kube-api-access-2gw69\") pod \"multus-additional-cni-plugins-gpzpc\" (UID: \"5edc4d01-8bc5-45b4-9ae3-ca95d913d778\") " pod="openshift-multus/multus-additional-cni-plugins-gpzpc"
Jul 20 15:26:59.468217 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.468197    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rk5cc\" (UniqueName: \"kubernetes.io/projected/f5208bb2-476e-43d5-b0aa-86c6383f6ec7-kube-api-access-rk5cc\") pod \"aws-ebs-csi-driver-node-zqfwk\" (UID: \"f5208bb2-476e-43d5-b0aa-86c6383f6ec7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zqfwk"
Jul 20 15:26:59.468658 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.468629    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bttc\" (UniqueName: \"kubernetes.io/projected/a2253ac6-88dd-4ed1-aa18-1279dd83bea8-kube-api-access-6bttc\") pod \"multus-tx286\" (UID: \"a2253ac6-88dd-4ed1-aa18-1279dd83bea8\") " pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.560308 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.560259    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-lib-modules\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.560466 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.560355    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-run-ovn\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.560466 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.560379    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/451bb108-e7e1-4ed7-9f73-f40794b199ee-env-overrides\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.560466 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.560401    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxgbf\" (UniqueName: \"kubernetes.io/projected/451bb108-e7e1-4ed7-9f73-f40794b199ee-kube-api-access-qxgbf\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.560466 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.560412    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-lib-modules\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.560466 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.560423    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-host-run-ovn-kubernetes\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.560466 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.560456    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-host-run-ovn-kubernetes\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.560747 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.560462    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-run-ovn\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.560747 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.560474    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q78mt\" (UniqueName: \"kubernetes.io/projected/7b5f0a63-5c26-49e1-a580-8b19d8a6e289-kube-api-access-q78mt\") pod \"network-metrics-daemon-4kzlg\" (UID: \"7b5f0a63-5c26-49e1-a580-8b19d8a6e289\") " pod="openshift-multus/network-metrics-daemon-4kzlg"
Jul 20 15:26:59.560747 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.560509    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-systemd-units\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.560747 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.560544    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b5f0a63-5c26-49e1-a580-8b19d8a6e289-metrics-certs\") pod \"network-metrics-daemon-4kzlg\" (UID: \"7b5f0a63-5c26-49e1-a580-8b19d8a6e289\") " pod="openshift-multus/network-metrics-daemon-4kzlg"
Jul 20 15:26:59.560747 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.560571    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-tmp\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.560747 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.560593    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-host-kubelet\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.560747 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.560615    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-var-lib-openvswitch\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.560747 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.560640    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-etc-systemd\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.560747 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.560660    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-host-slash\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.560747 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.560681    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-host-cni-netd\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.560747 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:26:59.560693    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 15:26:59.560747 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.560701    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-etc-sysctl-conf\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.560747 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:26:59.560752    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b5f0a63-5c26-49e1-a580-8b19d8a6e289-metrics-certs podName:7b5f0a63-5c26-49e1-a580-8b19d8a6e289 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:00.060733175 +0000 UTC m=+3.070354818 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b5f0a63-5c26-49e1-a580-8b19d8a6e289-metrics-certs") pod "network-metrics-daemon-4kzlg" (UID: "7b5f0a63-5c26-49e1-a580-8b19d8a6e289") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 15:26:59.560747 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.560750    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-var-lib-openvswitch\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.561444 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.560770    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-etc-modprobe-d\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.561444 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.560796    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-run-openvswitch\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.561444 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.560823    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/23b1e60d-e25e-48f8-9ab8-c361f433594c-kubelet-config\") pod \"global-pull-secret-syncer-t54tw\" (UID: \"23b1e60d-e25e-48f8-9ab8-c361f433594c\") " pod="kube-system/global-pull-secret-syncer-t54tw"
Jul 20 15:26:59.561444 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.560835    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-etc-sysctl-conf\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.561444 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.560846    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-host-cni-bin\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.561444 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.560867    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/451bb108-e7e1-4ed7-9f73-f40794b199ee-ovnkube-script-lib\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.561444 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.560872    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/451bb108-e7e1-4ed7-9f73-f40794b199ee-env-overrides\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.561444 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.560884    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-etc-systemd\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.561444 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.560889    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-etc-tuned\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.561444 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.560911    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b87d2983-fef9-4867-89c6-5f3fe76bd062-serviceca\") pod \"node-ca-pqqrs\" (UID: \"b87d2983-fef9-4867-89c6-5f3fe76bd062\") " pod="openshift-image-registry/node-ca-pqqrs"
Jul 20 15:26:59.561444 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.560925    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-host-slash\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.561444 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.560931    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-node-log\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.561444 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.560797    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-systemd-units\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.561444 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.560934    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-host-cni-netd\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.561444 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.560972    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-host-cni-bin\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.561444 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.560997    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-host\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.561444 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.561026    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv85s\" (UniqueName: \"kubernetes.io/projected/0293426e-c324-435c-873e-468da9602ff6-kube-api-access-jv85s\") pod \"network-check-target-n7pqm\" (UID: \"0293426e-c324-435c-873e-468da9602ff6\") " pod="openshift-network-diagnostics/network-check-target-n7pqm"
Jul 20 15:26:59.561444 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.561049    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-host-run-netns\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.562207 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.561071    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/23b1e60d-e25e-48f8-9ab8-c361f433594c-original-pull-secret\") pod \"global-pull-secret-syncer-t54tw\" (UID: \"23b1e60d-e25e-48f8-9ab8-c361f433594c\") " pod="kube-system/global-pull-secret-syncer-t54tw"
Jul 20 15:26:59.562207 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.561097    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-etc-kubernetes\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.562207 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.561117    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-etc-sysctl-d\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.562207 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.561138    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vd57p\" (UniqueName: \"kubernetes.io/projected/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-kube-api-access-vd57p\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.562207 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.561160    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-run-systemd\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.562207 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.561183    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/451bb108-e7e1-4ed7-9f73-f40794b199ee-ovn-node-metrics-cert\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.562207 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.561207    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-etc-sysconfig\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.562207 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.561227    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-sys\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.562207 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.561222    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-host-run-netns\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.562207 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.561258    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-etc-openvswitch\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.562207 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.561300    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/451bb108-e7e1-4ed7-9f73-f40794b199ee-ovnkube-config\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.562207 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.561301    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/23b1e60d-e25e-48f8-9ab8-c361f433594c-kubelet-config\") pod \"global-pull-secret-syncer-t54tw\" (UID: \"23b1e60d-e25e-48f8-9ab8-c361f433594c\") " pod="kube-system/global-pull-secret-syncer-t54tw"
Jul 20 15:26:59.562207 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.561325    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-var-lib-kubelet\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.562207 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.561336    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b87d2983-fef9-4867-89c6-5f3fe76bd062-serviceca\") pod \"node-ca-pqqrs\" (UID: \"b87d2983-fef9-4867-89c6-5f3fe76bd062\") " pod="openshift-image-registry/node-ca-pqqrs"
Jul 20 15:26:59.562207 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.561347    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-node-log\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.562207 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.561349    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.562207 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.561374    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-run-openvswitch\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.563084 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.561389    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-host\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.563084 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.561392    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.563084 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.561396    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-etc-modprobe-d\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.563084 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.561417    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/23b1e60d-e25e-48f8-9ab8-c361f433594c-dbus\") pod \"global-pull-secret-syncer-t54tw\" (UID: \"23b1e60d-e25e-48f8-9ab8-c361f433594c\") " pod="kube-system/global-pull-secret-syncer-t54tw"
Jul 20 15:26:59.563084 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.561440    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45pj2\" (UniqueName: \"kubernetes.io/projected/b87d2983-fef9-4867-89c6-5f3fe76bd062-kube-api-access-45pj2\") pod \"node-ca-pqqrs\" (UID: \"b87d2983-fef9-4867-89c6-5f3fe76bd062\") " pod="openshift-image-registry/node-ca-pqqrs"
Jul 20 15:26:59.563084 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.561444    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-host-kubelet\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.563084 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.561464    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-run\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.563084 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.561486    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b87d2983-fef9-4867-89c6-5f3fe76bd062-host\") pod \"node-ca-pqqrs\" (UID: \"b87d2983-fef9-4867-89c6-5f3fe76bd062\") " pod="openshift-image-registry/node-ca-pqqrs"
Jul 20 15:26:59.563084 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.561507    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-log-socket\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.563084 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.561508    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-etc-openvswitch\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.563084 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.561587    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-run-systemd\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.563084 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.561593    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-var-lib-kubelet\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.563084 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.561602    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-etc-sysctl-d\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.563084 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.561638    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/451bb108-e7e1-4ed7-9f73-f40794b199ee-log-socket\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.563084 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.561648    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-sys\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.563084 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.561659    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-etc-kubernetes\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.563084 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.561678    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-etc-sysconfig\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.563084 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:26:59.561712    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 15:26:59.563903 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.561749    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b87d2983-fef9-4867-89c6-5f3fe76bd062-host\") pod \"node-ca-pqqrs\" (UID: \"b87d2983-fef9-4867-89c6-5f3fe76bd062\") " pod="openshift-image-registry/node-ca-pqqrs"
Jul 20 15:26:59.563903 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.561809    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/23b1e60d-e25e-48f8-9ab8-c361f433594c-dbus\") pod \"global-pull-secret-syncer-t54tw\" (UID: \"23b1e60d-e25e-48f8-9ab8-c361f433594c\") " pod="kube-system/global-pull-secret-syncer-t54tw"
Jul 20 15:26:59.563903 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.561810    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-run\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.563903 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:26:59.561856    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23b1e60d-e25e-48f8-9ab8-c361f433594c-original-pull-secret podName:23b1e60d-e25e-48f8-9ab8-c361f433594c nodeName:}" failed. No retries permitted until 2026-07-20 15:27:00.061840489 +0000 UTC m=+3.071462140 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/23b1e60d-e25e-48f8-9ab8-c361f433594c-original-pull-secret") pod "global-pull-secret-syncer-t54tw" (UID: "23b1e60d-e25e-48f8-9ab8-c361f433594c") : object "kube-system"/"original-pull-secret" not registered
Jul 20 15:26:59.563903 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.561920    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/451bb108-e7e1-4ed7-9f73-f40794b199ee-ovnkube-script-lib\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.563903 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.562800    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/451bb108-e7e1-4ed7-9f73-f40794b199ee-ovnkube-config\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.563903 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.563828    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/451bb108-e7e1-4ed7-9f73-f40794b199ee-ovn-node-metrics-cert\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.564157 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.563926    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-etc-tuned\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.564157 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.564140    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-tmp\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.574701 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:26:59.574680    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 15:26:59.574701 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:26:59.574703    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 15:26:59.574845 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:26:59.574713    2559 projected.go:196] Error preparing data for projected volume kube-api-access-jv85s for pod openshift-network-diagnostics/network-check-target-n7pqm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 15:26:59.574845 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:26:59.574762    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0293426e-c324-435c-873e-468da9602ff6-kube-api-access-jv85s podName:0293426e-c324-435c-873e-468da9602ff6 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:00.074747345 +0000 UTC m=+3.084368986 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-jv85s" (UniqueName: "kubernetes.io/projected/0293426e-c324-435c-873e-468da9602ff6-kube-api-access-jv85s") pod "network-check-target-n7pqm" (UID: "0293426e-c324-435c-873e-468da9602ff6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 15:26:59.576377 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.576355    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45pj2\" (UniqueName: \"kubernetes.io/projected/b87d2983-fef9-4867-89c6-5f3fe76bd062-kube-api-access-45pj2\") pod \"node-ca-pqqrs\" (UID: \"b87d2983-fef9-4867-89c6-5f3fe76bd062\") " pod="openshift-image-registry/node-ca-pqqrs"
Jul 20 15:26:59.577700 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.577673    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vd57p\" (UniqueName: \"kubernetes.io/projected/c26d7ae2-4424-47c6-b4cf-2815b1f353f8-kube-api-access-vd57p\") pod \"tuned-9nxcn\" (UID: \"c26d7ae2-4424-47c6-b4cf-2815b1f353f8\") " pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.579196 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.579176    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q78mt\" (UniqueName: \"kubernetes.io/projected/7b5f0a63-5c26-49e1-a580-8b19d8a6e289-kube-api-access-q78mt\") pod \"network-metrics-daemon-4kzlg\" (UID: \"7b5f0a63-5c26-49e1-a580-8b19d8a6e289\") " pod="openshift-multus/network-metrics-daemon-4kzlg"
Jul 20 15:26:59.579338 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.579323    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxgbf\" (UniqueName: \"kubernetes.io/projected/451bb108-e7e1-4ed7-9f73-f40794b199ee-kube-api-access-qxgbf\") pod \"ovnkube-node-7hgpz\" (UID: \"451bb108-e7e1-4ed7-9f73-f40794b199ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.648094 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.648059    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gpzpc"
Jul 20 15:26:59.656015 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.655988    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tx286"
Jul 20 15:26:59.665572 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.665556    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2m28f"
Jul 20 15:26:59.674974 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.674958    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zqfwk"
Jul 20 15:26:59.683345 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.683327    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-j45gb"
Jul 20 15:26:59.689774 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.689755    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-t2ckp"
Jul 20 15:26:59.697240 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.697190    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:26:59.704386 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.704367    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9nxcn"
Jul 20 15:26:59.709806 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:26:59.709788    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pqqrs"
Jul 20 15:27:00.066272 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:00.066198    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b5f0a63-5c26-49e1-a580-8b19d8a6e289-metrics-certs\") pod \"network-metrics-daemon-4kzlg\" (UID: \"7b5f0a63-5c26-49e1-a580-8b19d8a6e289\") " pod="openshift-multus/network-metrics-daemon-4kzlg"
Jul 20 15:27:00.066272 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:00.066253    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/23b1e60d-e25e-48f8-9ab8-c361f433594c-original-pull-secret\") pod \"global-pull-secret-syncer-t54tw\" (UID: \"23b1e60d-e25e-48f8-9ab8-c361f433594c\") " pod="kube-system/global-pull-secret-syncer-t54tw"
Jul 20 15:27:00.066445 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:00.066377    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 15:27:00.066475 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:00.066450    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b5f0a63-5c26-49e1-a580-8b19d8a6e289-metrics-certs podName:7b5f0a63-5c26-49e1-a580-8b19d8a6e289 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:01.066430122 +0000 UTC m=+4.076051761 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b5f0a63-5c26-49e1-a580-8b19d8a6e289-metrics-certs") pod "network-metrics-daemon-4kzlg" (UID: "7b5f0a63-5c26-49e1-a580-8b19d8a6e289") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 15:27:00.066475 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:00.066376    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 15:27:00.066538 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:00.066527    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23b1e60d-e25e-48f8-9ab8-c361f433594c-original-pull-secret podName:23b1e60d-e25e-48f8-9ab8-c361f433594c nodeName:}" failed. No retries permitted until 2026-07-20 15:27:01.066509346 +0000 UTC m=+4.076130986 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/23b1e60d-e25e-48f8-9ab8-c361f433594c-original-pull-secret") pod "global-pull-secret-syncer-t54tw" (UID: "23b1e60d-e25e-48f8-9ab8-c361f433594c") : object "kube-system"/"original-pull-secret" not registered
Jul 20 15:27:00.167072 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:00.167038    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv85s\" (UniqueName: \"kubernetes.io/projected/0293426e-c324-435c-873e-468da9602ff6-kube-api-access-jv85s\") pod \"network-check-target-n7pqm\" (UID: \"0293426e-c324-435c-873e-468da9602ff6\") " pod="openshift-network-diagnostics/network-check-target-n7pqm"
Jul 20 15:27:00.167250 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:00.167209    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 15:27:00.167250 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:00.167235    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 15:27:00.167250 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:00.167247    2559 projected.go:196] Error preparing data for projected volume kube-api-access-jv85s for pod openshift-network-diagnostics/network-check-target-n7pqm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 15:27:00.167419 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:00.167332    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0293426e-c324-435c-873e-468da9602ff6-kube-api-access-jv85s podName:0293426e-c324-435c-873e-468da9602ff6 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:01.167313212 +0000 UTC m=+4.176934869 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-jv85s" (UniqueName: "kubernetes.io/projected/0293426e-c324-435c-873e-468da9602ff6-kube-api-access-jv85s") pod "network-check-target-n7pqm" (UID: "0293426e-c324-435c-873e-468da9602ff6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 15:27:00.413171 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:27:00.413124    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc26d7ae2_4424_47c6_b4cf_2815b1f353f8.slice/crio-e2bfe4ae72c1af901fe62abdf038131c20b7414a71c3b09d9825a6d037989686 WatchSource:0}: Error finding container e2bfe4ae72c1af901fe62abdf038131c20b7414a71c3b09d9825a6d037989686: Status 404 returned error can't find the container with id e2bfe4ae72c1af901fe62abdf038131c20b7414a71c3b09d9825a6d037989686
Jul 20 15:27:00.419765 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:27:00.419739    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb87d2983_fef9_4867_89c6_5f3fe76bd062.slice/crio-853ba9197818bbd12722b80984e4bba5be078586e5cc50b9dc0801eb295040a1 WatchSource:0}: Error finding container 853ba9197818bbd12722b80984e4bba5be078586e5cc50b9dc0801eb295040a1: Status 404 returned error can't find the container with id 853ba9197818bbd12722b80984e4bba5be078586e5cc50b9dc0801eb295040a1
Jul 20 15:27:00.421383 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:27:00.421348    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc8c759a_ed37_4c8d_bb52_86dfd95ea94b.slice/crio-4df6db332a23353798ecec260e4851bf454d748992a57310d6ae27dbeb631353 WatchSource:0}: Error finding container 4df6db332a23353798ecec260e4851bf454d748992a57310d6ae27dbeb631353: Status 404 returned error can't find the container with id 4df6db332a23353798ecec260e4851bf454d748992a57310d6ae27dbeb631353
Jul 20 15:27:00.422241 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:27:00.422218    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2253ac6_88dd_4ed1_aa18_1279dd83bea8.slice/crio-bf65f4345d50d175a9f60116f6afc14378bffdd093fe55436f0b29824f582a19 WatchSource:0}: Error finding container bf65f4345d50d175a9f60116f6afc14378bffdd093fe55436f0b29824f582a19: Status 404 returned error can't find the container with id bf65f4345d50d175a9f60116f6afc14378bffdd093fe55436f0b29824f582a19
Jul 20 15:27:00.423095 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:27:00.422994    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podada5dd42_e780_4cc0_a845_ea417f1eac27.slice/crio-6069ce0aefedca1a8d65cdccfc927959b8eccf7db5e125f512e4b820539206e1 WatchSource:0}: Error finding container 6069ce0aefedca1a8d65cdccfc927959b8eccf7db5e125f512e4b820539206e1: Status 404 returned error can't find the container with id 6069ce0aefedca1a8d65cdccfc927959b8eccf7db5e125f512e4b820539206e1
Jul 20 15:27:00.429366 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:00.429343    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 15:21:59 +0000 UTC" deadline="2028-04-23 03:18:11.641511706 +0000 UTC"
Jul 20 15:27:00.429366 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:00.429365    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15419h51m11.212149384s"
Jul 20 15:27:00.449876 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:27:00.449855    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5208bb2_476e_43d5_b0aa_86c6383f6ec7.slice/crio-acce87f0669fa28cb7d8c31557305e462b604dcd5305b3aee2fc80b39287e463 WatchSource:0}: Error finding container acce87f0669fa28cb7d8c31557305e462b604dcd5305b3aee2fc80b39287e463: Status 404 returned error can't find the container with id acce87f0669fa28cb7d8c31557305e462b604dcd5305b3aee2fc80b39287e463
Jul 20 15:27:00.516736 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:00.516705    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gpzpc" event={"ID":"5edc4d01-8bc5-45b4-9ae3-ca95d913d778","Type":"ContainerStarted","Data":"e8b43fdabdb0a13e1613b01ea59c9f296b8914e09dfa9badd04f48859b255198"}
Jul 20 15:27:00.518009 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:00.517987    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz" event={"ID":"451bb108-e7e1-4ed7-9f73-f40794b199ee","Type":"ContainerStarted","Data":"ef6be7f5ffe33935a82bc4f902ede4992caa27243bee3190070550ebb457a86b"}
Jul 20 15:27:00.519234 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:00.519198    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-j45gb" event={"ID":"ada5dd42-e780-4cc0-a845-ea417f1eac27","Type":"ContainerStarted","Data":"6069ce0aefedca1a8d65cdccfc927959b8eccf7db5e125f512e4b820539206e1"}
Jul 20 15:27:00.520481 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:00.520452    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tx286" event={"ID":"a2253ac6-88dd-4ed1-aa18-1279dd83bea8","Type":"ContainerStarted","Data":"bf65f4345d50d175a9f60116f6afc14378bffdd093fe55436f0b29824f582a19"}
Jul 20 15:27:00.522419 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:00.522364    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-t2ckp" event={"ID":"dc8c759a-ed37-4c8d-bb52-86dfd95ea94b","Type":"ContainerStarted","Data":"4df6db332a23353798ecec260e4851bf454d748992a57310d6ae27dbeb631353"}
Jul 20 15:27:00.524350 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:00.524322    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zqfwk" event={"ID":"f5208bb2-476e-43d5-b0aa-86c6383f6ec7","Type":"ContainerStarted","Data":"acce87f0669fa28cb7d8c31557305e462b604dcd5305b3aee2fc80b39287e463"}
Jul 20 15:27:00.525334 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:00.525316    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2m28f" event={"ID":"f60e8181-bece-4bff-b8a3-4ad06b349d6c","Type":"ContainerStarted","Data":"40fda57e0d6bfa526545ea33687e0749d7b830880c10c68999a9f5b2c0706611"}
Jul 20 15:27:00.526921 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:00.526902    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pqqrs" event={"ID":"b87d2983-fef9-4867-89c6-5f3fe76bd062","Type":"ContainerStarted","Data":"853ba9197818bbd12722b80984e4bba5be078586e5cc50b9dc0801eb295040a1"}
Jul 20 15:27:00.530640 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:00.530621    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9nxcn" event={"ID":"c26d7ae2-4424-47c6-b4cf-2815b1f353f8","Type":"ContainerStarted","Data":"e2bfe4ae72c1af901fe62abdf038131c20b7414a71c3b09d9825a6d037989686"}
Jul 20 15:27:01.074748 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:01.074514    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/23b1e60d-e25e-48f8-9ab8-c361f433594c-original-pull-secret\") pod \"global-pull-secret-syncer-t54tw\" (UID: \"23b1e60d-e25e-48f8-9ab8-c361f433594c\") " pod="kube-system/global-pull-secret-syncer-t54tw"
Jul 20 15:27:01.074941 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:01.074794    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b5f0a63-5c26-49e1-a580-8b19d8a6e289-metrics-certs\") pod \"network-metrics-daemon-4kzlg\" (UID: \"7b5f0a63-5c26-49e1-a580-8b19d8a6e289\") " pod="openshift-multus/network-metrics-daemon-4kzlg"
Jul 20 15:27:01.074941 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:01.074933    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 15:27:01.075078 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:01.074989    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b5f0a63-5c26-49e1-a580-8b19d8a6e289-metrics-certs podName:7b5f0a63-5c26-49e1-a580-8b19d8a6e289 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:03.074971659 +0000 UTC m=+6.084593312 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b5f0a63-5c26-49e1-a580-8b19d8a6e289-metrics-certs") pod "network-metrics-daemon-4kzlg" (UID: "7b5f0a63-5c26-49e1-a580-8b19d8a6e289") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 15:27:01.075505 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:01.075372    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 15:27:01.075505 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:01.075422    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23b1e60d-e25e-48f8-9ab8-c361f433594c-original-pull-secret podName:23b1e60d-e25e-48f8-9ab8-c361f433594c nodeName:}" failed. No retries permitted until 2026-07-20 15:27:03.075407465 +0000 UTC m=+6.085029111 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/23b1e60d-e25e-48f8-9ab8-c361f433594c-original-pull-secret") pod "global-pull-secret-syncer-t54tw" (UID: "23b1e60d-e25e-48f8-9ab8-c361f433594c") : object "kube-system"/"original-pull-secret" not registered
Jul 20 15:27:01.176205 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:01.176139    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv85s\" (UniqueName: \"kubernetes.io/projected/0293426e-c324-435c-873e-468da9602ff6-kube-api-access-jv85s\") pod \"network-check-target-n7pqm\" (UID: \"0293426e-c324-435c-873e-468da9602ff6\") " pod="openshift-network-diagnostics/network-check-target-n7pqm"
Jul 20 15:27:01.176376 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:01.176334    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 15:27:01.176376 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:01.176356    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 15:27:01.176376 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:01.176369    2559 projected.go:196] Error preparing data for projected volume kube-api-access-jv85s for pod openshift-network-diagnostics/network-check-target-n7pqm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 15:27:01.176546 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:01.176424    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0293426e-c324-435c-873e-468da9602ff6-kube-api-access-jv85s podName:0293426e-c324-435c-873e-468da9602ff6 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:03.176405854 +0000 UTC m=+6.186027500 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-jv85s" (UniqueName: "kubernetes.io/projected/0293426e-c324-435c-873e-468da9602ff6-kube-api-access-jv85s") pod "network-check-target-n7pqm" (UID: "0293426e-c324-435c-873e-468da9602ff6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 15:27:01.269031 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:01.268670    2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 15:27:01.432408 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:01.429611    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 15:21:59 +0000 UTC" deadline="2028-03-03 04:10:11.545228376 +0000 UTC"
Jul 20 15:27:01.432408 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:01.429665    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14196h43m10.115567586s"
Jul 20 15:27:01.511893 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:01.511046    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7pqm"
Jul 20 15:27:01.511893 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:01.511186    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7pqm" podUID="0293426e-c324-435c-873e-468da9602ff6"
Jul 20 15:27:01.511893 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:01.511594    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t54tw"
Jul 20 15:27:01.511893 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:01.511691    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t54tw" podUID="23b1e60d-e25e-48f8-9ab8-c361f433594c"
Jul 20 15:27:01.511893 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:01.511757    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kzlg"
Jul 20 15:27:01.511893 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:01.511843    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4kzlg" podUID="7b5f0a63-5c26-49e1-a580-8b19d8a6e289"
Jul 20 15:27:01.552621 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:01.552552    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-19.ec2.internal" event={"ID":"9120b8f40b5d9a133728742e2ee8b96d","Type":"ContainerStarted","Data":"11db6f40dd1668f6809d187eb763dd7070df0eea2565341f35b9e8f6e6b4e60a"}
Jul 20 15:27:01.562926 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:01.562898    2559 generic.go:356] "Generic (PLEG): container finished" podID="efe68e62a819b5c98670e409a64453e3" containerID="1ce6085ee3d8d74ea55cc60c955b23288c5fe9e87d3bd15545286ca42ffdcfca" exitCode=0
Jul 20 15:27:01.563025 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:01.562946    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.ec2.internal" event={"ID":"efe68e62a819b5c98670e409a64453e3","Type":"ContainerDied","Data":"1ce6085ee3d8d74ea55cc60c955b23288c5fe9e87d3bd15545286ca42ffdcfca"}
Jul 20 15:27:01.597343 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:01.597276    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-19.ec2.internal" podStartSLOduration=3.5972579959999997 podStartE2EDuration="3.597257996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:26:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:27:01.572137828 +0000 UTC m=+4.581759491" watchObservedRunningTime="2026-07-20 15:27:01.597257996 +0000 UTC m=+4.606879661"
Jul 20 15:27:02.579162 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:02.579127    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.ec2.internal" event={"ID":"efe68e62a819b5c98670e409a64453e3","Type":"ContainerStarted","Data":"437ded32bea0267d8b006a09261810cfd62c498695f759f4462be50fa36347f4"}
Jul 20 15:27:03.092876 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:03.092841    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b5f0a63-5c26-49e1-a580-8b19d8a6e289-metrics-certs\") pod \"network-metrics-daemon-4kzlg\" (UID: \"7b5f0a63-5c26-49e1-a580-8b19d8a6e289\") " pod="openshift-multus/network-metrics-daemon-4kzlg"
Jul 20 15:27:03.093061 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:03.092928    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/23b1e60d-e25e-48f8-9ab8-c361f433594c-original-pull-secret\") pod \"global-pull-secret-syncer-t54tw\" (UID: \"23b1e60d-e25e-48f8-9ab8-c361f433594c\") " pod="kube-system/global-pull-secret-syncer-t54tw"
Jul 20 15:27:03.093125 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:03.093083    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 15:27:03.093174 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:03.093145    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23b1e60d-e25e-48f8-9ab8-c361f433594c-original-pull-secret podName:23b1e60d-e25e-48f8-9ab8-c361f433594c nodeName:}" failed. No retries permitted until 2026-07-20 15:27:07.093125886 +0000 UTC m=+10.102747544 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/23b1e60d-e25e-48f8-9ab8-c361f433594c-original-pull-secret") pod "global-pull-secret-syncer-t54tw" (UID: "23b1e60d-e25e-48f8-9ab8-c361f433594c") : object "kube-system"/"original-pull-secret" not registered
Jul 20 15:27:03.093551 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:03.093532    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 15:27:03.093645 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:03.093635    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b5f0a63-5c26-49e1-a580-8b19d8a6e289-metrics-certs podName:7b5f0a63-5c26-49e1-a580-8b19d8a6e289 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:07.093570512 +0000 UTC m=+10.103192210 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b5f0a63-5c26-49e1-a580-8b19d8a6e289-metrics-certs") pod "network-metrics-daemon-4kzlg" (UID: "7b5f0a63-5c26-49e1-a580-8b19d8a6e289") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 15:27:03.193526 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:03.193485    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv85s\" (UniqueName: \"kubernetes.io/projected/0293426e-c324-435c-873e-468da9602ff6-kube-api-access-jv85s\") pod \"network-check-target-n7pqm\" (UID: \"0293426e-c324-435c-873e-468da9602ff6\") " pod="openshift-network-diagnostics/network-check-target-n7pqm"
Jul 20 15:27:03.193720 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:03.193705    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 15:27:03.193787 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:03.193765    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 15:27:03.193787 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:03.193781    2559 projected.go:196] Error preparing data for projected volume kube-api-access-jv85s for pod openshift-network-diagnostics/network-check-target-n7pqm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 15:27:03.193888 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:03.193838    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0293426e-c324-435c-873e-468da9602ff6-kube-api-access-jv85s podName:0293426e-c324-435c-873e-468da9602ff6 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:07.193821002 +0000 UTC m=+10.203442643 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-jv85s" (UniqueName: "kubernetes.io/projected/0293426e-c324-435c-873e-468da9602ff6-kube-api-access-jv85s") pod "network-check-target-n7pqm" (UID: "0293426e-c324-435c-873e-468da9602ff6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 15:27:03.510351 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:03.510129    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7pqm"
Jul 20 15:27:03.510351 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:03.510257    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7pqm" podUID="0293426e-c324-435c-873e-468da9602ff6"
Jul 20 15:27:03.510351 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:03.510339    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kzlg"
Jul 20 15:27:03.510563 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:03.510432    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4kzlg" podUID="7b5f0a63-5c26-49e1-a580-8b19d8a6e289"
Jul 20 15:27:03.510563 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:03.510469    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t54tw"
Jul 20 15:27:03.510563 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:03.510530    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t54tw" podUID="23b1e60d-e25e-48f8-9ab8-c361f433594c"
Jul 20 15:27:05.508975 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:05.508943    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t54tw"
Jul 20 15:27:05.509434 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:05.509073    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t54tw" podUID="23b1e60d-e25e-48f8-9ab8-c361f433594c"
Jul 20 15:27:05.509434 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:05.509325    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kzlg"
Jul 20 15:27:05.509556 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:05.509446    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7pqm"
Jul 20 15:27:05.509556 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:05.509439    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4kzlg" podUID="7b5f0a63-5c26-49e1-a580-8b19d8a6e289"
Jul 20 15:27:05.509556 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:05.509518    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7pqm" podUID="0293426e-c324-435c-873e-468da9602ff6"
Jul 20 15:27:07.125317 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:07.125213    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/23b1e60d-e25e-48f8-9ab8-c361f433594c-original-pull-secret\") pod \"global-pull-secret-syncer-t54tw\" (UID: \"23b1e60d-e25e-48f8-9ab8-c361f433594c\") " pod="kube-system/global-pull-secret-syncer-t54tw"
Jul 20 15:27:07.125317 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:07.125302    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b5f0a63-5c26-49e1-a580-8b19d8a6e289-metrics-certs\") pod \"network-metrics-daemon-4kzlg\" (UID: \"7b5f0a63-5c26-49e1-a580-8b19d8a6e289\") " pod="openshift-multus/network-metrics-daemon-4kzlg"
Jul 20 15:27:07.125829 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:07.125402    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 15:27:07.125829 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:07.125416    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 15:27:07.125829 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:07.125470    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23b1e60d-e25e-48f8-9ab8-c361f433594c-original-pull-secret podName:23b1e60d-e25e-48f8-9ab8-c361f433594c nodeName:}" failed. No retries permitted until 2026-07-20 15:27:15.125451021 +0000 UTC m=+18.135072662 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/23b1e60d-e25e-48f8-9ab8-c361f433594c-original-pull-secret") pod "global-pull-secret-syncer-t54tw" (UID: "23b1e60d-e25e-48f8-9ab8-c361f433594c") : object "kube-system"/"original-pull-secret" not registered
Jul 20 15:27:07.125829 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:07.125485    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b5f0a63-5c26-49e1-a580-8b19d8a6e289-metrics-certs podName:7b5f0a63-5c26-49e1-a580-8b19d8a6e289 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:15.125478085 +0000 UTC m=+18.135099725 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b5f0a63-5c26-49e1-a580-8b19d8a6e289-metrics-certs") pod "network-metrics-daemon-4kzlg" (UID: "7b5f0a63-5c26-49e1-a580-8b19d8a6e289") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 15:27:07.227191 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:07.226589    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv85s\" (UniqueName: \"kubernetes.io/projected/0293426e-c324-435c-873e-468da9602ff6-kube-api-access-jv85s\") pod \"network-check-target-n7pqm\" (UID: \"0293426e-c324-435c-873e-468da9602ff6\") " pod="openshift-network-diagnostics/network-check-target-n7pqm"
Jul 20 15:27:07.227191 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:07.226772    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 15:27:07.227191 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:07.226794    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 15:27:07.227191 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:07.226806    2559 projected.go:196] Error preparing data for projected volume kube-api-access-jv85s for pod openshift-network-diagnostics/network-check-target-n7pqm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 15:27:07.227191 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:07.226862    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0293426e-c324-435c-873e-468da9602ff6-kube-api-access-jv85s podName:0293426e-c324-435c-873e-468da9602ff6 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:15.226844448 +0000 UTC m=+18.236466094 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-jv85s" (UniqueName: "kubernetes.io/projected/0293426e-c324-435c-873e-468da9602ff6-kube-api-access-jv85s") pod "network-check-target-n7pqm" (UID: "0293426e-c324-435c-873e-468da9602ff6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 15:27:07.510473 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:07.510389    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t54tw"
Jul 20 15:27:07.510626 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:07.510535    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kzlg"
Jul 20 15:27:07.510626 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:07.510559    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t54tw" podUID="23b1e60d-e25e-48f8-9ab8-c361f433594c"
Jul 20 15:27:07.510626 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:07.510603    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7pqm"
Jul 20 15:27:07.510787 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:07.510704    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7pqm" podUID="0293426e-c324-435c-873e-468da9602ff6"
Jul 20 15:27:07.510841 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:07.510812    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4kzlg" podUID="7b5f0a63-5c26-49e1-a580-8b19d8a6e289"
Jul 20 15:27:09.508470 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:09.508330    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kzlg"
Jul 20 15:27:09.508470 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:09.508352    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7pqm"
Jul 20 15:27:09.508470 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:09.508362    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t54tw"
Jul 20 15:27:09.508981 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:09.508481    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4kzlg" podUID="7b5f0a63-5c26-49e1-a580-8b19d8a6e289"
Jul 20 15:27:09.508981 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:09.508543    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7pqm" podUID="0293426e-c324-435c-873e-468da9602ff6"
Jul 20 15:27:09.508981 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:09.508616    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t54tw" podUID="23b1e60d-e25e-48f8-9ab8-c361f433594c"
Jul 20 15:27:11.509133 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:11.509100    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kzlg"
Jul 20 15:27:11.509640 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:11.509237    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t54tw"
Jul 20 15:27:11.509640 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:11.509249    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4kzlg" podUID="7b5f0a63-5c26-49e1-a580-8b19d8a6e289"
Jul 20 15:27:11.509640 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:11.509377    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t54tw" podUID="23b1e60d-e25e-48f8-9ab8-c361f433594c"
Jul 20 15:27:11.509640 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:11.509433    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7pqm"
Jul 20 15:27:11.509640 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:11.509512    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7pqm" podUID="0293426e-c324-435c-873e-468da9602ff6"
Jul 20 15:27:13.512089 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:13.512059    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7pqm"
Jul 20 15:27:13.512089 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:13.512059    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t54tw"
Jul 20 15:27:13.512089 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:13.512091    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kzlg"
Jul 20 15:27:13.512664 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:13.512174    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7pqm" podUID="0293426e-c324-435c-873e-468da9602ff6"
Jul 20 15:27:13.512664 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:13.512302    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4kzlg" podUID="7b5f0a63-5c26-49e1-a580-8b19d8a6e289"
Jul 20 15:27:13.512664 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:13.512372    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t54tw" podUID="23b1e60d-e25e-48f8-9ab8-c361f433594c"
Jul 20 15:27:15.188698 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:15.188669    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/23b1e60d-e25e-48f8-9ab8-c361f433594c-original-pull-secret\") pod \"global-pull-secret-syncer-t54tw\" (UID: \"23b1e60d-e25e-48f8-9ab8-c361f433594c\") " pod="kube-system/global-pull-secret-syncer-t54tw"
Jul 20 15:27:15.189187 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:15.188728    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b5f0a63-5c26-49e1-a580-8b19d8a6e289-metrics-certs\") pod \"network-metrics-daemon-4kzlg\" (UID: \"7b5f0a63-5c26-49e1-a580-8b19d8a6e289\") " pod="openshift-multus/network-metrics-daemon-4kzlg"
Jul 20 15:27:15.189187 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:15.188838    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 15:27:15.189187 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:15.188895    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23b1e60d-e25e-48f8-9ab8-c361f433594c-original-pull-secret podName:23b1e60d-e25e-48f8-9ab8-c361f433594c nodeName:}" failed. No retries permitted until 2026-07-20 15:27:31.188882224 +0000 UTC m=+34.198503869 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/23b1e60d-e25e-48f8-9ab8-c361f433594c-original-pull-secret") pod "global-pull-secret-syncer-t54tw" (UID: "23b1e60d-e25e-48f8-9ab8-c361f433594c") : object "kube-system"/"original-pull-secret" not registered
Jul 20 15:27:15.189187 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:15.188838    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 15:27:15.189187 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:15.188961    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b5f0a63-5c26-49e1-a580-8b19d8a6e289-metrics-certs podName:7b5f0a63-5c26-49e1-a580-8b19d8a6e289 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:31.188947367 +0000 UTC m=+34.198569021 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b5f0a63-5c26-49e1-a580-8b19d8a6e289-metrics-certs") pod "network-metrics-daemon-4kzlg" (UID: "7b5f0a63-5c26-49e1-a580-8b19d8a6e289") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 15:27:15.289408 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:15.289372    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv85s\" (UniqueName: \"kubernetes.io/projected/0293426e-c324-435c-873e-468da9602ff6-kube-api-access-jv85s\") pod \"network-check-target-n7pqm\" (UID: \"0293426e-c324-435c-873e-468da9602ff6\") " pod="openshift-network-diagnostics/network-check-target-n7pqm"
Jul 20 15:27:15.289575 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:15.289534    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 15:27:15.289575 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:15.289555    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 15:27:15.289575 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:15.289567    2559 projected.go:196] Error preparing data for projected volume kube-api-access-jv85s for pod openshift-network-diagnostics/network-check-target-n7pqm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 15:27:15.289691 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:15.289613    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0293426e-c324-435c-873e-468da9602ff6-kube-api-access-jv85s podName:0293426e-c324-435c-873e-468da9602ff6 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:31.289599755 +0000 UTC m=+34.299221395 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-jv85s" (UniqueName: "kubernetes.io/projected/0293426e-c324-435c-873e-468da9602ff6-kube-api-access-jv85s") pod "network-check-target-n7pqm" (UID: "0293426e-c324-435c-873e-468da9602ff6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 15:27:15.508454 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:15.508370    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t54tw"
Jul 20 15:27:15.508454 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:15.508408    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kzlg"
Jul 20 15:27:15.508665 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:15.508370    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7pqm"
Jul 20 15:27:15.508665 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:15.508499    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t54tw" podUID="23b1e60d-e25e-48f8-9ab8-c361f433594c"
Jul 20 15:27:15.508665 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:15.508580    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7pqm" podUID="0293426e-c324-435c-873e-468da9602ff6"
Jul 20 15:27:15.508791 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:15.508665    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4kzlg" podUID="7b5f0a63-5c26-49e1-a580-8b19d8a6e289"
Jul 20 15:27:17.509917 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:17.509882    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7pqm"
Jul 20 15:27:17.510408 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:17.509994    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t54tw"
Jul 20 15:27:17.510408 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:17.510019    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7pqm" podUID="0293426e-c324-435c-873e-468da9602ff6"
Jul 20 15:27:17.510408 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:17.510111    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t54tw" podUID="23b1e60d-e25e-48f8-9ab8-c361f433594c"
Jul 20 15:27:17.510408 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:17.510171    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kzlg"
Jul 20 15:27:17.510408 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:17.510270    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4kzlg" podUID="7b5f0a63-5c26-49e1-a580-8b19d8a6e289"
Jul 20 15:27:19.509464 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:19.509181    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7pqm"
Jul 20 15:27:19.510090 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:19.509180    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t54tw"
Jul 20 15:27:19.510090 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:19.509540    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7pqm" podUID="0293426e-c324-435c-873e-468da9602ff6"
Jul 20 15:27:19.510090 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:19.509646    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t54tw" podUID="23b1e60d-e25e-48f8-9ab8-c361f433594c"
Jul 20 15:27:19.510090 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:19.509187    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kzlg"
Jul 20 15:27:19.510090 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:19.509782    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4kzlg" podUID="7b5f0a63-5c26-49e1-a580-8b19d8a6e289"
Jul 20 15:27:19.612984 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:19.612956    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zqfwk" event={"ID":"f5208bb2-476e-43d5-b0aa-86c6383f6ec7","Type":"ContainerStarted","Data":"fc5afbe4b589fd122cc1cf177aaf60e63c608e22183429b7155ad35183cf4358"}
Jul 20 15:27:19.614224 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:19.614202    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2m28f" event={"ID":"f60e8181-bece-4bff-b8a3-4ad06b349d6c","Type":"ContainerStarted","Data":"cd9955049dd1ef3d9a663d11e46a1fd25292871174febf38c36ebc8ec3a863fd"}
Jul 20 15:27:19.615530 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:19.615500    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pqqrs" event={"ID":"b87d2983-fef9-4867-89c6-5f3fe76bd062","Type":"ContainerStarted","Data":"50795db65e357e1ffc56554df18b47fbcfbbc035cf6cc2b2ebd6beabf9db20aa"}
Jul 20 15:27:19.616903 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:19.616879    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9nxcn" event={"ID":"c26d7ae2-4424-47c6-b4cf-2815b1f353f8","Type":"ContainerStarted","Data":"9c715baca86fe5b75a5bd53e4988a58dcce831a441d69ad813364240b64e4d5f"}
Jul 20 15:27:19.618437 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:19.618415    2559 generic.go:356] "Generic (PLEG): container finished" podID="5edc4d01-8bc5-45b4-9ae3-ca95d913d778" containerID="7c1685c74f28f1ac0fb7277fcad7d3a643fd7f8da67253592c4c0c4b2e0d74d7" exitCode=0
Jul 20 15:27:19.618540 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:19.618493    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gpzpc" event={"ID":"5edc4d01-8bc5-45b4-9ae3-ca95d913d778","Type":"ContainerDied","Data":"7c1685c74f28f1ac0fb7277fcad7d3a643fd7f8da67253592c4c0c4b2e0d74d7"}
Jul 20 15:27:19.621119 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:19.621098    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz" event={"ID":"451bb108-e7e1-4ed7-9f73-f40794b199ee","Type":"ContainerStarted","Data":"f689ead06fe9393ecf6173a6d21056868593a8679b2a2df28bd8a92c7b5a093d"}
Jul 20 15:27:19.621222 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:19.621155    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz" event={"ID":"451bb108-e7e1-4ed7-9f73-f40794b199ee","Type":"ContainerStarted","Data":"324f2fb299d2ed8f97b02207c9e13e2cb057e5851dd20b3a4299806b3f9abff8"}
Jul 20 15:27:19.621222 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:19.621205    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz" event={"ID":"451bb108-e7e1-4ed7-9f73-f40794b199ee","Type":"ContainerStarted","Data":"8fa652f131b6dc985481014bbfca7a974e05b5f82edf7b7a93c9def4652a00a5"}
Jul 20 15:27:19.621414 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:19.621249    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz" event={"ID":"451bb108-e7e1-4ed7-9f73-f40794b199ee","Type":"ContainerStarted","Data":"43102e450709739bffb69232ae58475113a879ba0c55d19b4aaf077865cb212a"}
Jul 20 15:27:19.621414 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:19.621312    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz" event={"ID":"451bb108-e7e1-4ed7-9f73-f40794b199ee","Type":"ContainerStarted","Data":"824acdfec81845bb864b38d2a35fe818678e323bb7d2a98310409609bd5f92ad"}
Jul 20 15:27:19.622368 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:19.622348    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-j45gb" event={"ID":"ada5dd42-e780-4cc0-a845-ea417f1eac27","Type":"ContainerStarted","Data":"b51d6f7056a2b5465e9d688a5ad7c4444abf7702f673c0414e917f00953b251a"}
Jul 20 15:27:19.623685 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:19.623667    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tx286" event={"ID":"a2253ac6-88dd-4ed1-aa18-1279dd83bea8","Type":"ContainerStarted","Data":"efc7f3aa6ddebf426630c47397e7719561c66385248209d9ad2837c00524c222"}
Jul 20 15:27:19.633912 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:19.633876    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-2m28f" podStartSLOduration=12.428422376 podStartE2EDuration="22.633865294s" totalImagesPullingTime="10.205442918s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:26:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:27:19.633744621 +0000 UTC m=+22.643366283" watchObservedRunningTime="2026-07-20 15:27:19.633865294 +0000 UTC m=+22.643486956"
Jul 20 15:27:19.634052 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:19.634024    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.ec2.internal" podStartSLOduration=21.634017086 podStartE2EDuration="21.634017086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:26:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:27:02.596908759 +0000 UTC m=+5.606530423" watchObservedRunningTime="2026-07-20 15:27:19.634017086 +0000 UTC m=+22.643638748"
Jul 20 15:27:19.651388 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:19.651343    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-pqqrs" podStartSLOduration=4.085779742 podStartE2EDuration="22.651326097s" totalImagesPullingTime="18.565546355s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:26:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:27:19.651254475 +0000 UTC m=+22.660876137" watchObservedRunningTime="2026-07-20 15:27:19.651326097 +0000 UTC m=+22.660947760"
Jul 20 15:27:19.700302 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:19.700247    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-j45gb" podStartSLOduration=4.140885523 podStartE2EDuration="22.700235131s" totalImagesPullingTime="18.559349608s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:26:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:27:19.699949227 +0000 UTC m=+22.709570888" watchObservedRunningTime="2026-07-20 15:27:19.700235131 +0000 UTC m=+22.709856792"
Jul 20 15:27:19.724556 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:19.724458    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-tx286" podStartSLOduration=4.103883972 podStartE2EDuration="22.724442072s" totalImagesPullingTime="18.6205581s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:26:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:27:19.724144863 +0000 UTC m=+22.733766549" watchObservedRunningTime="2026-07-20 15:27:19.724442072 +0000 UTC m=+22.734063736"
Jul 20 15:27:20.470880 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:20.470808    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-2m28f"
Jul 20 15:27:20.471408 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:20.471394    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-2m28f"
Jul 20 15:27:20.491592 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:20.491553    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9nxcn" podStartSLOduration=4.918077801 podStartE2EDuration="23.491539498s" totalImagesPullingTime="18.573461697s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:26:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:27:19.745729796 +0000 UTC m=+22.755351459" watchObservedRunningTime="2026-07-20 15:27:20.491539498 +0000 UTC m=+23.501161159"
Jul 20 15:27:20.630599 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:20.630569    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz" event={"ID":"451bb108-e7e1-4ed7-9f73-f40794b199ee","Type":"ContainerStarted","Data":"e6043fe6c4f67f0d81bb5ab047f8863396f85ef8fc47f168db7db8e8521933ac"}
Jul 20 15:27:20.632655 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:20.632229    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-t2ckp" event={"ID":"dc8c759a-ed37-4c8d-bb52-86dfd95ea94b","Type":"ContainerStarted","Data":"f7df9d703d1aa8af31186e0b15ee465b485902bb3375b8a80a965b8a855823be"}
Jul 20 15:27:20.650619 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:20.650574    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-t2ckp" podStartSLOduration=5.08608455 podStartE2EDuration="23.650556895s" totalImagesPullingTime="18.564472345s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:26:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:27:20.649801281 +0000 UTC m=+23.659422949" watchObservedRunningTime="2026-07-20 15:27:20.650556895 +0000 UTC m=+23.660178557"
Jul 20 15:27:20.717253 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:20.717229    2559 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 20 15:27:21.419215 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:21.419120    2559 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-20T15:27:20.71724626Z","UUID":"f8d8bade-a753-47ec-9918-1f8588d5ae05","Handler":null,"Name":"","Endpoint":""}
Jul 20 15:27:21.420457 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:21.420437    2559 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 20 15:27:21.420566 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:21.420470    2559 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 20 15:27:21.509127 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:21.509080    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7pqm"
Jul 20 15:27:21.509306 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:21.509088    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t54tw"
Jul 20 15:27:21.509306 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:21.509196    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kzlg"
Jul 20 15:27:21.509306 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:21.509194    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7pqm" podUID="0293426e-c324-435c-873e-468da9602ff6"
Jul 20 15:27:21.509306 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:21.509264    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t54tw" podUID="23b1e60d-e25e-48f8-9ab8-c361f433594c"
Jul 20 15:27:21.509586 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:21.509359    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4kzlg" podUID="7b5f0a63-5c26-49e1-a580-8b19d8a6e289"
Jul 20 15:27:21.636754 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:21.636730    2559 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 20 15:27:21.636754 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:21.636745    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zqfwk" event={"ID":"f5208bb2-476e-43d5-b0aa-86c6383f6ec7","Type":"ContainerStarted","Data":"6a259ef684f9f9b0fa185b71c1ef8a393259d275aa1cf5b9eca6de3a448d2eff"}
Jul 20 15:27:22.642628 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:22.642422    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz" event={"ID":"451bb108-e7e1-4ed7-9f73-f40794b199ee","Type":"ContainerStarted","Data":"c91d77fdcaec4c23fd5ee3dcbdf1a5aaf473275541e37458d934fe8579cf4ff6"}
Jul 20 15:27:22.644215 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:22.644190    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zqfwk" event={"ID":"f5208bb2-476e-43d5-b0aa-86c6383f6ec7","Type":"ContainerStarted","Data":"215f130b9616a9415f5d754a76aee14205fb6c1189f8b9801e55b996d6186572"}
Jul 20 15:27:22.663548 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:22.663511    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zqfwk" podStartSLOduration=4.463136751 podStartE2EDuration="25.663499189s" totalImagesPullingTime="21.200362438s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:26:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:27:22.663215057 +0000 UTC m=+25.672836720" watchObservedRunningTime="2026-07-20 15:27:22.663499189 +0000 UTC m=+25.673120850"
Jul 20 15:27:23.509257 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:23.509226    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t54tw"
Jul 20 15:27:23.509449 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:23.509230    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7pqm"
Jul 20 15:27:23.509449 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:23.509376    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kzlg"
Jul 20 15:27:23.509449 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:23.509372    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t54tw" podUID="23b1e60d-e25e-48f8-9ab8-c361f433594c"
Jul 20 15:27:23.509449 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:23.509426    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7pqm" podUID="0293426e-c324-435c-873e-468da9602ff6"
Jul 20 15:27:23.509663 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:23.509562    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4kzlg" podUID="7b5f0a63-5c26-49e1-a580-8b19d8a6e289"
Jul 20 15:27:24.102587 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:24.102158    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-2m28f"
Jul 20 15:27:24.102587 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:24.102280    2559 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 20 15:27:24.103795 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:24.102721    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-2m28f"
Jul 20 15:27:24.653102 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:24.653071    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz" event={"ID":"451bb108-e7e1-4ed7-9f73-f40794b199ee","Type":"ContainerStarted","Data":"21baf4cfd75b2d7cfb61423e9ec6d6d83e92bca8f27d39a25e68ce4836687a94"}
Jul 20 15:27:25.508823 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:25.508620    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t54tw"
Jul 20 15:27:25.509413 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:25.508629    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kzlg"
Jul 20 15:27:25.509413 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:25.508923    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t54tw" podUID="23b1e60d-e25e-48f8-9ab8-c361f433594c"
Jul 20 15:27:25.509413 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:25.508681    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7pqm"
Jul 20 15:27:25.509413 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:25.509023    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4kzlg" podUID="7b5f0a63-5c26-49e1-a580-8b19d8a6e289"
Jul 20 15:27:25.509413 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:25.509068    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7pqm" podUID="0293426e-c324-435c-873e-468da9602ff6"
Jul 20 15:27:25.655658 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:25.655561    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:27:25.655658 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:25.655650    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:27:25.655856 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:25.655695    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:27:25.673692 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:25.673668    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:27:25.673845 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:25.673788    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:27:25.723667 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:25.723621    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz" podStartSLOduration=10.102832649 podStartE2EDuration="28.723606131s" totalImagesPullingTime="18.620773482s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:26:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:27:24.711087658 +0000 UTC m=+27.720709319" watchObservedRunningTime="2026-07-20 15:27:25.723606131 +0000 UTC m=+28.733227784"
Jul 20 15:27:26.658910 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:26.658875    2559 generic.go:356] "Generic (PLEG): container finished" podID="5edc4d01-8bc5-45b4-9ae3-ca95d913d778" containerID="bbb42692c5de04b1e8ced63ddfac7661ffa6b20e537c1d05a023edff5220c16e" exitCode=0
Jul 20 15:27:26.659349 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:26.658946    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gpzpc" event={"ID":"5edc4d01-8bc5-45b4-9ae3-ca95d913d778","Type":"ContainerDied","Data":"bbb42692c5de04b1e8ced63ddfac7661ffa6b20e537c1d05a023edff5220c16e"}
Jul 20 15:27:27.512909 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:27.512883    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t54tw"
Jul 20 15:27:27.513087 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:27.512883    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kzlg"
Jul 20 15:27:27.513242 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:27.513227    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7pqm"
Jul 20 15:27:27.513321 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:27.513274    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t54tw" podUID="23b1e60d-e25e-48f8-9ab8-c361f433594c"
Jul 20 15:27:27.513382 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:27.513358    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4kzlg" podUID="7b5f0a63-5c26-49e1-a580-8b19d8a6e289"
Jul 20 15:27:27.513491 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:27.513432    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7pqm" podUID="0293426e-c324-435c-873e-468da9602ff6"
Jul 20 15:27:27.560745 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:27.560711    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:27.683432 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:27.683404    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:27.790838 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:27.790805    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:27.954254 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:27.954225    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:28.147425 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:28.147365    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:28.265945 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:28.265913    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:28.539887 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:28.539861    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:28.664501 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:28.664477    2559 generic.go:356] "Generic (PLEG): container finished" podID="5edc4d01-8bc5-45b4-9ae3-ca95d913d778" containerID="43baad810f0d74fe13b551925bf324d75ff93558fc615c8772f9d7572c67dc50" exitCode=0
Jul 20 15:27:28.664604 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:28.664528    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gpzpc" event={"ID":"5edc4d01-8bc5-45b4-9ae3-ca95d913d778","Type":"ContainerDied","Data":"43baad810f0d74fe13b551925bf324d75ff93558fc615c8772f9d7572c67dc50"}
Jul 20 15:27:28.691195 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:28.691170    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:28.979170 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:28.979084    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:29.453107 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:29.453078    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:29.509048 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:29.508977    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7pqm"
Jul 20 15:27:29.509048 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:29.508987    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t54tw"
Jul 20 15:27:29.509048 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:29.508987    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kzlg"
Jul 20 15:27:29.509270 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:29.509086    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7pqm" podUID="0293426e-c324-435c-873e-468da9602ff6"
Jul 20 15:27:29.509270 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:29.509138    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t54tw" podUID="23b1e60d-e25e-48f8-9ab8-c361f433594c"
Jul 20 15:27:29.509270 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:29.509217    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4kzlg" podUID="7b5f0a63-5c26-49e1-a580-8b19d8a6e289"
Jul 20 15:27:29.668625 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:29.668597    2559 generic.go:356] "Generic (PLEG): container finished" podID="5edc4d01-8bc5-45b4-9ae3-ca95d913d778" containerID="ddf0bc4d63f2b6a63be08b2ba885d67f01b68c36b60a55ac480be1c0919bc4c3" exitCode=0
Jul 20 15:27:29.668792 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:29.668664    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gpzpc" event={"ID":"5edc4d01-8bc5-45b4-9ae3-ca95d913d778","Type":"ContainerDied","Data":"ddf0bc4d63f2b6a63be08b2ba885d67f01b68c36b60a55ac480be1c0919bc4c3"}
Jul 20 15:27:29.922674 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:29.922645    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:30.144497 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:30.144468    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:30.352269 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:30.352232    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:30.569372 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:30.569343    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:30.783361 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:30.783322    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:31.000769 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:31.000742    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:31.154416 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:31.154338    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:31.211458 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:31.211428    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b5f0a63-5c26-49e1-a580-8b19d8a6e289-metrics-certs\") pod \"network-metrics-daemon-4kzlg\" (UID: \"7b5f0a63-5c26-49e1-a580-8b19d8a6e289\") " pod="openshift-multus/network-metrics-daemon-4kzlg"
Jul 20 15:27:31.211620 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:31.211498    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/23b1e60d-e25e-48f8-9ab8-c361f433594c-original-pull-secret\") pod \"global-pull-secret-syncer-t54tw\" (UID: \"23b1e60d-e25e-48f8-9ab8-c361f433594c\") " pod="kube-system/global-pull-secret-syncer-t54tw"
Jul 20 15:27:31.211620 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:31.211565    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 15:27:31.211620 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:31.211595    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 15:27:31.211768 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:31.211636    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b5f0a63-5c26-49e1-a580-8b19d8a6e289-metrics-certs podName:7b5f0a63-5c26-49e1-a580-8b19d8a6e289 nodeName:}" failed. No retries permitted until 2026-07-20 15:28:03.211616925 +0000 UTC m=+66.221238585 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b5f0a63-5c26-49e1-a580-8b19d8a6e289-metrics-certs") pod "network-metrics-daemon-4kzlg" (UID: "7b5f0a63-5c26-49e1-a580-8b19d8a6e289") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 15:27:31.211768 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:31.211656    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23b1e60d-e25e-48f8-9ab8-c361f433594c-original-pull-secret podName:23b1e60d-e25e-48f8-9ab8-c361f433594c nodeName:}" failed. No retries permitted until 2026-07-20 15:28:03.211647314 +0000 UTC m=+66.221268954 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/23b1e60d-e25e-48f8-9ab8-c361f433594c-original-pull-secret") pod "global-pull-secret-syncer-t54tw" (UID: "23b1e60d-e25e-48f8-9ab8-c361f433594c") : object "kube-system"/"original-pull-secret" not registered
Jul 20 15:27:31.312687 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:31.312655    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv85s\" (UniqueName: \"kubernetes.io/projected/0293426e-c324-435c-873e-468da9602ff6-kube-api-access-jv85s\") pod \"network-check-target-n7pqm\" (UID: \"0293426e-c324-435c-873e-468da9602ff6\") " pod="openshift-network-diagnostics/network-check-target-n7pqm"
Jul 20 15:27:31.312836 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:31.312805    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 15:27:31.312836 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:31.312822    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 15:27:31.312836 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:31.312832    2559 projected.go:196] Error preparing data for projected volume kube-api-access-jv85s for pod openshift-network-diagnostics/network-check-target-n7pqm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 15:27:31.312983 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:31.312886    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0293426e-c324-435c-873e-468da9602ff6-kube-api-access-jv85s podName:0293426e-c324-435c-873e-468da9602ff6 nodeName:}" failed. No retries permitted until 2026-07-20 15:28:03.312870115 +0000 UTC m=+66.322491774 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-jv85s" (UniqueName: "kubernetes.io/projected/0293426e-c324-435c-873e-468da9602ff6-kube-api-access-jv85s") pod "network-check-target-n7pqm" (UID: "0293426e-c324-435c-873e-468da9602ff6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 15:27:31.446480 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:31.446403    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:31.508851 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:31.508743    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7pqm"
Jul 20 15:27:31.508851 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:31.508763    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t54tw"
Jul 20 15:27:31.509730 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:31.508974    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7pqm" podUID="0293426e-c324-435c-873e-468da9602ff6"
Jul 20 15:27:31.509730 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:31.509110    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t54tw" podUID="23b1e60d-e25e-48f8-9ab8-c361f433594c"
Jul 20 15:27:31.509730 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:31.509398    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kzlg"
Jul 20 15:27:31.509980 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:31.509604    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4kzlg" podUID="7b5f0a63-5c26-49e1-a580-8b19d8a6e289"
Jul 20 15:27:31.586733 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:31.586704    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:31.788900 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:31.788849    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-n7pqm"]
Jul 20 15:27:31.789068 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:31.788966    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7pqm"
Jul 20 15:27:31.789136 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:31.789065    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7pqm" podUID="0293426e-c324-435c-873e-468da9602ff6"
Jul 20 15:27:31.790964 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:31.790937    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-t54tw"]
Jul 20 15:27:31.791067 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:31.791031    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t54tw"
Jul 20 15:27:31.791150 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:31.791128    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t54tw" podUID="23b1e60d-e25e-48f8-9ab8-c361f433594c"
Jul 20 15:27:31.795703 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:31.795649    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4kzlg"]
Jul 20 15:27:31.796008 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:31.795740    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kzlg"
Jul 20 15:27:31.796008 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:31.795847    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4kzlg" podUID="7b5f0a63-5c26-49e1-a580-8b19d8a6e289"
Jul 20 15:27:33.512121 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:33.512089    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t54tw"
Jul 20 15:27:33.512583 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:33.512194    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7pqm"
Jul 20 15:27:33.512583 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:33.512206    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t54tw" podUID="23b1e60d-e25e-48f8-9ab8-c361f433594c"
Jul 20 15:27:33.512583 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:33.512222    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kzlg"
Jul 20 15:27:33.512583 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:33.512304    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7pqm" podUID="0293426e-c324-435c-873e-468da9602ff6"
Jul 20 15:27:33.512583 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:33.512380    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4kzlg" podUID="7b5f0a63-5c26-49e1-a580-8b19d8a6e289"
Jul 20 15:27:34.683202 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:34.683035    2559 generic.go:356] "Generic (PLEG): container finished" podID="5edc4d01-8bc5-45b4-9ae3-ca95d913d778" containerID="869081fbe093b66a805c78eecbcb8e51ec5573a26c6e2d83762ec49804e78456" exitCode=0
Jul 20 15:27:34.683614 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:34.683121    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gpzpc" event={"ID":"5edc4d01-8bc5-45b4-9ae3-ca95d913d778","Type":"ContainerDied","Data":"869081fbe093b66a805c78eecbcb8e51ec5573a26c6e2d83762ec49804e78456"}
Jul 20 15:27:35.509065 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:35.509042    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t54tw"
Jul 20 15:27:35.509167 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:35.509042    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7pqm"
Jul 20 15:27:35.509167 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:35.509157    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kzlg"
Jul 20 15:27:35.509228 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:35.509200    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t54tw" podUID="23b1e60d-e25e-48f8-9ab8-c361f433594c"
Jul 20 15:27:35.509261 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:35.509234    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7pqm" podUID="0293426e-c324-435c-873e-468da9602ff6"
Jul 20 15:27:35.509369 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:35.509349    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4kzlg" podUID="7b5f0a63-5c26-49e1-a580-8b19d8a6e289"
Jul 20 15:27:35.691744 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:35.691707    2559 generic.go:356] "Generic (PLEG): container finished" podID="5edc4d01-8bc5-45b4-9ae3-ca95d913d778" containerID="8ef13cb0e407263df38187b9d423a5a2b1a7cbaeb99fe2d2e008cad121dc4eca" exitCode=0
Jul 20 15:27:35.692258 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:35.691744    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gpzpc" event={"ID":"5edc4d01-8bc5-45b4-9ae3-ca95d913d778","Type":"ContainerDied","Data":"8ef13cb0e407263df38187b9d423a5a2b1a7cbaeb99fe2d2e008cad121dc4eca"}
Jul 20 15:27:35.876986 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:35.876964    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-19.ec2.internal" event="NodeReady"
Jul 20 15:27:35.877116 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:35.877070    2559 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 20 15:27:35.911049 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:35.911023    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6cb967f748-tsjhw"]
Jul 20 15:27:35.922327 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:35.922303    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cb967f748-tsjhw"
Jul 20 15:27:35.925039 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:35.924965    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 20 15:27:35.925039 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:35.924965    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 20 15:27:35.925039 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:35.925013    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xvlmx\""
Jul 20 15:27:35.925244 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:35.925079    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 20 15:27:35.928204 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:35.928054    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6cb967f748-tsjhw"]
Jul 20 15:27:35.931346 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:35.931083    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 20 15:27:35.931993 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:35.931974    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2bz87"]
Jul 20 15:27:35.942196 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:35.942177    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-fxdt4"]
Jul 20 15:27:35.942320 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:35.942304    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2bz87"
Jul 20 15:27:35.945159 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:35.945137    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 20 15:27:35.945311 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:35.945275    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7xj82\""
Jul 20 15:27:35.945386 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:35.945372    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 20 15:27:35.952269 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:35.952253    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fxdt4"]
Jul 20 15:27:35.952386 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:35.952279    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2bz87"]
Jul 20 15:27:35.952386 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:35.952368    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fxdt4"
Jul 20 15:27:35.955017 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:35.954994    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 20 15:27:35.955106 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:35.955030    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 20 15:27:35.955106 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:35.955030    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-qzpxj\""
Jul 20 15:27:35.955263 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:35.955249    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 20 15:27:36.052164 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.052140    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/87c5f3e8-bc71-446b-9098-047de625ee8d-ca-trust-extracted\") pod \"image-registry-6cb967f748-tsjhw\" (UID: \"87c5f3e8-bc71-446b-9098-047de625ee8d\") " pod="openshift-image-registry/image-registry-6cb967f748-tsjhw"
Jul 20 15:27:36.052325 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.052172    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ca3c4e74-af1d-4595-aea5-19ebd06b7263-config-volume\") pod \"dns-default-2bz87\" (UID: \"ca3c4e74-af1d-4595-aea5-19ebd06b7263\") " pod="openshift-dns/dns-default-2bz87"
Jul 20 15:27:36.052325 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.052243    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/87c5f3e8-bc71-446b-9098-047de625ee8d-trusted-ca\") pod \"image-registry-6cb967f748-tsjhw\" (UID: \"87c5f3e8-bc71-446b-9098-047de625ee8d\") " pod="openshift-image-registry/image-registry-6cb967f748-tsjhw"
Jul 20 15:27:36.052325 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.052272    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/144b485c-ce5b-4ac5-82c7-28f6fa5320af-cert\") pod \"ingress-canary-fxdt4\" (UID: \"144b485c-ce5b-4ac5-82c7-28f6fa5320af\") " pod="openshift-ingress-canary/ingress-canary-fxdt4"
Jul 20 15:27:36.052325 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.052310    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcclf\" (UniqueName: \"kubernetes.io/projected/144b485c-ce5b-4ac5-82c7-28f6fa5320af-kube-api-access-hcclf\") pod \"ingress-canary-fxdt4\" (UID: \"144b485c-ce5b-4ac5-82c7-28f6fa5320af\") " pod="openshift-ingress-canary/ingress-canary-fxdt4"
Jul 20 15:27:36.052463 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.052328    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/87c5f3e8-bc71-446b-9098-047de625ee8d-bound-sa-token\") pod \"image-registry-6cb967f748-tsjhw\" (UID: \"87c5f3e8-bc71-446b-9098-047de625ee8d\") " pod="openshift-image-registry/image-registry-6cb967f748-tsjhw"
Jul 20 15:27:36.052463 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.052344    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/87c5f3e8-bc71-446b-9098-047de625ee8d-registry-tls\") pod \"image-registry-6cb967f748-tsjhw\" (UID: \"87c5f3e8-bc71-446b-9098-047de625ee8d\") " pod="openshift-image-registry/image-registry-6cb967f748-tsjhw"
Jul 20 15:27:36.052463 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.052359    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/87c5f3e8-bc71-446b-9098-047de625ee8d-registry-certificates\") pod \"image-registry-6cb967f748-tsjhw\" (UID: \"87c5f3e8-bc71-446b-9098-047de625ee8d\") " pod="openshift-image-registry/image-registry-6cb967f748-tsjhw"
Jul 20 15:27:36.052463 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.052373    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ca3c4e74-af1d-4595-aea5-19ebd06b7263-metrics-tls\") pod \"dns-default-2bz87\" (UID: \"ca3c4e74-af1d-4595-aea5-19ebd06b7263\") " pod="openshift-dns/dns-default-2bz87"
Jul 20 15:27:36.052463 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.052400    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/87c5f3e8-bc71-446b-9098-047de625ee8d-image-registry-private-configuration\") pod \"image-registry-6cb967f748-tsjhw\" (UID: \"87c5f3e8-bc71-446b-9098-047de625ee8d\") " pod="openshift-image-registry/image-registry-6cb967f748-tsjhw"
Jul 20 15:27:36.052463 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.052431    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/87c5f3e8-bc71-446b-9098-047de625ee8d-installation-pull-secrets\") pod \"image-registry-6cb967f748-tsjhw\" (UID: \"87c5f3e8-bc71-446b-9098-047de625ee8d\") " pod="openshift-image-registry/image-registry-6cb967f748-tsjhw"
Jul 20 15:27:36.052463 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.052449    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bb8zv\" (UniqueName: \"kubernetes.io/projected/87c5f3e8-bc71-446b-9098-047de625ee8d-kube-api-access-bb8zv\") pod \"image-registry-6cb967f748-tsjhw\" (UID: \"87c5f3e8-bc71-446b-9098-047de625ee8d\") " pod="openshift-image-registry/image-registry-6cb967f748-tsjhw"
Jul 20 15:27:36.052463 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.052463    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca3c4e74-af1d-4595-aea5-19ebd06b7263-tmp-dir\") pod \"dns-default-2bz87\" (UID: \"ca3c4e74-af1d-4595-aea5-19ebd06b7263\") " pod="openshift-dns/dns-default-2bz87"
Jul 20 15:27:36.052683 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.052489    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhv4m\" (UniqueName: \"kubernetes.io/projected/ca3c4e74-af1d-4595-aea5-19ebd06b7263-kube-api-access-mhv4m\") pod \"dns-default-2bz87\" (UID: \"ca3c4e74-af1d-4595-aea5-19ebd06b7263\") " pod="openshift-dns/dns-default-2bz87"
Jul 20 15:27:36.070255 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.070227    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:36.152906 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.152878    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/87c5f3e8-bc71-446b-9098-047de625ee8d-ca-trust-extracted\") pod \"image-registry-6cb967f748-tsjhw\" (UID: \"87c5f3e8-bc71-446b-9098-047de625ee8d\") " pod="openshift-image-registry/image-registry-6cb967f748-tsjhw"
Jul 20 15:27:36.153034 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.152909    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ca3c4e74-af1d-4595-aea5-19ebd06b7263-config-volume\") pod \"dns-default-2bz87\" (UID: \"ca3c4e74-af1d-4595-aea5-19ebd06b7263\") " pod="openshift-dns/dns-default-2bz87"
Jul 20 15:27:36.153034 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.152950    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/87c5f3e8-bc71-446b-9098-047de625ee8d-trusted-ca\") pod \"image-registry-6cb967f748-tsjhw\" (UID: \"87c5f3e8-bc71-446b-9098-047de625ee8d\") " pod="openshift-image-registry/image-registry-6cb967f748-tsjhw"
Jul 20 15:27:36.153034 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.152976    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/144b485c-ce5b-4ac5-82c7-28f6fa5320af-cert\") pod \"ingress-canary-fxdt4\" (UID: \"144b485c-ce5b-4ac5-82c7-28f6fa5320af\") " pod="openshift-ingress-canary/ingress-canary-fxdt4"
Jul 20 15:27:36.153034 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.152999    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcclf\" (UniqueName: \"kubernetes.io/projected/144b485c-ce5b-4ac5-82c7-28f6fa5320af-kube-api-access-hcclf\") pod \"ingress-canary-fxdt4\" (UID: \"144b485c-ce5b-4ac5-82c7-28f6fa5320af\") " pod="openshift-ingress-canary/ingress-canary-fxdt4"
Jul 20 15:27:36.153034 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.153019    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/87c5f3e8-bc71-446b-9098-047de625ee8d-bound-sa-token\") pod \"image-registry-6cb967f748-tsjhw\" (UID: \"87c5f3e8-bc71-446b-9098-047de625ee8d\") " pod="openshift-image-registry/image-registry-6cb967f748-tsjhw"
Jul 20 15:27:36.153270 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.153045    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/87c5f3e8-bc71-446b-9098-047de625ee8d-registry-tls\") pod \"image-registry-6cb967f748-tsjhw\" (UID: \"87c5f3e8-bc71-446b-9098-047de625ee8d\") " pod="openshift-image-registry/image-registry-6cb967f748-tsjhw"
Jul 20 15:27:36.153270 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.153069    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/87c5f3e8-bc71-446b-9098-047de625ee8d-registry-certificates\") pod \"image-registry-6cb967f748-tsjhw\" (UID: \"87c5f3e8-bc71-446b-9098-047de625ee8d\") " pod="openshift-image-registry/image-registry-6cb967f748-tsjhw"
Jul 20 15:27:36.153407 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.153328    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ca3c4e74-af1d-4595-aea5-19ebd06b7263-metrics-tls\") pod \"dns-default-2bz87\" (UID: \"ca3c4e74-af1d-4595-aea5-19ebd06b7263\") " pod="openshift-dns/dns-default-2bz87"
Jul 20 15:27:36.153407 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.153370    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/87c5f3e8-bc71-446b-9098-047de625ee8d-ca-trust-extracted\") pod \"image-registry-6cb967f748-tsjhw\" (UID: \"87c5f3e8-bc71-446b-9098-047de625ee8d\") " pod="openshift-image-registry/image-registry-6cb967f748-tsjhw"
Jul 20 15:27:36.153407 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.153387    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/87c5f3e8-bc71-446b-9098-047de625ee8d-image-registry-private-configuration\") pod \"image-registry-6cb967f748-tsjhw\" (UID: \"87c5f3e8-bc71-446b-9098-047de625ee8d\") " pod="openshift-image-registry/image-registry-6cb967f748-tsjhw"
Jul 20 15:27:36.153545 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.153431    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/87c5f3e8-bc71-446b-9098-047de625ee8d-installation-pull-secrets\") pod \"image-registry-6cb967f748-tsjhw\" (UID: \"87c5f3e8-bc71-446b-9098-047de625ee8d\") " pod="openshift-image-registry/image-registry-6cb967f748-tsjhw"
Jul 20 15:27:36.153545 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.153459    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bb8zv\" (UniqueName: \"kubernetes.io/projected/87c5f3e8-bc71-446b-9098-047de625ee8d-kube-api-access-bb8zv\") pod \"image-registry-6cb967f748-tsjhw\" (UID: \"87c5f3e8-bc71-446b-9098-047de625ee8d\") " pod="openshift-image-registry/image-registry-6cb967f748-tsjhw"
Jul 20 15:27:36.153545 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.153483    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca3c4e74-af1d-4595-aea5-19ebd06b7263-tmp-dir\") pod \"dns-default-2bz87\" (UID: \"ca3c4e74-af1d-4595-aea5-19ebd06b7263\") " pod="openshift-dns/dns-default-2bz87"
Jul 20 15:27:36.153545 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.153507    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhv4m\" (UniqueName: \"kubernetes.io/projected/ca3c4e74-af1d-4595-aea5-19ebd06b7263-kube-api-access-mhv4m\") pod \"dns-default-2bz87\" (UID: \"ca3c4e74-af1d-4595-aea5-19ebd06b7263\") " pod="openshift-dns/dns-default-2bz87"
Jul 20 15:27:36.153723 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.153619    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ca3c4e74-af1d-4595-aea5-19ebd06b7263-config-volume\") pod \"dns-default-2bz87\" (UID: \"ca3c4e74-af1d-4595-aea5-19ebd06b7263\") " pod="openshift-dns/dns-default-2bz87"
Jul 20 15:27:36.154113 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.154048    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/87c5f3e8-bc71-446b-9098-047de625ee8d-trusted-ca\") pod \"image-registry-6cb967f748-tsjhw\" (UID: \"87c5f3e8-bc71-446b-9098-047de625ee8d\") " pod="openshift-image-registry/image-registry-6cb967f748-tsjhw"
Jul 20 15:27:36.154113 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.154067    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca3c4e74-af1d-4595-aea5-19ebd06b7263-tmp-dir\") pod \"dns-default-2bz87\" (UID: \"ca3c4e74-af1d-4595-aea5-19ebd06b7263\") " pod="openshift-dns/dns-default-2bz87"
Jul 20 15:27:36.154535 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.154500    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/87c5f3e8-bc71-446b-9098-047de625ee8d-registry-certificates\") pod \"image-registry-6cb967f748-tsjhw\" (UID: \"87c5f3e8-bc71-446b-9098-047de625ee8d\") " pod="openshift-image-registry/image-registry-6cb967f748-tsjhw"
Jul 20 15:27:36.157488 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.157466    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ca3c4e74-af1d-4595-aea5-19ebd06b7263-metrics-tls\") pod \"dns-default-2bz87\" (UID: \"ca3c4e74-af1d-4595-aea5-19ebd06b7263\") " pod="openshift-dns/dns-default-2bz87"
Jul 20 15:27:36.157592 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.157551    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/87c5f3e8-bc71-446b-9098-047de625ee8d-installation-pull-secrets\") pod \"image-registry-6cb967f748-tsjhw\" (UID: \"87c5f3e8-bc71-446b-9098-047de625ee8d\") " pod="openshift-image-registry/image-registry-6cb967f748-tsjhw"
Jul 20 15:27:36.157637 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.157587    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/144b485c-ce5b-4ac5-82c7-28f6fa5320af-cert\") pod \"ingress-canary-fxdt4\" (UID: \"144b485c-ce5b-4ac5-82c7-28f6fa5320af\") " pod="openshift-ingress-canary/ingress-canary-fxdt4"
Jul 20 15:27:36.157637 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.157610    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/87c5f3e8-bc71-446b-9098-047de625ee8d-registry-tls\") pod \"image-registry-6cb967f748-tsjhw\" (UID: \"87c5f3e8-bc71-446b-9098-047de625ee8d\") " pod="openshift-image-registry/image-registry-6cb967f748-tsjhw"
Jul 20 15:27:36.157701 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.157634    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/87c5f3e8-bc71-446b-9098-047de625ee8d-image-registry-private-configuration\") pod \"image-registry-6cb967f748-tsjhw\" (UID: \"87c5f3e8-bc71-446b-9098-047de625ee8d\") " pod="openshift-image-registry/image-registry-6cb967f748-tsjhw"
Jul 20 15:27:36.161393 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.161365    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhv4m\" (UniqueName: \"kubernetes.io/projected/ca3c4e74-af1d-4595-aea5-19ebd06b7263-kube-api-access-mhv4m\") pod \"dns-default-2bz87\" (UID: \"ca3c4e74-af1d-4595-aea5-19ebd06b7263\") " pod="openshift-dns/dns-default-2bz87"
Jul 20 15:27:36.161718 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.161684    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bb8zv\" (UniqueName: \"kubernetes.io/projected/87c5f3e8-bc71-446b-9098-047de625ee8d-kube-api-access-bb8zv\") pod \"image-registry-6cb967f748-tsjhw\" (UID: \"87c5f3e8-bc71-446b-9098-047de625ee8d\") " pod="openshift-image-registry/image-registry-6cb967f748-tsjhw"
Jul 20 15:27:36.161979 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.161961    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/87c5f3e8-bc71-446b-9098-047de625ee8d-bound-sa-token\") pod \"image-registry-6cb967f748-tsjhw\" (UID: \"87c5f3e8-bc71-446b-9098-047de625ee8d\") " pod="openshift-image-registry/image-registry-6cb967f748-tsjhw"
Jul 20 15:27:36.162052 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.162034    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcclf\" (UniqueName: \"kubernetes.io/projected/144b485c-ce5b-4ac5-82c7-28f6fa5320af-kube-api-access-hcclf\") pod \"ingress-canary-fxdt4\" (UID: \"144b485c-ce5b-4ac5-82c7-28f6fa5320af\") " pod="openshift-ingress-canary/ingress-canary-fxdt4"
Jul 20 15:27:36.169966 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.169923    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:36.234894 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.234860    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cb967f748-tsjhw"
Jul 20 15:27:36.252015 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.251995    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2bz87"
Jul 20 15:27:36.260205 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.260191    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fxdt4"
Jul 20 15:27:36.463632 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.463427    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2bz87"]
Jul 20 15:27:36.466099 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:27:36.466074    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca3c4e74_af1d_4595_aea5_19ebd06b7263.slice/crio-b7ba6ebc7947c0ac4577b35f2bd765c70da7df8e0c2cc246b0de73e10db06ec3 WatchSource:0}: Error finding container b7ba6ebc7947c0ac4577b35f2bd765c70da7df8e0c2cc246b0de73e10db06ec3: Status 404 returned error can't find the container with id b7ba6ebc7947c0ac4577b35f2bd765c70da7df8e0c2cc246b0de73e10db06ec3
Jul 20 15:27:36.471691 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.471669    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6cb967f748-tsjhw"]
Jul 20 15:27:36.474322 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.474297    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fxdt4"]
Jul 20 15:27:36.474756 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:27:36.474734    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87c5f3e8_bc71_446b_9098_047de625ee8d.slice/crio-29b8493e05b53a3cc6f53c0d979986508e35acf77ac49eac494c2963cb8c8991 WatchSource:0}: Error finding container 29b8493e05b53a3cc6f53c0d979986508e35acf77ac49eac494c2963cb8c8991: Status 404 returned error can't find the container with id 29b8493e05b53a3cc6f53c0d979986508e35acf77ac49eac494c2963cb8c8991
Jul 20 15:27:36.477073 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:27:36.477055    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod144b485c_ce5b_4ac5_82c7_28f6fa5320af.slice/crio-96f0fcd4fe9c5ac79629013e86e157a991fdef607e85e66f37eaae44ba4c5514 WatchSource:0}: Error finding container 96f0fcd4fe9c5ac79629013e86e157a991fdef607e85e66f37eaae44ba4c5514: Status 404 returned error can't find the container with id 96f0fcd4fe9c5ac79629013e86e157a991fdef607e85e66f37eaae44ba4c5514
Jul 20 15:27:36.695494 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.695463    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fxdt4" event={"ID":"144b485c-ce5b-4ac5-82c7-28f6fa5320af","Type":"ContainerStarted","Data":"96f0fcd4fe9c5ac79629013e86e157a991fdef607e85e66f37eaae44ba4c5514"}
Jul 20 15:27:36.696590 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.696566    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2bz87" event={"ID":"ca3c4e74-af1d-4595-aea5-19ebd06b7263","Type":"ContainerStarted","Data":"b7ba6ebc7947c0ac4577b35f2bd765c70da7df8e0c2cc246b0de73e10db06ec3"}
Jul 20 15:27:36.702495 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.702468    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gpzpc" event={"ID":"5edc4d01-8bc5-45b4-9ae3-ca95d913d778","Type":"ContainerStarted","Data":"27da2d1cf0d919d60b7506fcf474b10309319cdeaa09ccfdd7f330312b389f86"}
Jul 20 15:27:36.703770 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.703751    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cb967f748-tsjhw" event={"ID":"87c5f3e8-bc71-446b-9098-047de625ee8d","Type":"ContainerStarted","Data":"1099543ae86732af4a114bd3b9e2429f51515fb3de0d9f9dc9eb0420e60fff66"}
Jul 20 15:27:36.703878 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.703785    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cb967f748-tsjhw" event={"ID":"87c5f3e8-bc71-446b-9098-047de625ee8d","Type":"ContainerStarted","Data":"29b8493e05b53a3cc6f53c0d979986508e35acf77ac49eac494c2963cb8c8991"}
Jul 20 15:27:36.703878 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.703851    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6cb967f748-tsjhw"
Jul 20 15:27:36.725528 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.725489    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-gpzpc" podStartSLOduration=7.998399772 podStartE2EDuration="39.725475419s" totalImagesPullingTime="31.727075647s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:26:57 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:27:36.72375907 +0000 UTC m=+39.733380731" watchObservedRunningTime="2026-07-20 15:27:36.725475419 +0000 UTC m=+39.735097104"
Jul 20 15:27:36.748095 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.748053    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6cb967f748-tsjhw" podStartSLOduration=6.748042302 podStartE2EDuration="6.748042302s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:27:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:27:36.748005416 +0000 UTC m=+39.757627102" watchObservedRunningTime="2026-07-20 15:27:36.748042302 +0000 UTC m=+39.757663943"
Jul 20 15:27:36.850222 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.850142    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:36.929630 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:36.929593    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:37.149168 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:37.149086    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:37.249366 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:37.249329    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:37.512648 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:37.512559    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t54tw"
Jul 20 15:27:37.513150 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:37.512872    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7pqm"
Jul 20 15:27:37.513150 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:37.513024    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kzlg"
Jul 20 15:27:37.515441 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:37.515368    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 20 15:27:37.516663 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:37.516642    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zgzvz\""
Jul 20 15:27:37.516738 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:37.516660    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 20 15:27:37.517129 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:37.516963    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 20 15:27:37.517129 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:37.517035    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 20 15:27:37.517129 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:37.517076    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tnhrs\""
Jul 20 15:27:37.654361 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:37.654325    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:37.798102 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:37.798057    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:37.898993 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:37.898963    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:38.002511 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:38.002467    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:38.188459 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:38.188375    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:38.241779 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:38.241747    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:38.681887 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:38.681850    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:38.780953 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:38.780919    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:38.882069 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:38.882031    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:38.994088 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:38.993992    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:39.049090 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:39.049054    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:39.418433 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:39.418402    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:39.518837 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:39.518809    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:39.634442 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:39.634414    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:39.712667 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:39.712642    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fxdt4" event={"ID":"144b485c-ce5b-4ac5-82c7-28f6fa5320af","Type":"ContainerStarted","Data":"6711be89f1583f37ef43df6f73cf87e620c24c95a3cdc319b1bfadecb2269b71"}
Jul 20 15:27:39.714115 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:39.714095    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2bz87" event={"ID":"ca3c4e74-af1d-4595-aea5-19ebd06b7263","Type":"ContainerStarted","Data":"7290e4e9a45be36f287d9502b170ec1022fcdce7282749e53a3b9c340478617e"}
Jul 20 15:27:39.812575 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:39.812548    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:39.943712 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:39.943678    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:40.045147 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:40.045118    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:40.214167 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:40.214131    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:40.475646 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:40.475569    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-gd8kb"]
Jul 20 15:27:40.492531 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:40.492512    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gd8kb"
Jul 20 15:27:40.496261 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:40.496239    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 20 15:27:40.496390 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:40.496305    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 20 15:27:40.496441 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:40.496248    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 20 15:27:40.497364 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:40.497345    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9rlnk\""
Jul 20 15:27:40.497473 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:40.497429    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:40.497656 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:40.497622    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 20 15:27:40.497800 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:40.497785    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gd8kb"]
Jul 20 15:27:40.570636 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:40.570599    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:40.592662 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:40.592633    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-crio-socket\") pod \"insights-runtime-extractor-gd8kb\" (UID: \"1797129a-35ab-4364-98a7-7d6dc8f8e6b1\") " pod="openshift-insights/insights-runtime-extractor-gd8kb"
Jul 20 15:27:40.592662 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:40.592661    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gd8kb\" (UID: \"1797129a-35ab-4364-98a7-7d6dc8f8e6b1\") " pod="openshift-insights/insights-runtime-extractor-gd8kb"
Jul 20 15:27:40.592795 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:40.592681    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gd8kb\" (UID: \"1797129a-35ab-4364-98a7-7d6dc8f8e6b1\") " pod="openshift-insights/insights-runtime-extractor-gd8kb"
Jul 20 15:27:40.592795 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:40.592720    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfxnv\" (UniqueName: \"kubernetes.io/projected/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-kube-api-access-jfxnv\") pod \"insights-runtime-extractor-gd8kb\" (UID: \"1797129a-35ab-4364-98a7-7d6dc8f8e6b1\") " pod="openshift-insights/insights-runtime-extractor-gd8kb"
Jul 20 15:27:40.592795 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:40.592739    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-data-volume\") pod \"insights-runtime-extractor-gd8kb\" (UID: \"1797129a-35ab-4364-98a7-7d6dc8f8e6b1\") " pod="openshift-insights/insights-runtime-extractor-gd8kb"
Jul 20 15:27:40.675933 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:40.675897    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:40.693984 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:40.693963    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-crio-socket\") pod \"insights-runtime-extractor-gd8kb\" (UID: \"1797129a-35ab-4364-98a7-7d6dc8f8e6b1\") " pod="openshift-insights/insights-runtime-extractor-gd8kb"
Jul 20 15:27:40.694085 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:40.693990    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gd8kb\" (UID: \"1797129a-35ab-4364-98a7-7d6dc8f8e6b1\") " pod="openshift-insights/insights-runtime-extractor-gd8kb"
Jul 20 15:27:40.694085 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:40.694008    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gd8kb\" (UID: \"1797129a-35ab-4364-98a7-7d6dc8f8e6b1\") " pod="openshift-insights/insights-runtime-extractor-gd8kb"
Jul 20 15:27:40.694085 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:40.694040    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfxnv\" (UniqueName: \"kubernetes.io/projected/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-kube-api-access-jfxnv\") pod \"insights-runtime-extractor-gd8kb\" (UID: \"1797129a-35ab-4364-98a7-7d6dc8f8e6b1\") " pod="openshift-insights/insights-runtime-extractor-gd8kb"
Jul 20 15:27:40.694085 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:40.694067    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-data-volume\") pod \"insights-runtime-extractor-gd8kb\" (UID: \"1797129a-35ab-4364-98a7-7d6dc8f8e6b1\") " pod="openshift-insights/insights-runtime-extractor-gd8kb"
Jul 20 15:27:40.694263 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:40.694099    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 20 15:27:40.694263 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:40.694134    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 15:27:40.694263 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:40.694160    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-kube-rbac-proxy-cm podName:1797129a-35ab-4364-98a7-7d6dc8f8e6b1 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:41.194140602 +0000 UTC m=+44.203762262 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gd8kb" (UID: "1797129a-35ab-4364-98a7-7d6dc8f8e6b1") : configmap "kube-rbac-proxy" not found
Jul 20 15:27:40.694263 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:40.694187    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-insights-runtime-extractor-tls podName:1797129a-35ab-4364-98a7-7d6dc8f8e6b1 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:41.194171532 +0000 UTC m=+44.203793183 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gd8kb" (UID: "1797129a-35ab-4364-98a7-7d6dc8f8e6b1") : secret "insights-runtime-extractor-tls" not found
Jul 20 15:27:40.694263 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:40.694190    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-crio-socket\") pod \"insights-runtime-extractor-gd8kb\" (UID: \"1797129a-35ab-4364-98a7-7d6dc8f8e6b1\") " pod="openshift-insights/insights-runtime-extractor-gd8kb"
Jul 20 15:27:40.694471 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:40.694388    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-data-volume\") pod \"insights-runtime-extractor-gd8kb\" (UID: \"1797129a-35ab-4364-98a7-7d6dc8f8e6b1\") " pod="openshift-insights/insights-runtime-extractor-gd8kb"
Jul 20 15:27:40.704074 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:40.704059    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfxnv\" (UniqueName: \"kubernetes.io/projected/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-kube-api-access-jfxnv\") pod \"insights-runtime-extractor-gd8kb\" (UID: \"1797129a-35ab-4364-98a7-7d6dc8f8e6b1\") " pod="openshift-insights/insights-runtime-extractor-gd8kb"
Jul 20 15:27:40.718481 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:40.718455    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2bz87" event={"ID":"ca3c4e74-af1d-4595-aea5-19ebd06b7263","Type":"ContainerStarted","Data":"1541a4ca2c98d025d3a6288a6262cac5a2bd45a43fc116c228049499e7665bba"}
Jul 20 15:27:40.718675 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:40.718661    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2bz87"
Jul 20 15:27:40.740857 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:40.740788    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2bz87" podStartSLOduration=2.631065043 podStartE2EDuration="5.740775772s" totalImagesPullingTime="3.109710729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:27:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:27:40.740164862 +0000 UTC m=+43.749786524" watchObservedRunningTime="2026-07-20 15:27:40.740775772 +0000 UTC m=+43.750397435"
Jul 20 15:27:40.758589 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:40.758553    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-fxdt4" podStartSLOduration=2.653094868 podStartE2EDuration="5.758539851s" totalImagesPullingTime="3.105444983s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:27:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:27:40.758529245 +0000 UTC m=+43.768150907" watchObservedRunningTime="2026-07-20 15:27:40.758539851 +0000 UTC m=+43.768161513"
Jul 20 15:27:41.068894 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:41.068870    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2bz87_ca3c4e74-af1d-4595-aea5-19ebd06b7263/dns/0.log"
Jul 20 15:27:41.197190 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:41.197162    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gd8kb\" (UID: \"1797129a-35ab-4364-98a7-7d6dc8f8e6b1\") " pod="openshift-insights/insights-runtime-extractor-gd8kb"
Jul 20 15:27:41.197324 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:41.197195    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gd8kb\" (UID: \"1797129a-35ab-4364-98a7-7d6dc8f8e6b1\") " pod="openshift-insights/insights-runtime-extractor-gd8kb"
Jul 20 15:27:41.197324 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:41.197262    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 20 15:27:41.197399 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:41.197337    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-kube-rbac-proxy-cm podName:1797129a-35ab-4364-98a7-7d6dc8f8e6b1 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:42.197323665 +0000 UTC m=+45.206945306 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gd8kb" (UID: "1797129a-35ab-4364-98a7-7d6dc8f8e6b1") : configmap "kube-rbac-proxy" not found
Jul 20 15:27:41.197399 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:41.197380    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 15:27:41.197467 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:41.197427    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-insights-runtime-extractor-tls podName:1797129a-35ab-4364-98a7-7d6dc8f8e6b1 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:42.197416955 +0000 UTC m=+45.207038599 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gd8kb" (UID: "1797129a-35ab-4364-98a7-7d6dc8f8e6b1") : secret "insights-runtime-extractor-tls" not found
Jul 20 15:27:41.269332 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:41.269308    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2bz87_ca3c4e74-af1d-4595-aea5-19ebd06b7263/kube-rbac-proxy/0.log"
Jul 20 15:27:41.875518 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:41.875488    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:41.971602 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:41.971570    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:42.073634 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:42.073606    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:42.203615 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:42.203552    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gd8kb\" (UID: \"1797129a-35ab-4364-98a7-7d6dc8f8e6b1\") " pod="openshift-insights/insights-runtime-extractor-gd8kb"
Jul 20 15:27:42.203615 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:42.203592    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gd8kb\" (UID: \"1797129a-35ab-4364-98a7-7d6dc8f8e6b1\") " pod="openshift-insights/insights-runtime-extractor-gd8kb"
Jul 20 15:27:42.203760 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:42.203671    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 20 15:27:42.203760 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:42.203724    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 15:27:42.203760 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:42.203727    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-kube-rbac-proxy-cm podName:1797129a-35ab-4364-98a7-7d6dc8f8e6b1 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:44.203713273 +0000 UTC m=+47.213334918 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gd8kb" (UID: "1797129a-35ab-4364-98a7-7d6dc8f8e6b1") : configmap "kube-rbac-proxy" not found
Jul 20 15:27:42.203866 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:42.203772    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-insights-runtime-extractor-tls podName:1797129a-35ab-4364-98a7-7d6dc8f8e6b1 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:44.203759025 +0000 UTC m=+47.213380665 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gd8kb" (UID: "1797129a-35ab-4364-98a7-7d6dc8f8e6b1") : secret "insights-runtime-extractor-tls" not found
Jul 20 15:27:42.468839 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:42.468781    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-j45gb_ada5dd42-e780-4cc0-a845-ea417f1eac27/dns-node-resolver/0.log"
Jul 20 15:27:43.069772 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:43.069739    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6cb967f748-tsjhw_87c5f3e8-bc71-446b-9098-047de625ee8d/registry/0.log"
Jul 20 15:27:43.669387 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:43.669364    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pqqrs_b87d2983-fef9-4867-89c6-5f3fe76bd062/node-ca/0.log"
Jul 20 15:27:43.831560 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:43.831533    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:43.898908 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:43.898884    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:44.068947 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:44.068926    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fxdt4_144b485c-ce5b-4ac5-82c7-28f6fa5320af/serve-healthcheck-canary/0.log"
Jul 20 15:27:44.216463 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:44.216431    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gd8kb\" (UID: \"1797129a-35ab-4364-98a7-7d6dc8f8e6b1\") " pod="openshift-insights/insights-runtime-extractor-gd8kb"
Jul 20 15:27:44.216631 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:44.216470    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gd8kb\" (UID: \"1797129a-35ab-4364-98a7-7d6dc8f8e6b1\") " pod="openshift-insights/insights-runtime-extractor-gd8kb"
Jul 20 15:27:44.216631 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:44.216549    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 20 15:27:44.216631 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:44.216583    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 15:27:44.216631 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:44.216613    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-kube-rbac-proxy-cm podName:1797129a-35ab-4364-98a7-7d6dc8f8e6b1 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:48.216596319 +0000 UTC m=+51.226217959 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gd8kb" (UID: "1797129a-35ab-4364-98a7-7d6dc8f8e6b1") : configmap "kube-rbac-proxy" not found
Jul 20 15:27:44.216631 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:44.216634    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-insights-runtime-extractor-tls podName:1797129a-35ab-4364-98a7-7d6dc8f8e6b1 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:48.216619114 +0000 UTC m=+51.226240756 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gd8kb" (UID: "1797129a-35ab-4364-98a7-7d6dc8f8e6b1") : secret "insights-runtime-extractor-tls" not found
Jul 20 15:27:44.355301 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:44.355207    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:44.537982 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:44.537954    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:44.587638 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:44.587610    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:46.059677 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:46.059646    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:46.326780 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:46.326702    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:46.499359 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:46.499320    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:47.204710 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:47.204666    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:47.266931 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:47.266905    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:48.244988 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:48.244954    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gd8kb\" (UID: \"1797129a-35ab-4364-98a7-7d6dc8f8e6b1\") " pod="openshift-insights/insights-runtime-extractor-gd8kb"
Jul 20 15:27:48.244988 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:48.244987    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gd8kb\" (UID: \"1797129a-35ab-4364-98a7-7d6dc8f8e6b1\") " pod="openshift-insights/insights-runtime-extractor-gd8kb"
Jul 20 15:27:48.245389 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:48.245067    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 20 15:27:48.245389 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:48.245087    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 15:27:48.245389 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:48.245130    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-kube-rbac-proxy-cm podName:1797129a-35ab-4364-98a7-7d6dc8f8e6b1 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:56.245112304 +0000 UTC m=+59.254733945 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gd8kb" (UID: "1797129a-35ab-4364-98a7-7d6dc8f8e6b1") : configmap "kube-rbac-proxy" not found
Jul 20 15:27:48.245389 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:48.245145    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-insights-runtime-extractor-tls podName:1797129a-35ab-4364-98a7-7d6dc8f8e6b1 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:56.245139359 +0000 UTC m=+59.254760999 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gd8kb" (UID: "1797129a-35ab-4364-98a7-7d6dc8f8e6b1") : secret "insights-runtime-extractor-tls" not found
Jul 20 15:27:48.804716 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:48.804686    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:48.859475 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:48.859449    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:49.049614 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.049579    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:49.336658 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.336628    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:49.363825 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.363803    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5j487"]
Jul 20 15:27:49.379925 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.379910    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5j487"
Jul 20 15:27:49.384415 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.384399    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 20 15:27:49.385077 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.385055    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 20 15:27:49.385217 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.385202    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 20 15:27:49.385422 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.385398    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 20 15:27:49.385657 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.385623    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 20 15:27:49.385657 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.385651    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 20 15:27:49.385831 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.385729    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-4jxgh\""
Jul 20 15:27:49.397461 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.397443    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-xmqng"]
Jul 20 15:27:49.411942 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.411919    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-xmqng"
Jul 20 15:27:49.414481 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.414463    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 20 15:27:49.414590 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.414471    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 20 15:27:49.414656 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.414501    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 20 15:27:49.414656 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.414542    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-xsfbr\""
Jul 20 15:27:49.414961 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.414944    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-xmqng"]
Jul 20 15:27:49.434068 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.434037    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:49.452059 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.452042    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-xmqng\" (UID: \"aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-xmqng"
Jul 20 15:27:49.452147 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.452068    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a93e0f62-0ea5-4f63-ac6a-ad5252b879c7-sys\") pod \"node-exporter-5j487\" (UID: \"a93e0f62-0ea5-4f63-ac6a-ad5252b879c7\") " pod="openshift-monitoring/node-exporter-5j487"
Jul 20 15:27:49.452147 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.452087    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a93e0f62-0ea5-4f63-ac6a-ad5252b879c7-metrics-client-ca\") pod \"node-exporter-5j487\" (UID: \"a93e0f62-0ea5-4f63-ac6a-ad5252b879c7\") " pod="openshift-monitoring/node-exporter-5j487"
Jul 20 15:27:49.452147 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.452138    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-xmqng\" (UID: \"aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-xmqng"
Jul 20 15:27:49.452261 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.452164    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a93e0f62-0ea5-4f63-ac6a-ad5252b879c7-node-exporter-wtmp\") pod \"node-exporter-5j487\" (UID: \"a93e0f62-0ea5-4f63-ac6a-ad5252b879c7\") " pod="openshift-monitoring/node-exporter-5j487"
Jul 20 15:27:49.452261 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.452202    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cxz4\" (UniqueName: \"kubernetes.io/projected/a93e0f62-0ea5-4f63-ac6a-ad5252b879c7-kube-api-access-7cxz4\") pod \"node-exporter-5j487\" (UID: \"a93e0f62-0ea5-4f63-ac6a-ad5252b879c7\") " pod="openshift-monitoring/node-exporter-5j487"
Jul 20 15:27:49.452261 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.452247    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a93e0f62-0ea5-4f63-ac6a-ad5252b879c7-node-exporter-accelerators-collector-config\") pod \"node-exporter-5j487\" (UID: \"a93e0f62-0ea5-4f63-ac6a-ad5252b879c7\") " pod="openshift-monitoring/node-exporter-5j487"
Jul 20 15:27:49.452392 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.452278    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-xmqng\" (UID: \"aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-xmqng"
Jul 20 15:27:49.452392 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.452330    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-xmqng\" (UID: \"aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-xmqng"
Jul 20 15:27:49.452392 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.452366    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a93e0f62-0ea5-4f63-ac6a-ad5252b879c7-node-exporter-tls\") pod \"node-exporter-5j487\" (UID: \"a93e0f62-0ea5-4f63-ac6a-ad5252b879c7\") " pod="openshift-monitoring/node-exporter-5j487"
Jul 20 15:27:49.452392 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.452390    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a93e0f62-0ea5-4f63-ac6a-ad5252b879c7-root\") pod \"node-exporter-5j487\" (UID: \"a93e0f62-0ea5-4f63-ac6a-ad5252b879c7\") " pod="openshift-monitoring/node-exporter-5j487"
Jul 20 15:27:49.452529 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.452435    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-xmqng\" (UID: \"aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-xmqng"
Jul 20 15:27:49.452529 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.452463    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a93e0f62-0ea5-4f63-ac6a-ad5252b879c7-node-exporter-textfile\") pod \"node-exporter-5j487\" (UID: \"a93e0f62-0ea5-4f63-ac6a-ad5252b879c7\") " pod="openshift-monitoring/node-exporter-5j487"
Jul 20 15:27:49.452529 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.452504    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a93e0f62-0ea5-4f63-ac6a-ad5252b879c7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5j487\" (UID: \"a93e0f62-0ea5-4f63-ac6a-ad5252b879c7\") " pod="openshift-monitoring/node-exporter-5j487"
Jul 20 15:27:49.452628 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.452540    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kfzh\" (UniqueName: \"kubernetes.io/projected/aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4-kube-api-access-2kfzh\") pod \"kube-state-metrics-56c5594674-xmqng\" (UID: \"aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-xmqng"
Jul 20 15:27:49.536617 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.536595    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:49.552901 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.552879    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kfzh\" (UniqueName: \"kubernetes.io/projected/aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4-kube-api-access-2kfzh\") pod \"kube-state-metrics-56c5594674-xmqng\" (UID: \"aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-xmqng"
Jul 20 15:27:49.552977 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.552907    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-xmqng\" (UID: \"aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-xmqng"
Jul 20 15:27:49.552977 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.552923    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a93e0f62-0ea5-4f63-ac6a-ad5252b879c7-sys\") pod \"node-exporter-5j487\" (UID: \"a93e0f62-0ea5-4f63-ac6a-ad5252b879c7\") " pod="openshift-monitoring/node-exporter-5j487"
Jul 20 15:27:49.552977 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.552938    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a93e0f62-0ea5-4f63-ac6a-ad5252b879c7-metrics-client-ca\") pod \"node-exporter-5j487\" (UID: \"a93e0f62-0ea5-4f63-ac6a-ad5252b879c7\") " pod="openshift-monitoring/node-exporter-5j487"
Jul 20 15:27:49.553086 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.552994    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a93e0f62-0ea5-4f63-ac6a-ad5252b879c7-sys\") pod \"node-exporter-5j487\" (UID: \"a93e0f62-0ea5-4f63-ac6a-ad5252b879c7\") " pod="openshift-monitoring/node-exporter-5j487"
Jul 20 15:27:49.553086 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.553035    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-xmqng\" (UID: \"aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-xmqng"
Jul 20 15:27:49.553086 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.553056    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a93e0f62-0ea5-4f63-ac6a-ad5252b879c7-node-exporter-wtmp\") pod \"node-exporter-5j487\" (UID: \"a93e0f62-0ea5-4f63-ac6a-ad5252b879c7\") " pod="openshift-monitoring/node-exporter-5j487"
Jul 20 15:27:49.553218 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.553088    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cxz4\" (UniqueName: \"kubernetes.io/projected/a93e0f62-0ea5-4f63-ac6a-ad5252b879c7-kube-api-access-7cxz4\") pod \"node-exporter-5j487\" (UID: \"a93e0f62-0ea5-4f63-ac6a-ad5252b879c7\") " pod="openshift-monitoring/node-exporter-5j487"
Jul 20 15:27:49.553218 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.553106    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a93e0f62-0ea5-4f63-ac6a-ad5252b879c7-node-exporter-accelerators-collector-config\") pod \"node-exporter-5j487\" (UID: \"a93e0f62-0ea5-4f63-ac6a-ad5252b879c7\") " pod="openshift-monitoring/node-exporter-5j487"
Jul 20 15:27:49.553218 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.553126    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-xmqng\" (UID: \"aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-xmqng"
Jul 20 15:27:49.553218 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.553179    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-xmqng\" (UID: \"aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-xmqng"
Jul 20 15:27:49.553218 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.553214    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a93e0f62-0ea5-4f63-ac6a-ad5252b879c7-node-exporter-tls\") pod \"node-exporter-5j487\" (UID: \"a93e0f62-0ea5-4f63-ac6a-ad5252b879c7\") " pod="openshift-monitoring/node-exporter-5j487"
Jul 20 15:27:49.553483 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.553232    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a93e0f62-0ea5-4f63-ac6a-ad5252b879c7-node-exporter-wtmp\") pod \"node-exporter-5j487\" (UID: \"a93e0f62-0ea5-4f63-ac6a-ad5252b879c7\") " pod="openshift-monitoring/node-exporter-5j487"
Jul 20 15:27:49.553483 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.553249    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a93e0f62-0ea5-4f63-ac6a-ad5252b879c7-root\") pod \"node-exporter-5j487\" (UID: \"a93e0f62-0ea5-4f63-ac6a-ad5252b879c7\") " pod="openshift-monitoring/node-exporter-5j487"
Jul 20 15:27:49.553483 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.553310    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-xmqng\" (UID: \"aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-xmqng"
Jul 20 15:27:49.553483 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.553335    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a93e0f62-0ea5-4f63-ac6a-ad5252b879c7-node-exporter-textfile\") pod \"node-exporter-5j487\" (UID: \"a93e0f62-0ea5-4f63-ac6a-ad5252b879c7\") " pod="openshift-monitoring/node-exporter-5j487"
Jul 20 15:27:49.553483 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.553379    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a93e0f62-0ea5-4f63-ac6a-ad5252b879c7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5j487\" (UID: \"a93e0f62-0ea5-4f63-ac6a-ad5252b879c7\") " pod="openshift-monitoring/node-exporter-5j487"
Jul 20 15:27:49.553483 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.553446    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a93e0f62-0ea5-4f63-ac6a-ad5252b879c7-root\") pod \"node-exporter-5j487\" (UID: \"a93e0f62-0ea5-4f63-ac6a-ad5252b879c7\") " pod="openshift-monitoring/node-exporter-5j487"
Jul 20 15:27:49.553915 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:49.553535    2559 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 20 15:27:49.553915 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.553551    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a93e0f62-0ea5-4f63-ac6a-ad5252b879c7-metrics-client-ca\") pod \"node-exporter-5j487\" (UID: \"a93e0f62-0ea5-4f63-ac6a-ad5252b879c7\") " pod="openshift-monitoring/node-exporter-5j487"
Jul 20 15:27:49.553915 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:49.553599    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a93e0f62-0ea5-4f63-ac6a-ad5252b879c7-node-exporter-tls podName:a93e0f62-0ea5-4f63-ac6a-ad5252b879c7 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:50.053579917 +0000 UTC m=+53.063201563 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/a93e0f62-0ea5-4f63-ac6a-ad5252b879c7-node-exporter-tls") pod "node-exporter-5j487" (UID: "a93e0f62-0ea5-4f63-ac6a-ad5252b879c7") : secret "node-exporter-tls" not found
Jul 20 15:27:49.553915 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.553689    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a93e0f62-0ea5-4f63-ac6a-ad5252b879c7-node-exporter-accelerators-collector-config\") pod \"node-exporter-5j487\" (UID: \"a93e0f62-0ea5-4f63-ac6a-ad5252b879c7\") " pod="openshift-monitoring/node-exporter-5j487"
Jul 20 15:27:49.553915 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.553764    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-xmqng\" (UID: \"aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-xmqng"
Jul 20 15:27:49.553915 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.553880    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-xmqng\" (UID: \"aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-xmqng"
Jul 20 15:27:49.554211 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.553933    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a93e0f62-0ea5-4f63-ac6a-ad5252b879c7-node-exporter-textfile\") pod \"node-exporter-5j487\" (UID: \"a93e0f62-0ea5-4f63-ac6a-ad5252b879c7\") " pod="openshift-monitoring/node-exporter-5j487"
Jul 20 15:27:49.554211 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.553958    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-xmqng\" (UID: \"aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-xmqng"
Jul 20 15:27:49.555654 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.555632    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-xmqng\" (UID: \"aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-xmqng"
Jul 20 15:27:49.555731 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.555632    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a93e0f62-0ea5-4f63-ac6a-ad5252b879c7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5j487\" (UID: \"a93e0f62-0ea5-4f63-ac6a-ad5252b879c7\") " pod="openshift-monitoring/node-exporter-5j487"
Jul 20 15:27:49.555842 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.555825    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-xmqng\" (UID: \"aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-xmqng"
Jul 20 15:27:49.572759 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.572737    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kfzh\" (UniqueName: \"kubernetes.io/projected/aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4-kube-api-access-2kfzh\") pod \"kube-state-metrics-56c5594674-xmqng\" (UID: \"aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-xmqng"
Jul 20 15:27:49.573146 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.573130    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cxz4\" (UniqueName: \"kubernetes.io/projected/a93e0f62-0ea5-4f63-ac6a-ad5252b879c7-kube-api-access-7cxz4\") pod \"node-exporter-5j487\" (UID: \"a93e0f62-0ea5-4f63-ac6a-ad5252b879c7\") " pod="openshift-monitoring/node-exporter-5j487"
Jul 20 15:27:49.689048 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.688987    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:49.720776 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.720751    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-xmqng"
Jul 20 15:27:49.793365 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.792869    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:49.871731 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:49.871698    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-xmqng"]
Jul 20 15:27:49.874525 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:27:49.874502    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa525c7a_d6d4_4a82_b068_14c7d0f5dbe4.slice/crio-bb47e993a72a13fcd09c4c0158e515425034c5f7ef89cb508c2e375529e8f685 WatchSource:0}: Error finding container bb47e993a72a13fcd09c4c0158e515425034c5f7ef89cb508c2e375529e8f685: Status 404 returned error can't find the container with id bb47e993a72a13fcd09c4c0158e515425034c5f7ef89cb508c2e375529e8f685
Jul 20 15:27:50.057374 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:50.057338    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a93e0f62-0ea5-4f63-ac6a-ad5252b879c7-node-exporter-tls\") pod \"node-exporter-5j487\" (UID: \"a93e0f62-0ea5-4f63-ac6a-ad5252b879c7\") " pod="openshift-monitoring/node-exporter-5j487"
Jul 20 15:27:50.059542 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:50.059525    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a93e0f62-0ea5-4f63-ac6a-ad5252b879c7-node-exporter-tls\") pod \"node-exporter-5j487\" (UID: \"a93e0f62-0ea5-4f63-ac6a-ad5252b879c7\") " pod="openshift-monitoring/node-exporter-5j487"
Jul 20 15:27:50.102939 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:50.102910    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:50.289347 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:50.289316    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5j487"
Jul 20 15:27:50.322983 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:50.322851    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:50.325266 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:27:50.325245    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda93e0f62_0ea5_4f63_ac6a_ad5252b879c7.slice/crio-5393bef6a47a453a62931f68ba364cde389d6c48e5aab50d6df893cd9a6873f8 WatchSource:0}: Error finding container 5393bef6a47a453a62931f68ba364cde389d6c48e5aab50d6df893cd9a6873f8: Status 404 returned error can't find the container with id 5393bef6a47a453a62931f68ba364cde389d6c48e5aab50d6df893cd9a6873f8
Jul 20 15:27:50.428650 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:50.428621    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:50.533024 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:50.533000    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:50.632503 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:50.632404    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:50.723624 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:50.723600    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2bz87"
Jul 20 15:27:50.742438 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:50.742402    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-xmqng" event={"ID":"aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4","Type":"ContainerStarted","Data":"bb47e993a72a13fcd09c4c0158e515425034c5f7ef89cb508c2e375529e8f685"}
Jul 20 15:27:50.743522 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:50.743501    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5j487" event={"ID":"a93e0f62-0ea5-4f63-ac6a-ad5252b879c7","Type":"ContainerStarted","Data":"5393bef6a47a453a62931f68ba364cde389d6c48e5aab50d6df893cd9a6873f8"}
Jul 20 15:27:50.951395 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:50.951310    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:51.019912 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:51.019876    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:52.238648 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.238618    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:52.337788 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.337751    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:52.403980 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.403944    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-778645887f-jd5cm"]
Jul 20 15:27:52.406257 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.406234    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-778645887f-jd5cm"
Jul 20 15:27:52.409527 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.409504    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 20 15:27:52.409688 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.409667    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 20 15:27:52.410089 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.410067    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:52.414017 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.413958    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 20 15:27:52.414134 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.414042    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-7ds2k\""
Jul 20 15:27:52.414134 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.414042    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 20 15:27:52.414345 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.414329    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 20 15:27:52.414397 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.414351    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-8okouig3ffhmb\""
Jul 20 15:27:52.421424 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.421406    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-778645887f-jd5cm"]
Jul 20 15:27:52.478364 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.478334    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8d92cbb5-adb9-4c90-b192-4f2f14738ef7-secret-thanos-querier-tls\") pod \"thanos-querier-778645887f-jd5cm\" (UID: \"8d92cbb5-adb9-4c90-b192-4f2f14738ef7\") " pod="openshift-monitoring/thanos-querier-778645887f-jd5cm"
Jul 20 15:27:52.478503 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.478378    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8d92cbb5-adb9-4c90-b192-4f2f14738ef7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-778645887f-jd5cm\" (UID: \"8d92cbb5-adb9-4c90-b192-4f2f14738ef7\") " pod="openshift-monitoring/thanos-querier-778645887f-jd5cm"
Jul 20 15:27:52.478503 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.478414    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8d92cbb5-adb9-4c90-b192-4f2f14738ef7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-778645887f-jd5cm\" (UID: \"8d92cbb5-adb9-4c90-b192-4f2f14738ef7\") " pod="openshift-monitoring/thanos-querier-778645887f-jd5cm"
Jul 20 15:27:52.478503 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.478472    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8d92cbb5-adb9-4c90-b192-4f2f14738ef7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-778645887f-jd5cm\" (UID: \"8d92cbb5-adb9-4c90-b192-4f2f14738ef7\") " pod="openshift-monitoring/thanos-querier-778645887f-jd5cm"
Jul 20 15:27:52.478611 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.478512    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d92cbb5-adb9-4c90-b192-4f2f14738ef7-metrics-client-ca\") pod \"thanos-querier-778645887f-jd5cm\" (UID: \"8d92cbb5-adb9-4c90-b192-4f2f14738ef7\") " pod="openshift-monitoring/thanos-querier-778645887f-jd5cm"
Jul 20 15:27:52.478611 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.478535    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8d92cbb5-adb9-4c90-b192-4f2f14738ef7-secret-grpc-tls\") pod \"thanos-querier-778645887f-jd5cm\" (UID: \"8d92cbb5-adb9-4c90-b192-4f2f14738ef7\") " pod="openshift-monitoring/thanos-querier-778645887f-jd5cm"
Jul 20 15:27:52.478611 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.478562    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hb7p\" (UniqueName: \"kubernetes.io/projected/8d92cbb5-adb9-4c90-b192-4f2f14738ef7-kube-api-access-4hb7p\") pod \"thanos-querier-778645887f-jd5cm\" (UID: \"8d92cbb5-adb9-4c90-b192-4f2f14738ef7\") " pod="openshift-monitoring/thanos-querier-778645887f-jd5cm"
Jul 20 15:27:52.478713 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.478606    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8d92cbb5-adb9-4c90-b192-4f2f14738ef7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-778645887f-jd5cm\" (UID: \"8d92cbb5-adb9-4c90-b192-4f2f14738ef7\") " pod="openshift-monitoring/thanos-querier-778645887f-jd5cm"
Jul 20 15:27:52.484654 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.484626    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:52.579621 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.579588    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8d92cbb5-adb9-4c90-b192-4f2f14738ef7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-778645887f-jd5cm\" (UID: \"8d92cbb5-adb9-4c90-b192-4f2f14738ef7\") " pod="openshift-monitoring/thanos-querier-778645887f-jd5cm"
Jul 20 15:27:52.579810 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.579659    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8d92cbb5-adb9-4c90-b192-4f2f14738ef7-secret-thanos-querier-tls\") pod \"thanos-querier-778645887f-jd5cm\" (UID: \"8d92cbb5-adb9-4c90-b192-4f2f14738ef7\") " pod="openshift-monitoring/thanos-querier-778645887f-jd5cm"
Jul 20 15:27:52.579810 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.579690    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8d92cbb5-adb9-4c90-b192-4f2f14738ef7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-778645887f-jd5cm\" (UID: \"8d92cbb5-adb9-4c90-b192-4f2f14738ef7\") " pod="openshift-monitoring/thanos-querier-778645887f-jd5cm"
Jul 20 15:27:52.579810 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.579719    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8d92cbb5-adb9-4c90-b192-4f2f14738ef7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-778645887f-jd5cm\" (UID: \"8d92cbb5-adb9-4c90-b192-4f2f14738ef7\") " pod="openshift-monitoring/thanos-querier-778645887f-jd5cm"
Jul 20 15:27:52.579810 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.579747    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8d92cbb5-adb9-4c90-b192-4f2f14738ef7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-778645887f-jd5cm\" (UID: \"8d92cbb5-adb9-4c90-b192-4f2f14738ef7\") " pod="openshift-monitoring/thanos-querier-778645887f-jd5cm"
Jul 20 15:27:52.579810 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.579789    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d92cbb5-adb9-4c90-b192-4f2f14738ef7-metrics-client-ca\") pod \"thanos-querier-778645887f-jd5cm\" (UID: \"8d92cbb5-adb9-4c90-b192-4f2f14738ef7\") " pod="openshift-monitoring/thanos-querier-778645887f-jd5cm"
Jul 20 15:27:52.580054 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.579821    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8d92cbb5-adb9-4c90-b192-4f2f14738ef7-secret-grpc-tls\") pod \"thanos-querier-778645887f-jd5cm\" (UID: \"8d92cbb5-adb9-4c90-b192-4f2f14738ef7\") " pod="openshift-monitoring/thanos-querier-778645887f-jd5cm"
Jul 20 15:27:52.580054 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.579862    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hb7p\" (UniqueName: \"kubernetes.io/projected/8d92cbb5-adb9-4c90-b192-4f2f14738ef7-kube-api-access-4hb7p\") pod \"thanos-querier-778645887f-jd5cm\" (UID: \"8d92cbb5-adb9-4c90-b192-4f2f14738ef7\") " pod="openshift-monitoring/thanos-querier-778645887f-jd5cm"
Jul 20 15:27:52.580661 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.580637    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d92cbb5-adb9-4c90-b192-4f2f14738ef7-metrics-client-ca\") pod \"thanos-querier-778645887f-jd5cm\" (UID: \"8d92cbb5-adb9-4c90-b192-4f2f14738ef7\") " pod="openshift-monitoring/thanos-querier-778645887f-jd5cm"
Jul 20 15:27:52.582318 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.582272    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8d92cbb5-adb9-4c90-b192-4f2f14738ef7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-778645887f-jd5cm\" (UID: \"8d92cbb5-adb9-4c90-b192-4f2f14738ef7\") " pod="openshift-monitoring/thanos-querier-778645887f-jd5cm"
Jul 20 15:27:52.582408 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.582350    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8d92cbb5-adb9-4c90-b192-4f2f14738ef7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-778645887f-jd5cm\" (UID: \"8d92cbb5-adb9-4c90-b192-4f2f14738ef7\") " pod="openshift-monitoring/thanos-querier-778645887f-jd5cm"
Jul 20 15:27:52.582611 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.582591    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8d92cbb5-adb9-4c90-b192-4f2f14738ef7-secret-thanos-querier-tls\") pod \"thanos-querier-778645887f-jd5cm\" (UID: \"8d92cbb5-adb9-4c90-b192-4f2f14738ef7\") " pod="openshift-monitoring/thanos-querier-778645887f-jd5cm"
Jul 20 15:27:52.582680 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.582667    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8d92cbb5-adb9-4c90-b192-4f2f14738ef7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-778645887f-jd5cm\" (UID: \"8d92cbb5-adb9-4c90-b192-4f2f14738ef7\") " pod="openshift-monitoring/thanos-querier-778645887f-jd5cm"
Jul 20 15:27:52.582726 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.582677    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8d92cbb5-adb9-4c90-b192-4f2f14738ef7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-778645887f-jd5cm\" (UID: \"8d92cbb5-adb9-4c90-b192-4f2f14738ef7\") " pod="openshift-monitoring/thanos-querier-778645887f-jd5cm"
Jul 20 15:27:52.582812 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.582795    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8d92cbb5-adb9-4c90-b192-4f2f14738ef7-secret-grpc-tls\") pod \"thanos-querier-778645887f-jd5cm\" (UID: \"8d92cbb5-adb9-4c90-b192-4f2f14738ef7\") " pod="openshift-monitoring/thanos-querier-778645887f-jd5cm"
Jul 20 15:27:52.587347 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.587330    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hb7p\" (UniqueName: \"kubernetes.io/projected/8d92cbb5-adb9-4c90-b192-4f2f14738ef7-kube-api-access-4hb7p\") pod \"thanos-querier-778645887f-jd5cm\" (UID: \"8d92cbb5-adb9-4c90-b192-4f2f14738ef7\") " pod="openshift-monitoring/thanos-querier-778645887f-jd5cm"
Jul 20 15:27:52.714637 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.714610    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-778645887f-jd5cm"
Jul 20 15:27:52.748072 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.748045    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:52.751100 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.750849    2559 generic.go:356] "Generic (PLEG): container finished" podID="a93e0f62-0ea5-4f63-ac6a-ad5252b879c7" containerID="2cc2ac2d17c91b27ca3c82d14171eaf7278b3b37be13ddbad7b42422943f41e4" exitCode=0
Jul 20 15:27:52.751100 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.750958    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5j487" event={"ID":"a93e0f62-0ea5-4f63-ac6a-ad5252b879c7","Type":"ContainerDied","Data":"2cc2ac2d17c91b27ca3c82d14171eaf7278b3b37be13ddbad7b42422943f41e4"}
Jul 20 15:27:52.753811 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.753787    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-xmqng" event={"ID":"aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4","Type":"ContainerStarted","Data":"9af3bab5d6e5cd3ce80f3dd144375fd65e81416d24225e7f91c336ee483942b7"}
Jul 20 15:27:52.753911 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.753843    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-xmqng" event={"ID":"aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4","Type":"ContainerStarted","Data":"f49d60ce6a5ceb0acddf18e6832d074a1abf85301baf1c28ca5e018b9e98a0ee"}
Jul 20 15:27:52.753911 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.753872    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-xmqng" event={"ID":"aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4","Type":"ContainerStarted","Data":"eda127106584a7dbf55602d4d47521c07aee38d9d4bbde3f97b831973c643bfd"}
Jul 20 15:27:52.797834 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.797782    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-xmqng" podStartSLOduration=1.936679438 podStartE2EDuration="3.797761711s" totalImagesPullingTime="1.861082273s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:27:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:27:52.797761025 +0000 UTC m=+55.807382688" watchObservedRunningTime="2026-07-20 15:27:52.797761711 +0000 UTC m=+55.807383373"
Jul 20 15:27:52.848960 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.848932    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-778645887f-jd5cm"]
Jul 20 15:27:52.849168 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:52.849152    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:52.853310 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:27:52.853269    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d92cbb5_adb9_4c90_b192_4f2f14738ef7.slice/crio-62ca6bc37d3a688eb74624907b5c0b6c1f33eb12d4c995e8c5291bf21c8ae1c9 WatchSource:0}: Error finding container 62ca6bc37d3a688eb74624907b5c0b6c1f33eb12d4c995e8c5291bf21c8ae1c9: Status 404 returned error can't find the container with id 62ca6bc37d3a688eb74624907b5c0b6c1f33eb12d4c995e8c5291bf21c8ae1c9
Jul 20 15:27:53.759623 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:53.759585    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-778645887f-jd5cm" event={"ID":"8d92cbb5-adb9-4c90-b192-4f2f14738ef7","Type":"ContainerStarted","Data":"62ca6bc37d3a688eb74624907b5c0b6c1f33eb12d4c995e8c5291bf21c8ae1c9"}
Jul 20 15:27:53.761966 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:53.761935    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5j487" event={"ID":"a93e0f62-0ea5-4f63-ac6a-ad5252b879c7","Type":"ContainerStarted","Data":"8634c02fb7929c96ff2988fa173f9c607b931e12569f55c5275a609b3d6af998"}
Jul 20 15:27:53.762099 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:53.761989    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5j487" event={"ID":"a93e0f62-0ea5-4f63-ac6a-ad5252b879c7","Type":"ContainerStarted","Data":"f8be855fb5238b76d93025fdd43f2cd08ea2e3b0aaff367b2a068429397dbaf4"}
Jul 20 15:27:53.788101 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:53.788051    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5j487" podStartSLOduration=3.297938079 podStartE2EDuration="4.788035895s" totalImagesPullingTime="1.411883427s" totalInitContainerRuntime="78.214389ms" isStatefulPod=true podCreationTimestamp="2026-07-20 15:27:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:27:53.786604292 +0000 UTC m=+56.796225953" watchObservedRunningTime="2026-07-20 15:27:53.788035895 +0000 UTC m=+56.797657556"
Jul 20 15:27:53.983064 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:53.983034    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:54.001884 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:54.001855    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-zmnbs"]
Jul 20 15:27:54.003959 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:54.003938    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-zmnbs"
Jul 20 15:27:54.006475 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:54.006449    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-xm6v2\""
Jul 20 15:27:54.006566 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:54.006555    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 20 15:27:54.014184 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:54.014165    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-zmnbs"]
Jul 20 15:27:54.057412 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:54.057388    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:54.094343 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:54.094313    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c64mj\" (UniqueName: \"kubernetes.io/projected/947ca3d1-14d5-4e72-b73c-b04324947ddb-kube-api-access-c64mj\") pod \"monitoring-plugin-7b7b7bc8bf-zmnbs\" (UID: \"947ca3d1-14d5-4e72-b73c-b04324947ddb\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-zmnbs"
Jul 20 15:27:54.094473 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:54.094372    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/947ca3d1-14d5-4e72-b73c-b04324947ddb-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-zmnbs\" (UID: \"947ca3d1-14d5-4e72-b73c-b04324947ddb\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-zmnbs"
Jul 20 15:27:54.194871 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:54.194845    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c64mj\" (UniqueName: \"kubernetes.io/projected/947ca3d1-14d5-4e72-b73c-b04324947ddb-kube-api-access-c64mj\") pod \"monitoring-plugin-7b7b7bc8bf-zmnbs\" (UID: \"947ca3d1-14d5-4e72-b73c-b04324947ddb\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-zmnbs"
Jul 20 15:27:54.195016 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:54.194902    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/947ca3d1-14d5-4e72-b73c-b04324947ddb-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-zmnbs\" (UID: \"947ca3d1-14d5-4e72-b73c-b04324947ddb\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-zmnbs"
Jul 20 15:27:54.195125 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:54.195102    2559 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 20 15:27:54.195210 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:27:54.195198    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/947ca3d1-14d5-4e72-b73c-b04324947ddb-monitoring-plugin-cert podName:947ca3d1-14d5-4e72-b73c-b04324947ddb nodeName:}" failed. No retries permitted until 2026-07-20 15:27:54.695177697 +0000 UTC m=+57.704799354 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/947ca3d1-14d5-4e72-b73c-b04324947ddb-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-zmnbs" (UID: "947ca3d1-14d5-4e72-b73c-b04324947ddb") : secret "monitoring-plugin-cert" not found
Jul 20 15:27:54.203838 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:54.203817    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c64mj\" (UniqueName: \"kubernetes.io/projected/947ca3d1-14d5-4e72-b73c-b04324947ddb-kube-api-access-c64mj\") pod \"monitoring-plugin-7b7b7bc8bf-zmnbs\" (UID: \"947ca3d1-14d5-4e72-b73c-b04324947ddb\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-zmnbs"
Jul 20 15:27:54.699372 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:54.699342    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/947ca3d1-14d5-4e72-b73c-b04324947ddb-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-zmnbs\" (UID: \"947ca3d1-14d5-4e72-b73c-b04324947ddb\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-zmnbs"
Jul 20 15:27:54.702017 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:54.701990    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/947ca3d1-14d5-4e72-b73c-b04324947ddb-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-zmnbs\" (UID: \"947ca3d1-14d5-4e72-b73c-b04324947ddb\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-zmnbs"
Jul 20 15:27:54.747223 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:54.747196    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:54.914155 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:54.914127    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-zmnbs"
Jul 20 15:27:55.054666 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.054638    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-zmnbs"]
Jul 20 15:27:55.063503 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:27:55.063462    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod947ca3d1_14d5_4e72_b73c_b04324947ddb.slice/crio-39a02823c4788e4699f049395debc6abcf03d0d43f3b1083659aa8b45bf7b6ec WatchSource:0}: Error finding container 39a02823c4788e4699f049395debc6abcf03d0d43f3b1083659aa8b45bf7b6ec: Status 404 returned error can't find the container with id 39a02823c4788e4699f049395debc6abcf03d0d43f3b1083659aa8b45bf7b6ec
Jul 20 15:27:55.296347 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.296316    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:55.353566 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.353541    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:55.627817 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.627718    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:55.648227 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.648191    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 20 15:27:55.651250 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.651227    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.653874 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.653846    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 20 15:27:55.654031 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.653879    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 20 15:27:55.654130 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.654115    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 20 15:27:55.654196 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.654153    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-rh4pt\""
Jul 20 15:27:55.654411 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.654391    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 20 15:27:55.654541 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.654425    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 20 15:27:55.654619 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.654606    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 20 15:27:55.654678 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.654661    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 20 15:27:55.654767 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.654751    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 20 15:27:55.654856 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.654793    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 20 15:27:55.654988 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.654960    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 20 15:27:55.655087 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.655045    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 20 15:27:55.655144 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.655088    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7m758ul8gkle\""
Jul 20 15:27:55.655216 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.655152    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 20 15:27:55.655700 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.655463    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 20 15:27:55.658218 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.658196    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 20 15:27:55.661195 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.660700    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 20 15:27:55.669730 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.669707    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 20 15:27:55.707189 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.707156    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.707425 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.707224    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-config\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.707425 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.707260    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.707425 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.707282    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.707425 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.707330    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.707425 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.707367    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.707425 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.707403    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljkr6\" (UniqueName: \"kubernetes.io/projected/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-kube-api-access-ljkr6\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.707786 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.707431    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.707786 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.707482    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.707786 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.707508    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-web-config\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.707786 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.707534    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.707786 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.707574    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.707786 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.707610    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.707786 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.707636    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.707786 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.707666    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.707786 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.707691    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.707786 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.707737    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.707786 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.707759    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-config-out\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.707786 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.707785    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.708370 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.707823    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.772350 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.772315    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-778645887f-jd5cm" event={"ID":"8d92cbb5-adb9-4c90-b192-4f2f14738ef7","Type":"ContainerStarted","Data":"6f54511a20d3bfbe35e9cbe1ca7ec31870fdd312a78c2b240541721d74e9dc47"}
Jul 20 15:27:55.772469 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.772378    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-778645887f-jd5cm" event={"ID":"8d92cbb5-adb9-4c90-b192-4f2f14738ef7","Type":"ContainerStarted","Data":"1b240287cdccaa5114ab261c897b503f8108e076f5ac245af969fcc614d800cf"}
Jul 20 15:27:55.772469 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.772413    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-778645887f-jd5cm" event={"ID":"8d92cbb5-adb9-4c90-b192-4f2f14738ef7","Type":"ContainerStarted","Data":"d4a311fd6ce67ed73ea985fbd330c65a70447acfd77dda5f91de5fb689720a3f"}
Jul 20 15:27:55.773623 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.773597    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-zmnbs" event={"ID":"947ca3d1-14d5-4e72-b73c-b04324947ddb","Type":"ContainerStarted","Data":"39a02823c4788e4699f049395debc6abcf03d0d43f3b1083659aa8b45bf7b6ec"}
Jul 20 15:27:55.808842 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.808812    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.808979 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.808862    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.808979 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.808913    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-config\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.808979 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.808942    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.808979 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.808964    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.809169 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.808982    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.809169 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.809007    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.809169 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.809044    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljkr6\" (UniqueName: \"kubernetes.io/projected/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-kube-api-access-ljkr6\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.809169 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.809070    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.809169 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.809099    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.809169 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.809115    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-web-config\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.809169 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.809129    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.809169 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.809165    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.809572 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.809192    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.809572 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.809215    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.809572 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.809242    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.809572 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.809268    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.809572 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.809333    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.809572 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.809356    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-config-out\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.809572 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.809384    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.809904 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.809778    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.809904 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.809837    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.812035 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.810856    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.812035 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.810859    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.812035 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.811798    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.812035 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.811891    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.812349 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.812327    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.812949 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.812889    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.814337 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.814307    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.815611 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.814778    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.815611 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.815151    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-config\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.815611 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.815170    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-web-config\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.815611 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.815390    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.815611 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.815557    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.815611 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.815575    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.816322 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.816266    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.817022 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.816999    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.817128 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.817081    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-config-out\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.818055 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.818011    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.820214 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.820184    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljkr6\" (UniqueName: \"kubernetes.io/projected/bcbcfeb8-056b-42d8-be5a-7d0e6b845afe-kube-api-access-ljkr6\") pod \"prometheus-k8s-0\" (UID: \"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:55.963597 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:55.963575    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:27:56.083957 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:56.083920    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:56.117480 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:56.117451    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 20 15:27:56.118511 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:27:56.118483    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcbcfeb8_056b_42d8_be5a_7d0e6b845afe.slice/crio-cde13c2fa3f557969bed2dab9a2226125af6e4609438f08b3923f4a3f9ae942a WatchSource:0}: Error finding container cde13c2fa3f557969bed2dab9a2226125af6e4609438f08b3923f4a3f9ae942a: Status 404 returned error can't find the container with id cde13c2fa3f557969bed2dab9a2226125af6e4609438f08b3923f4a3f9ae942a
Jul 20 15:27:56.181584 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:56.181503    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:56.286012 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:56.285983    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:56.315235 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:56.315209    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gd8kb\" (UID: \"1797129a-35ab-4364-98a7-7d6dc8f8e6b1\") " pod="openshift-insights/insights-runtime-extractor-gd8kb"
Jul 20 15:27:56.315403 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:56.315310    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gd8kb\" (UID: \"1797129a-35ab-4364-98a7-7d6dc8f8e6b1\") " pod="openshift-insights/insights-runtime-extractor-gd8kb"
Jul 20 15:27:56.315805 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:56.315776    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gd8kb\" (UID: \"1797129a-35ab-4364-98a7-7d6dc8f8e6b1\") " pod="openshift-insights/insights-runtime-extractor-gd8kb"
Jul 20 15:27:56.317783 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:56.317761    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1797129a-35ab-4364-98a7-7d6dc8f8e6b1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gd8kb\" (UID: \"1797129a-35ab-4364-98a7-7d6dc8f8e6b1\") " pod="openshift-insights/insights-runtime-extractor-gd8kb"
Jul 20 15:27:56.402718 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:56.402689    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gd8kb"
Jul 20 15:27:56.560253 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:56.560219    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gd8kb"]
Jul 20 15:27:56.780766 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:56.780732    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-778645887f-jd5cm" event={"ID":"8d92cbb5-adb9-4c90-b192-4f2f14738ef7","Type":"ContainerStarted","Data":"2b997713b9fd620837434cac83014cafa77863e45127a10ea9dbf21d905fe4fc"}
Jul 20 15:27:56.780942 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:56.780827    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-778645887f-jd5cm" event={"ID":"8d92cbb5-adb9-4c90-b192-4f2f14738ef7","Type":"ContainerStarted","Data":"b9624b3d3a9c85020faca5db41f78ab8b5c4c0db0c5be1b6930e6a5a43b5f448"}
Jul 20 15:27:56.780942 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:56.780881    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-778645887f-jd5cm"
Jul 20 15:27:56.780942 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:56.780930    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-778645887f-jd5cm" event={"ID":"8d92cbb5-adb9-4c90-b192-4f2f14738ef7","Type":"ContainerStarted","Data":"367e1e24156f8a799a4bdaf34a27d49dfdbc418e62f0287e75aea9cf33f1baaf"}
Jul 20 15:27:56.782224 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:56.782196    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe","Type":"ContainerStarted","Data":"cde13c2fa3f557969bed2dab9a2226125af6e4609438f08b3923f4a3f9ae942a"}
Jul 20 15:27:56.783523 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:56.783492    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gd8kb" event={"ID":"1797129a-35ab-4364-98a7-7d6dc8f8e6b1","Type":"ContainerStarted","Data":"997511e096bdfab9042403eae1173ccaa975cd94857ef3d8bfbbe8f9ac3fc915"}
Jul 20 15:27:56.785010 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:56.784979    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-zmnbs" event={"ID":"947ca3d1-14d5-4e72-b73c-b04324947ddb","Type":"ContainerStarted","Data":"08a0a5b3d426fcdb862fba2ceb5370a747c51649481454fc3c33d5ac68e8caba"}
Jul 20 15:27:56.785257 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:56.785196    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-zmnbs"
Jul 20 15:27:56.790915 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:56.790895    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-zmnbs"
Jul 20 15:27:56.805389 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:56.805350    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-778645887f-jd5cm" podStartSLOduration=1.9132795599999999 podStartE2EDuration="4.805336611s" totalImagesPullingTime="2.892057051s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:27:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:27:56.803751645 +0000 UTC m=+59.813373321" watchObservedRunningTime="2026-07-20 15:27:56.805336611 +0000 UTC m=+59.814958251"
Jul 20 15:27:56.819214 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:56.819169    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-zmnbs" podStartSLOduration=2.415767037 podStartE2EDuration="3.819156214s" totalImagesPullingTime="1.403389177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:27:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:27:56.81825886 +0000 UTC m=+59.827880538" watchObservedRunningTime="2026-07-20 15:27:56.819156214 +0000 UTC m=+59.828777876"
Jul 20 15:27:57.679399 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:57.679369    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7hgpz"
Jul 20 15:27:57.714880 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:57.714065    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6cb967f748-tsjhw"
Jul 20 15:27:57.794697 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:57.791876    2559 generic.go:356] "Generic (PLEG): container finished" podID="bcbcfeb8-056b-42d8-be5a-7d0e6b845afe" containerID="585fcf11aa521cb3ff44db4ee62ec3dc05fa52d7d99d8c48307e7a041e4395a9" exitCode=0
Jul 20 15:27:57.794697 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:57.793170    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe","Type":"ContainerDied","Data":"585fcf11aa521cb3ff44db4ee62ec3dc05fa52d7d99d8c48307e7a041e4395a9"}
Jul 20 15:27:57.829829 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:57.829788    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:57.928119 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:57.928076    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:57.992386 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:57.992342    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:59.683386 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:59.683352    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:59.784355 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:59.784319    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:59.884670 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:59.884635    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:27:59.984876 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:27:59.984779    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:28:00.595447 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:00.595408    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:28:00.696107 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:00.696070    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:28:02.147744 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:02.147715    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:28:02.250059 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:02.250026    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:28:02.800701 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:02.800678    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-778645887f-jd5cm"
Jul 20 15:28:03.280442 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:03.280411    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b5f0a63-5c26-49e1-a580-8b19d8a6e289-metrics-certs\") pod \"network-metrics-daemon-4kzlg\" (UID: \"7b5f0a63-5c26-49e1-a580-8b19d8a6e289\") " pod="openshift-multus/network-metrics-daemon-4kzlg"
Jul 20 15:28:03.280785 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:03.280501    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/23b1e60d-e25e-48f8-9ab8-c361f433594c-original-pull-secret\") pod \"global-pull-secret-syncer-t54tw\" (UID: \"23b1e60d-e25e-48f8-9ab8-c361f433594c\") " pod="kube-system/global-pull-secret-syncer-t54tw"
Jul 20 15:28:03.283239 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:03.283217    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 20 15:28:03.283524 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:03.283505    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 20 15:28:03.293260 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:03.293214    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b5f0a63-5c26-49e1-a580-8b19d8a6e289-metrics-certs\") pod \"network-metrics-daemon-4kzlg\" (UID: \"7b5f0a63-5c26-49e1-a580-8b19d8a6e289\") " pod="openshift-multus/network-metrics-daemon-4kzlg"
Jul 20 15:28:03.293518 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:03.293498    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/23b1e60d-e25e-48f8-9ab8-c361f433594c-original-pull-secret\") pod \"global-pull-secret-syncer-t54tw\" (UID: \"23b1e60d-e25e-48f8-9ab8-c361f433594c\") " pod="kube-system/global-pull-secret-syncer-t54tw"
Jul 20 15:28:03.326537 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:03.326513    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t54tw"
Jul 20 15:28:03.343148 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:03.343113    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zgzvz\""
Jul 20 15:28:03.351159 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:03.351132    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kzlg"
Jul 20 15:28:03.381561 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:03.380974    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv85s\" (UniqueName: \"kubernetes.io/projected/0293426e-c324-435c-873e-468da9602ff6-kube-api-access-jv85s\") pod \"network-check-target-n7pqm\" (UID: \"0293426e-c324-435c-873e-468da9602ff6\") " pod="openshift-network-diagnostics/network-check-target-n7pqm"
Jul 20 15:28:03.384946 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:03.384681    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 20 15:28:03.395434 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:03.395234    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 20 15:28:03.410368 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:03.407492    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv85s\" (UniqueName: \"kubernetes.io/projected/0293426e-c324-435c-873e-468da9602ff6-kube-api-access-jv85s\") pod \"network-check-target-n7pqm\" (UID: \"0293426e-c324-435c-873e-468da9602ff6\") " pod="openshift-network-diagnostics/network-check-target-n7pqm"
Jul 20 15:28:03.540420 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:03.540194    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-t54tw"]
Jul 20 15:28:03.542718 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:28:03.542695    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23b1e60d_e25e_48f8_9ab8_c361f433594c.slice/crio-aebdfcd8ff9e6105401bebbfc4ad114e5db0b6b15d2e8bbec5c980fa8e8ddfa6 WatchSource:0}: Error finding container aebdfcd8ff9e6105401bebbfc4ad114e5db0b6b15d2e8bbec5c980fa8e8ddfa6: Status 404 returned error can't find the container with id aebdfcd8ff9e6105401bebbfc4ad114e5db0b6b15d2e8bbec5c980fa8e8ddfa6
Jul 20 15:28:03.555638 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:03.555604    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:28:03.564751 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:03.564667    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4kzlg"]
Jul 20 15:28:03.567057 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:28:03.567035    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b5f0a63_5c26_49e1_a580_8b19d8a6e289.slice/crio-31743d799b5e08b20edda314a593c277d9a4550e8401ea868ca6b1bbfdfca71b WatchSource:0}: Error finding container 31743d799b5e08b20edda314a593c277d9a4550e8401ea868ca6b1bbfdfca71b: Status 404 returned error can't find the container with id 31743d799b5e08b20edda314a593c277d9a4550e8401ea868ca6b1bbfdfca71b
Jul 20 15:28:03.636570 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:03.636545    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tnhrs\""
Jul 20 15:28:03.644575 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:03.644557    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7pqm"
Jul 20 15:28:03.655428 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:03.655401    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:28:03.762447 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:03.762419    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-n7pqm"]
Jul 20 15:28:03.765188 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:28:03.765161    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0293426e_c324_435c_873e_468da9602ff6.slice/crio-79bd5677378a35bc734a26a47c5f4d2bbabc192b664d89beb27c307f54e57afd WatchSource:0}: Error finding container 79bd5677378a35bc734a26a47c5f4d2bbabc192b664d89beb27c307f54e57afd: Status 404 returned error can't find the container with id 79bd5677378a35bc734a26a47c5f4d2bbabc192b664d89beb27c307f54e57afd
Jul 20 15:28:03.821019 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:03.820986    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gd8kb" event={"ID":"1797129a-35ab-4364-98a7-7d6dc8f8e6b1","Type":"ContainerStarted","Data":"a5cd44b3adaa3feda8e0c7551840ddb577a79187b8d77f6238123b6215fd3107"}
Jul 20 15:28:03.827187 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:03.827161    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe","Type":"ContainerStarted","Data":"1478574ff0f164e3e590aab8993feb4752138e1d63c5eddcc9c003045811b176"}
Jul 20 15:28:03.827330 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:03.827242    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe","Type":"ContainerStarted","Data":"674ba5a817a0cdaa3c56fc1b6fb0607eea359bd3770c909a81df58f86d99c9e4"}
Jul 20 15:28:03.827330 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:03.827308    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe","Type":"ContainerStarted","Data":"4598eac6e76eb43ec54968e2ad1e7b6171a5c07348786d59a1d8474fd3322bf7"}
Jul 20 15:28:03.827452 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:03.827360    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe","Type":"ContainerStarted","Data":"ee23cdef96b80bc33e3ade61bf141d972caff862a1e41e49c83539ee83d957a4"}
Jul 20 15:28:03.827452 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:03.827399    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe","Type":"ContainerStarted","Data":"aa51b44c82d10b1fa7dd5b7c5b889466651ae7a66fd13ee1af7813af02ef8123"}
Jul 20 15:28:03.827452 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:03.827439    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bcbcfeb8-056b-42d8-be5a-7d0e6b845afe","Type":"ContainerStarted","Data":"906ef61c470f8b201d2dee5ddda5b5cafbe8a103e8992fbbdb9f173d273cf9b9"}
Jul 20 15:28:03.830309 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:03.830270    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-n7pqm" event={"ID":"0293426e-c324-435c-873e-468da9602ff6","Type":"ContainerStarted","Data":"79bd5677378a35bc734a26a47c5f4d2bbabc192b664d89beb27c307f54e57afd"}
Jul 20 15:28:03.832746 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:03.832693    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4kzlg" event={"ID":"7b5f0a63-5c26-49e1-a580-8b19d8a6e289","Type":"ContainerStarted","Data":"31743d799b5e08b20edda314a593c277d9a4550e8401ea868ca6b1bbfdfca71b"}
Jul 20 15:28:03.834220 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:03.834197    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-t54tw" event={"ID":"23b1e60d-e25e-48f8-9ab8-c361f433594c","Type":"ContainerStarted","Data":"aebdfcd8ff9e6105401bebbfc4ad114e5db0b6b15d2e8bbec5c980fa8e8ddfa6"}
Jul 20 15:28:03.861083 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:03.860902    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.120625121 podStartE2EDuration="8.860882917s" totalImagesPullingTime="6.71716982s" totalInitContainerRuntime="23.087976ms" isStatefulPod=false podCreationTimestamp="2026-07-20 15:27:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:28:03.859895563 +0000 UTC m=+66.869517239" watchObservedRunningTime="2026-07-20 15:28:03.860882917 +0000 UTC m=+66.870504580"
Jul 20 15:28:04.364127 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:04.364089    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:28:04.465927 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:04.465890    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:28:05.964438 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:05.964375    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:28:10.222750 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:10.222713    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:28:10.285555 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:10.285526    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:28:10.343054 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:10.343027    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:28:10.866175 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:10.866136    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4kzlg" event={"ID":"7b5f0a63-5c26-49e1-a580-8b19d8a6e289","Type":"ContainerStarted","Data":"0262aebf0f043b0d6af01cd4877960148778d9e1cd09265feebd6697dcc76b35"}
Jul 20 15:28:10.866372 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:10.866207    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4kzlg" event={"ID":"7b5f0a63-5c26-49e1-a580-8b19d8a6e289","Type":"ContainerStarted","Data":"710802bc4b9735f43431154463fa43d5c98974cb3d6e770e14876ad57178fb6e"}
Jul 20 15:28:10.868221 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:10.868190    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-t54tw" event={"ID":"23b1e60d-e25e-48f8-9ab8-c361f433594c","Type":"ContainerStarted","Data":"9c03e1a9cbe803655f643eeb6a0b5b6dbf51a40b563190f3326bfec7c4a78563"}
Jul 20 15:28:10.870320 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:10.870265    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gd8kb" event={"ID":"1797129a-35ab-4364-98a7-7d6dc8f8e6b1","Type":"ContainerStarted","Data":"3769cb8faef6cb30a61a44adff3c08cef52c83ee31c6870a8e2bd38d6a03bee0"}
Jul 20 15:28:10.872123 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:10.872098    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-n7pqm" event={"ID":"0293426e-c324-435c-873e-468da9602ff6","Type":"ContainerStarted","Data":"70b206012f0c6cbc43bb8715192bca8d9bd22a2d09d81c1f45c918f4a3b91023"}
Jul 20 15:28:10.872266 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:10.872253    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-n7pqm"
Jul 20 15:28:10.886077 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:10.886036    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-4kzlg" podStartSLOduration=67.633105726 podStartE2EDuration="1m13.886021642s" totalImagesPullingTime="6.252915916s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:26:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:28:10.884131181 +0000 UTC m=+73.893752838" watchObservedRunningTime="2026-07-20 15:28:10.886021642 +0000 UTC m=+73.895643303"
Jul 20 15:28:10.901268 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:10.901059    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-n7pqm" podStartSLOduration=67.84578918 podStartE2EDuration="1m13.901042974s" totalImagesPullingTime="6.055253794s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:26:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:28:10.899699924 +0000 UTC m=+73.909321588" watchObservedRunningTime="2026-07-20 15:28:10.901042974 +0000 UTC m=+73.910664637"
Jul 20 15:28:10.915110 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:10.915062    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-t54tw" podStartSLOduration=67.637611478 podStartE2EDuration="1m13.9150466s" totalImagesPullingTime="6.277435122s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:26:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:28:10.914164838 +0000 UTC m=+73.923786500" watchObservedRunningTime="2026-07-20 15:28:10.9150466 +0000 UTC m=+73.924668262"
Jul 20 15:28:11.132685 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:11.132599    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:28:11.233708 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:11.233681    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:28:13.886814 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:13.886781    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gd8kb" event={"ID":"1797129a-35ab-4364-98a7-7d6dc8f8e6b1","Type":"ContainerStarted","Data":"f2824c508bcc54eb5fe78b90b78de4865412655f99af5428446630e7f9ce6f59"}
Jul 20 15:28:13.904693 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:13.904650    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-gd8kb" podStartSLOduration=17.230399031 podStartE2EDuration="33.904631163s" totalImagesPullingTime="16.674232132s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:27:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:28:13.904496422 +0000 UTC m=+76.914118084" watchObservedRunningTime="2026-07-20 15:28:13.904631163 +0000 UTC m=+76.914252825"
Jul 20 15:28:14.681301 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:14.681261    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:28:14.740563 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:14.740535    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:28:41.879404 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:41.879368    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-n7pqm"
Jul 20 15:28:44.419910 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:44.419873    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:28:44.487611 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:44.487582    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:28:45.014758 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:45.014732    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 15:28:48.508599 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:48.508546    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:28:48.601715 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:48.601677    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:28:58.061426 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:58.061389    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:28:58.132493 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:58.132467    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:28:58.187734 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:28:58.187709    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:29:03.704197 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:29:03.704155    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:29:03.763961 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:29:03.763922    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:29:08.457379 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:29:08.457340    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:29:08.534617 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:29:08.534586    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:29:08.766519 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:29:08.766486    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:29:08.866681 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:29:08.866654    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:29:09.373986 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:29:09.373951    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:29:09.434629 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:29:09.434603    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:29:10.364821 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:29:10.364785    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:29:10.463796 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:29:10.463762    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:29:10.679081 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:29:10.679010    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:29:15.752142 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:29:15.752111    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:29:15.811421 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:29:15.811392    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:29:19.464000 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:29:19.463965    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:29:19.565531 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:29:19.565496    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:29:19.666104 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:29:19.666072    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:29:29.690171 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:29:29.690135    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:29:29.789469 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:29:29.789436    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:29:29.850813 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:29:29.850779    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:30:03.132575 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:30:03.132494    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:30:03.234627 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:30:03.234598    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:30:41.175866 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:30:41.175822    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:30:41.533568 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:30:41.533540    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:30:45.188687 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:30:45.188655    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:30:45.303527 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:30:45.303501    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:30:46.639408 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:30:46.639380    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:30:48.016807 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:30:48.016772    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:30:49.020702 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:30:49.020668    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:30:49.443961 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:30:49.443893    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:30:51.569737 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:30:51.569699    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:30:51.682396 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:30:51.682369    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:30:54.710037 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:30:54.710010    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:30:54.842729 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:30:54.842701    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:30:56.256332 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:30:56.256299    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:02.144816 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:02.144778    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:02.371548 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:02.371505    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:04.059662 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:04.059623    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:04.161034 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:04.160994    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:07.568865 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:07.568827    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:07.688387 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:07.688339    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:09.859947 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:09.859906    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:11.670217 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:11.670189    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:19.943316 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:19.943211    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:21.379267 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:21.379228    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:21.476631 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:21.476591    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:22.028861 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:22.028831    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:22.422419 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:22.422344    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:22.523703 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:22.523674    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:22.550962 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:22.550941    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-srbk6"]
Jul 20 15:31:22.553457 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:22.553444    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-srbk6"
Jul 20 15:31:22.557388 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:22.557372    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 20 15:31:22.557597 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:22.557585    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-k4m5s\""
Jul 20 15:31:22.558148 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:22.558132    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 20 15:31:22.558231 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:22.558161    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 20 15:31:22.561497 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:22.561476    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 20 15:31:22.581524 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:22.581503    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-srbk6"]
Jul 20 15:31:22.670978 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:22.670949    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65bbbccfd9-srbk6\" (UID: \"4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f\") " pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-srbk6"
Jul 20 15:31:22.671082 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:22.670982    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f-webhook-cert\") pod \"opendatahub-operator-controller-manager-65bbbccfd9-srbk6\" (UID: \"4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f\") " pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-srbk6"
Jul 20 15:31:22.671082 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:22.671004    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f-tmp\") pod \"opendatahub-operator-controller-manager-65bbbccfd9-srbk6\" (UID: \"4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f\") " pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-srbk6"
Jul 20 15:31:22.671082 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:22.671026    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gs678\" (UniqueName: \"kubernetes.io/projected/4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f-kube-api-access-gs678\") pod \"opendatahub-operator-controller-manager-65bbbccfd9-srbk6\" (UID: \"4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f\") " pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-srbk6"
Jul 20 15:31:22.671185 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:22.671087    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f-manifests\") pod \"opendatahub-operator-controller-manager-65bbbccfd9-srbk6\" (UID: \"4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f\") " pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-srbk6"
Jul 20 15:31:22.736727 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:22.736660    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:22.772335 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:22.772314    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65bbbccfd9-srbk6\" (UID: \"4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f\") " pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-srbk6"
Jul 20 15:31:22.772417 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:22.772351    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f-webhook-cert\") pod \"opendatahub-operator-controller-manager-65bbbccfd9-srbk6\" (UID: \"4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f\") " pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-srbk6"
Jul 20 15:31:22.772417 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:22.772374    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f-tmp\") pod \"opendatahub-operator-controller-manager-65bbbccfd9-srbk6\" (UID: \"4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f\") " pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-srbk6"
Jul 20 15:31:22.772709 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:22.772687    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs678\" (UniqueName: \"kubernetes.io/projected/4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f-kube-api-access-gs678\") pod \"opendatahub-operator-controller-manager-65bbbccfd9-srbk6\" (UID: \"4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f\") " pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-srbk6"
Jul 20 15:31:22.772787 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:22.772741    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f-manifests\") pod \"opendatahub-operator-controller-manager-65bbbccfd9-srbk6\" (UID: \"4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f\") " pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-srbk6"
Jul 20 15:31:22.773068 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:22.773042    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f-manifests\") pod \"opendatahub-operator-controller-manager-65bbbccfd9-srbk6\" (UID: \"4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f\") " pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-srbk6"
Jul 20 15:31:22.774579 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:22.774563    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f-tmp\") pod \"opendatahub-operator-controller-manager-65bbbccfd9-srbk6\" (UID: \"4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f\") " pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-srbk6"
Jul 20 15:31:22.774783 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:22.774766    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f-webhook-cert\") pod \"opendatahub-operator-controller-manager-65bbbccfd9-srbk6\" (UID: \"4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f\") " pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-srbk6"
Jul 20 15:31:22.774845 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:22.774800    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65bbbccfd9-srbk6\" (UID: \"4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f\") " pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-srbk6"
Jul 20 15:31:22.781396 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:22.781374    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gs678\" (UniqueName: \"kubernetes.io/projected/4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f-kube-api-access-gs678\") pod \"opendatahub-operator-controller-manager-65bbbccfd9-srbk6\" (UID: \"4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f\") " pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-srbk6"
Jul 20 15:31:22.831147 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:22.831120    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:22.863823 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:22.863803    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-srbk6"
Jul 20 15:31:22.932553 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:22.932526    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:23.044575 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:23.044552    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-srbk6"]
Jul 20 15:31:23.046863 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:31:23.046835    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a1bc22f_d0b8_4034_8e37_db8cb8c7f00f.slice/crio-f1d227064f8ba19f3825ea170f7a12cfe3c67a2c952ce27ec508819e1fadc166 WatchSource:0}: Error finding container f1d227064f8ba19f3825ea170f7a12cfe3c67a2c952ce27ec508819e1fadc166: Status 404 returned error can't find the container with id f1d227064f8ba19f3825ea170f7a12cfe3c67a2c952ce27ec508819e1fadc166
Jul 20 15:31:23.133232 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:23.133205    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:23.208785 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:23.208763    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:23.336045 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:23.335991    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:23.422559 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:23.422540    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:23.542611 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:23.542586    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-srbk6" event={"ID":"4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f","Type":"ContainerStarted","Data":"f1d227064f8ba19f3825ea170f7a12cfe3c67a2c952ce27ec508819e1fadc166"}
Jul 20 15:31:26.556606 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:26.556565    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-srbk6" event={"ID":"4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f","Type":"ContainerStarted","Data":"d312c3f74b7bf760ca370261df38c8ca11af82a4b1d7a48ac8e4f377c081f9d7"}
Jul 20 15:31:27.561158 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:27.561128    2559 generic.go:356] "Generic (PLEG): container finished" podID="4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f" containerID="d312c3f74b7bf760ca370261df38c8ca11af82a4b1d7a48ac8e4f377c081f9d7" exitCode=0
Jul 20 15:31:27.561158 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:27.561162    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-srbk6" event={"ID":"4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f","Type":"ContainerDied","Data":"d312c3f74b7bf760ca370261df38c8ca11af82a4b1d7a48ac8e4f377c081f9d7"}
Jul 20 15:31:28.567559 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:28.567523    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-srbk6" event={"ID":"4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f","Type":"ContainerStarted","Data":"1b546a43dea435b95f56a5aff967f317d24bd92342c527852dff8a268cc4b7c7"}
Jul 20 15:31:28.567958 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:28.567661    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-srbk6"
Jul 20 15:31:31.955763 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:31.955731    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:32.068671 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:32.068627    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:32.177059 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:32.177028    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:32.330773 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:32.330744    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:33.798754 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:33.798699    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:39.573326 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:39.573299    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-srbk6"
Jul 20 15:31:39.600300 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:39.600247    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-srbk6" podStartSLOduration=12.975196184 podStartE2EDuration="17.600232777s" totalImagesPullingTime="3.631489436s" totalInitContainerRuntime="993.547157ms" isStatefulPod=false podCreationTimestamp="2026-07-20 15:31:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:31:28.619914881 +0000 UTC m=+271.629536544" watchObservedRunningTime="2026-07-20 15:31:39.600232777 +0000 UTC m=+282.609854438"
Jul 20 15:31:42.111561 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:42.111506    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:42.229354 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:42.229322    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:42.820348 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:42.820317    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:42.929777 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:42.929739    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6598cc5cf9-hg6cj"]
Jul 20 15:31:42.941388 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:42.941359    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6598cc5cf9-hg6cj"
Jul 20 15:31:42.951426 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:42.951403    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 20 15:31:42.951763 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:42.951412    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 20 15:31:42.952077 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:42.952063    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 20 15:31:42.952226 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:42.952208    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 20 15:31:42.953338 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:42.953315    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6598cc5cf9-hg6cj"]
Jul 20 15:31:42.962092 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:42.962064    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-cd2b4"]
Jul 20 15:31:42.964659 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:42.964637    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-cd2b4"
Jul 20 15:31:42.969496 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:42.969472    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 20 15:31:42.969599 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:42.969556    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-2b9st\""
Jul 20 15:31:43.005718 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.005691    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-cd2b4"]
Jul 20 15:31:43.036171 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.036139    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k56z\" (UniqueName: \"kubernetes.io/projected/9f41b33a-4491-4c6c-a238-636fe2b70a46-kube-api-access-8k56z\") pod \"kube-auth-proxy-6598cc5cf9-hg6cj\" (UID: \"9f41b33a-4491-4c6c-a238-636fe2b70a46\") " pod="openshift-ingress/kube-auth-proxy-6598cc5cf9-hg6cj"
Jul 20 15:31:43.036324 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.036184    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61b825d3-0c47-4a83-a3b9-04cf6a519bec-cert\") pod \"odh-model-controller-7998c67dc4-cd2b4\" (UID: \"61b825d3-0c47-4a83-a3b9-04cf6a519bec\") " pod="opendatahub/odh-model-controller-7998c67dc4-cd2b4"
Jul 20 15:31:43.036324 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.036212    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f41b33a-4491-4c6c-a238-636fe2b70a46-tls-certs\") pod \"kube-auth-proxy-6598cc5cf9-hg6cj\" (UID: \"9f41b33a-4491-4c6c-a238-636fe2b70a46\") " pod="openshift-ingress/kube-auth-proxy-6598cc5cf9-hg6cj"
Jul 20 15:31:43.036324 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.036233    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f41b33a-4491-4c6c-a238-636fe2b70a46-tmp\") pod \"kube-auth-proxy-6598cc5cf9-hg6cj\" (UID: \"9f41b33a-4491-4c6c-a238-636fe2b70a46\") " pod="openshift-ingress/kube-auth-proxy-6598cc5cf9-hg6cj"
Jul 20 15:31:43.036455 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.036346    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnwt2\" (UniqueName: \"kubernetes.io/projected/61b825d3-0c47-4a83-a3b9-04cf6a519bec-kube-api-access-fnwt2\") pod \"odh-model-controller-7998c67dc4-cd2b4\" (UID: \"61b825d3-0c47-4a83-a3b9-04cf6a519bec\") " pod="opendatahub/odh-model-controller-7998c67dc4-cd2b4"
Jul 20 15:31:43.117216 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.117133    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:43.137430 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.137400    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k56z\" (UniqueName: \"kubernetes.io/projected/9f41b33a-4491-4c6c-a238-636fe2b70a46-kube-api-access-8k56z\") pod \"kube-auth-proxy-6598cc5cf9-hg6cj\" (UID: \"9f41b33a-4491-4c6c-a238-636fe2b70a46\") " pod="openshift-ingress/kube-auth-proxy-6598cc5cf9-hg6cj"
Jul 20 15:31:43.137634 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.137619    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61b825d3-0c47-4a83-a3b9-04cf6a519bec-cert\") pod \"odh-model-controller-7998c67dc4-cd2b4\" (UID: \"61b825d3-0c47-4a83-a3b9-04cf6a519bec\") " pod="opendatahub/odh-model-controller-7998c67dc4-cd2b4"
Jul 20 15:31:43.137737 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.137724    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f41b33a-4491-4c6c-a238-636fe2b70a46-tls-certs\") pod \"kube-auth-proxy-6598cc5cf9-hg6cj\" (UID: \"9f41b33a-4491-4c6c-a238-636fe2b70a46\") " pod="openshift-ingress/kube-auth-proxy-6598cc5cf9-hg6cj"
Jul 20 15:31:43.137826 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.137811    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f41b33a-4491-4c6c-a238-636fe2b70a46-tmp\") pod \"kube-auth-proxy-6598cc5cf9-hg6cj\" (UID: \"9f41b33a-4491-4c6c-a238-636fe2b70a46\") " pod="openshift-ingress/kube-auth-proxy-6598cc5cf9-hg6cj"
Jul 20 15:31:43.137930 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.137918    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnwt2\" (UniqueName: \"kubernetes.io/projected/61b825d3-0c47-4a83-a3b9-04cf6a519bec-kube-api-access-fnwt2\") pod \"odh-model-controller-7998c67dc4-cd2b4\" (UID: \"61b825d3-0c47-4a83-a3b9-04cf6a519bec\") " pod="opendatahub/odh-model-controller-7998c67dc4-cd2b4"
Jul 20 15:31:43.138421 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:31:43.138400    2559 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 20 15:31:43.139228 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:31:43.139212    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61b825d3-0c47-4a83-a3b9-04cf6a519bec-cert podName:61b825d3-0c47-4a83-a3b9-04cf6a519bec nodeName:}" failed. No retries permitted until 2026-07-20 15:31:43.639185922 +0000 UTC m=+286.648807766 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/61b825d3-0c47-4a83-a3b9-04cf6a519bec-cert") pod "odh-model-controller-7998c67dc4-cd2b4" (UID: "61b825d3-0c47-4a83-a3b9-04cf6a519bec") : secret "odh-model-controller-webhook-cert" not found
Jul 20 15:31:43.141480 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.141458    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f41b33a-4491-4c6c-a238-636fe2b70a46-tls-certs\") pod \"kube-auth-proxy-6598cc5cf9-hg6cj\" (UID: \"9f41b33a-4491-4c6c-a238-636fe2b70a46\") " pod="openshift-ingress/kube-auth-proxy-6598cc5cf9-hg6cj"
Jul 20 15:31:43.141844 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.141816    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f41b33a-4491-4c6c-a238-636fe2b70a46-tmp\") pod \"kube-auth-proxy-6598cc5cf9-hg6cj\" (UID: \"9f41b33a-4491-4c6c-a238-636fe2b70a46\") " pod="openshift-ingress/kube-auth-proxy-6598cc5cf9-hg6cj"
Jul 20 15:31:43.153769 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.153748    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnwt2\" (UniqueName: \"kubernetes.io/projected/61b825d3-0c47-4a83-a3b9-04cf6a519bec-kube-api-access-fnwt2\") pod \"odh-model-controller-7998c67dc4-cd2b4\" (UID: \"61b825d3-0c47-4a83-a3b9-04cf6a519bec\") " pod="opendatahub/odh-model-controller-7998c67dc4-cd2b4"
Jul 20 15:31:43.154139 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.154119    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k56z\" (UniqueName: \"kubernetes.io/projected/9f41b33a-4491-4c6c-a238-636fe2b70a46-kube-api-access-8k56z\") pod \"kube-auth-proxy-6598cc5cf9-hg6cj\" (UID: \"9f41b33a-4491-4c6c-a238-636fe2b70a46\") " pod="openshift-ingress/kube-auth-proxy-6598cc5cf9-hg6cj"
Jul 20 15:31:43.240346 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.240316    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b8ffc8644-n5rcf"]
Jul 20 15:31:43.242788 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.242771    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5b8ffc8644-n5rcf"
Jul 20 15:31:43.246180 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.246160    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 20 15:31:43.246269 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.246164    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 20 15:31:43.246269 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.246196    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 20 15:31:43.246988 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.246966    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 20 15:31:43.246988 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.246980    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-tz2kk\""
Jul 20 15:31:43.247125 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.247042    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 20 15:31:43.252233 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.252216    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6598cc5cf9-hg6cj"
Jul 20 15:31:43.256816 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.256793    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b8ffc8644-n5rcf"]
Jul 20 15:31:43.274731 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.274250    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:43.340352 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.340317    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlrrj\" (UniqueName: \"kubernetes.io/projected/e65179e3-c7f6-4d9e-9a4f-99f773d3e1c9-kube-api-access-tlrrj\") pod \"lws-controller-manager-5b8ffc8644-n5rcf\" (UID: \"e65179e3-c7f6-4d9e-9a4f-99f773d3e1c9\") " pod="openshift-lws-operator/lws-controller-manager-5b8ffc8644-n5rcf"
Jul 20 15:31:43.340494 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.340386    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e65179e3-c7f6-4d9e-9a4f-99f773d3e1c9-cert\") pod \"lws-controller-manager-5b8ffc8644-n5rcf\" (UID: \"e65179e3-c7f6-4d9e-9a4f-99f773d3e1c9\") " pod="openshift-lws-operator/lws-controller-manager-5b8ffc8644-n5rcf"
Jul 20 15:31:43.340494 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.340422    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e65179e3-c7f6-4d9e-9a4f-99f773d3e1c9-metrics-cert\") pod \"lws-controller-manager-5b8ffc8644-n5rcf\" (UID: \"e65179e3-c7f6-4d9e-9a4f-99f773d3e1c9\") " pod="openshift-lws-operator/lws-controller-manager-5b8ffc8644-n5rcf"
Jul 20 15:31:43.340494 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.340491    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e65179e3-c7f6-4d9e-9a4f-99f773d3e1c9-manager-config\") pod \"lws-controller-manager-5b8ffc8644-n5rcf\" (UID: \"e65179e3-c7f6-4d9e-9a4f-99f773d3e1c9\") " pod="openshift-lws-operator/lws-controller-manager-5b8ffc8644-n5rcf"
Jul 20 15:31:43.349710 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.349568    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:43.405717 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.405646    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6598cc5cf9-hg6cj"]
Jul 20 15:31:43.408996 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:31:43.408972    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f41b33a_4491_4c6c_a238_636fe2b70a46.slice/crio-dd75988e41e9e603f72a4de9c5fdfb7405a73208fbc9b2758c4b06b1a3cda298 WatchSource:0}: Error finding container dd75988e41e9e603f72a4de9c5fdfb7405a73208fbc9b2758c4b06b1a3cda298: Status 404 returned error can't find the container with id dd75988e41e9e603f72a4de9c5fdfb7405a73208fbc9b2758c4b06b1a3cda298
Jul 20 15:31:43.441709 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.441681    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e65179e3-c7f6-4d9e-9a4f-99f773d3e1c9-metrics-cert\") pod \"lws-controller-manager-5b8ffc8644-n5rcf\" (UID: \"e65179e3-c7f6-4d9e-9a4f-99f773d3e1c9\") " pod="openshift-lws-operator/lws-controller-manager-5b8ffc8644-n5rcf"
Jul 20 15:31:43.441808 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.441749    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e65179e3-c7f6-4d9e-9a4f-99f773d3e1c9-manager-config\") pod \"lws-controller-manager-5b8ffc8644-n5rcf\" (UID: \"e65179e3-c7f6-4d9e-9a4f-99f773d3e1c9\") " pod="openshift-lws-operator/lws-controller-manager-5b8ffc8644-n5rcf"
Jul 20 15:31:43.441874 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.441827    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlrrj\" (UniqueName: \"kubernetes.io/projected/e65179e3-c7f6-4d9e-9a4f-99f773d3e1c9-kube-api-access-tlrrj\") pod \"lws-controller-manager-5b8ffc8644-n5rcf\" (UID: \"e65179e3-c7f6-4d9e-9a4f-99f773d3e1c9\") " pod="openshift-lws-operator/lws-controller-manager-5b8ffc8644-n5rcf"
Jul 20 15:31:43.441930 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.441890    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e65179e3-c7f6-4d9e-9a4f-99f773d3e1c9-cert\") pod \"lws-controller-manager-5b8ffc8644-n5rcf\" (UID: \"e65179e3-c7f6-4d9e-9a4f-99f773d3e1c9\") " pod="openshift-lws-operator/lws-controller-manager-5b8ffc8644-n5rcf"
Jul 20 15:31:43.442392 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.442365    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e65179e3-c7f6-4d9e-9a4f-99f773d3e1c9-manager-config\") pod \"lws-controller-manager-5b8ffc8644-n5rcf\" (UID: \"e65179e3-c7f6-4d9e-9a4f-99f773d3e1c9\") " pod="openshift-lws-operator/lws-controller-manager-5b8ffc8644-n5rcf"
Jul 20 15:31:43.444111 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.444092    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e65179e3-c7f6-4d9e-9a4f-99f773d3e1c9-cert\") pod \"lws-controller-manager-5b8ffc8644-n5rcf\" (UID: \"e65179e3-c7f6-4d9e-9a4f-99f773d3e1c9\") " pod="openshift-lws-operator/lws-controller-manager-5b8ffc8644-n5rcf"
Jul 20 15:31:43.444333 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.444313    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e65179e3-c7f6-4d9e-9a4f-99f773d3e1c9-metrics-cert\") pod \"lws-controller-manager-5b8ffc8644-n5rcf\" (UID: \"e65179e3-c7f6-4d9e-9a4f-99f773d3e1c9\") " pod="openshift-lws-operator/lws-controller-manager-5b8ffc8644-n5rcf"
Jul 20 15:31:43.451786 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.451764    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlrrj\" (UniqueName: \"kubernetes.io/projected/e65179e3-c7f6-4d9e-9a4f-99f773d3e1c9-kube-api-access-tlrrj\") pod \"lws-controller-manager-5b8ffc8644-n5rcf\" (UID: \"e65179e3-c7f6-4d9e-9a4f-99f773d3e1c9\") " pod="openshift-lws-operator/lws-controller-manager-5b8ffc8644-n5rcf"
Jul 20 15:31:43.552561 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.552524    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5b8ffc8644-n5rcf"
Jul 20 15:31:43.621388 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.621356    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6598cc5cf9-hg6cj" event={"ID":"9f41b33a-4491-4c6c-a238-636fe2b70a46","Type":"ContainerStarted","Data":"dd75988e41e9e603f72a4de9c5fdfb7405a73208fbc9b2758c4b06b1a3cda298"}
Jul 20 15:31:43.643861 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.643834    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61b825d3-0c47-4a83-a3b9-04cf6a519bec-cert\") pod \"odh-model-controller-7998c67dc4-cd2b4\" (UID: \"61b825d3-0c47-4a83-a3b9-04cf6a519bec\") " pod="opendatahub/odh-model-controller-7998c67dc4-cd2b4"
Jul 20 15:31:43.646928 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.646906    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61b825d3-0c47-4a83-a3b9-04cf6a519bec-cert\") pod \"odh-model-controller-7998c67dc4-cd2b4\" (UID: \"61b825d3-0c47-4a83-a3b9-04cf6a519bec\") " pod="opendatahub/odh-model-controller-7998c67dc4-cd2b4"
Jul 20 15:31:43.692914 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.692888    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b8ffc8644-n5rcf"]
Jul 20 15:31:43.695237 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:31:43.695214    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode65179e3_c7f6_4d9e_9a4f_99f773d3e1c9.slice/crio-92b9ce178bc128a42c24b188dea6b44a5c8b0190c5e4fd38a7de4bb231f9f7a7 WatchSource:0}: Error finding container 92b9ce178bc128a42c24b188dea6b44a5c8b0190c5e4fd38a7de4bb231f9f7a7: Status 404 returned error can't find the container with id 92b9ce178bc128a42c24b188dea6b44a5c8b0190c5e4fd38a7de4bb231f9f7a7
Jul 20 15:31:43.875423 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:43.875385    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-cd2b4"
Jul 20 15:31:44.057080 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:44.057051    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-cd2b4"]
Jul 20 15:31:44.059950 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:31:44.059925    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61b825d3_0c47_4a83_a3b9_04cf6a519bec.slice/crio-6d4b0afb517d8801d60fb0e31269f034636c0337d28efc932098ddc7282269b8 WatchSource:0}: Error finding container 6d4b0afb517d8801d60fb0e31269f034636c0337d28efc932098ddc7282269b8: Status 404 returned error can't find the container with id 6d4b0afb517d8801d60fb0e31269f034636c0337d28efc932098ddc7282269b8
Jul 20 15:31:44.141421 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:44.141396    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:44.432651 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:44.432607    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:44.628446 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:44.628402    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5b8ffc8644-n5rcf" event={"ID":"e65179e3-c7f6-4d9e-9a4f-99f773d3e1c9","Type":"ContainerStarted","Data":"92b9ce178bc128a42c24b188dea6b44a5c8b0190c5e4fd38a7de4bb231f9f7a7"}
Jul 20 15:31:44.629942 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:44.629869    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-cd2b4" event={"ID":"61b825d3-0c47-4a83-a3b9-04cf6a519bec","Type":"ContainerStarted","Data":"6d4b0afb517d8801d60fb0e31269f034636c0337d28efc932098ddc7282269b8"}
Jul 20 15:31:45.133918 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:45.133814    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:45.272075 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:45.272041    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:46.655794 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:46.655761    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:49.655665 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:49.655628    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5b8ffc8644-n5rcf" event={"ID":"e65179e3-c7f6-4d9e-9a4f-99f773d3e1c9","Type":"ContainerStarted","Data":"bf33c2ac51fe77a576a846ca2552e7930d919ed74a4f3478e7fbd53a6fb89298"}
Jul 20 15:31:49.656169 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:49.655757    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5b8ffc8644-n5rcf"
Jul 20 15:31:49.657117 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:49.657093    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-cd2b4" event={"ID":"61b825d3-0c47-4a83-a3b9-04cf6a519bec","Type":"ContainerStarted","Data":"130317c200873bc06504ec4b6abb58f9edb02b7d49eee102b74837512743972e"}
Jul 20 15:31:49.657260 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:49.657201    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-cd2b4"
Jul 20 15:31:49.658516 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:49.658496    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6598cc5cf9-hg6cj" event={"ID":"9f41b33a-4491-4c6c-a238-636fe2b70a46","Type":"ContainerStarted","Data":"6046408196c2c20f3b143b0f238946ab2bec8d0d292756acaaa1344082914062"}
Jul 20 15:31:49.684188 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:49.684123    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5b8ffc8644-n5rcf" podStartSLOduration=1.6487909859999998 podStartE2EDuration="6.684103059s" totalImagesPullingTime="5.035312073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:31:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:31:49.681099264 +0000 UTC m=+292.690720927" watchObservedRunningTime="2026-07-20 15:31:49.684103059 +0000 UTC m=+292.693724723"
Jul 20 15:31:49.704387 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:49.704337    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6598cc5cf9-hg6cj" podStartSLOduration=2.380474429 podStartE2EDuration="7.704318057s" totalImagesPullingTime="5.323843628s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:31:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:31:49.703672 +0000 UTC m=+292.713293665" watchObservedRunningTime="2026-07-20 15:31:49.704318057 +0000 UTC m=+292.713939721"
Jul 20 15:31:49.709642 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:49.709618    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:49.727592 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:49.727550    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-cd2b4" podStartSLOduration=2.993655549 podStartE2EDuration="7.727534978s" totalImagesPullingTime="4.733879429s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:31:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:31:49.725879747 +0000 UTC m=+292.735501408" watchObservedRunningTime="2026-07-20 15:31:49.727534978 +0000 UTC m=+292.737156640"
Jul 20 15:31:49.808189 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:49.808157    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:50.577083 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:50.577045    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:50.904521 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:50.904440    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:51.009277 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:51.009251    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:55.084765 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:55.084719    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:55.195736 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:55.195698    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-jg2lz"]
Jul 20 15:31:55.198837 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:55.198819    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jg2lz"
Jul 20 15:31:55.202246 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:55.202224    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 20 15:31:55.202510 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:55.202493    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-dbg5p\""
Jul 20 15:31:55.226826 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:55.226805    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-jg2lz"]
Jul 20 15:31:55.264966 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:55.264943    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:55.346160 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:55.346087    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9m656\" (UniqueName: \"kubernetes.io/projected/82c3d216-9a4e-4896-b740-fe3448a8e2e1-kube-api-access-9m656\") pod \"llmisvc-controller-manager-777c6bd7d5-jg2lz\" (UID: \"82c3d216-9a4e-4896-b740-fe3448a8e2e1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jg2lz"
Jul 20 15:31:55.346160 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:55.346131    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/82c3d216-9a4e-4896-b740-fe3448a8e2e1-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jg2lz\" (UID: \"82c3d216-9a4e-4896-b740-fe3448a8e2e1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jg2lz"
Jul 20 15:31:55.446724 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:55.446693    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9m656\" (UniqueName: \"kubernetes.io/projected/82c3d216-9a4e-4896-b740-fe3448a8e2e1-kube-api-access-9m656\") pod \"llmisvc-controller-manager-777c6bd7d5-jg2lz\" (UID: \"82c3d216-9a4e-4896-b740-fe3448a8e2e1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jg2lz"
Jul 20 15:31:55.446891 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:55.446737    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/82c3d216-9a4e-4896-b740-fe3448a8e2e1-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jg2lz\" (UID: \"82c3d216-9a4e-4896-b740-fe3448a8e2e1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jg2lz"
Jul 20 15:31:55.446891 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:31:55.446839    2559 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 20 15:31:55.446891 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:31:55.446891    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82c3d216-9a4e-4896-b740-fe3448a8e2e1-cert podName:82c3d216-9a4e-4896-b740-fe3448a8e2e1 nodeName:}" failed. No retries permitted until 2026-07-20 15:31:55.946874775 +0000 UTC m=+298.956496414 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/82c3d216-9a4e-4896-b740-fe3448a8e2e1-cert") pod "llmisvc-controller-manager-777c6bd7d5-jg2lz" (UID: "82c3d216-9a4e-4896-b740-fe3448a8e2e1") : secret "llmisvc-webhook-server-cert" not found
Jul 20 15:31:55.458407 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:55.458384    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9m656\" (UniqueName: \"kubernetes.io/projected/82c3d216-9a4e-4896-b740-fe3448a8e2e1-kube-api-access-9m656\") pod \"llmisvc-controller-manager-777c6bd7d5-jg2lz\" (UID: \"82c3d216-9a4e-4896-b740-fe3448a8e2e1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jg2lz"
Jul 20 15:31:55.512166 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:55.512142    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:55.885109 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:55.885073    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:55.950641 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:55.950608    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/82c3d216-9a4e-4896-b740-fe3448a8e2e1-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jg2lz\" (UID: \"82c3d216-9a4e-4896-b740-fe3448a8e2e1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jg2lz"
Jul 20 15:31:55.952915 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:55.952893    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/82c3d216-9a4e-4896-b740-fe3448a8e2e1-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jg2lz\" (UID: \"82c3d216-9a4e-4896-b740-fe3448a8e2e1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jg2lz"
Jul 20 15:31:56.110832 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:56.110801    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jg2lz"
Jul 20 15:31:56.290001 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:56.289975    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-jg2lz"]
Jul 20 15:31:56.308903 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:31:56.308867    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod82c3d216_9a4e_4896_b740_fe3448a8e2e1.slice/crio-86c77c32933a431601cb707e9453b458e55afb9045033234c060f7107992f014 WatchSource:0}: Error finding container 86c77c32933a431601cb707e9453b458e55afb9045033234c060f7107992f014: Status 404 returned error can't find the container with id 86c77c32933a431601cb707e9453b458e55afb9045033234c060f7107992f014
Jul 20 15:31:56.432979 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:56.432904    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:56.532980 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:56.532946    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:56.649928 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:56.649893    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:31:56.686073 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:56.686007    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jg2lz" event={"ID":"82c3d216-9a4e-4896-b740-fe3448a8e2e1","Type":"ContainerStarted","Data":"86c77c32933a431601cb707e9453b458e55afb9045033234c060f7107992f014"}
Jul 20 15:31:57.734876 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:31:57.734854    2559 kubelet.go:1648] "Image garbage collection succeeded"
Jul 20 15:32:00.665076 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:32:00.665045    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-cd2b4"
Jul 20 15:32:00.665568 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:32:00.665126    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5b8ffc8644-n5rcf"
Jul 20 15:32:00.709430 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:32:00.709401    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jg2lz" event={"ID":"82c3d216-9a4e-4896-b740-fe3448a8e2e1","Type":"ContainerStarted","Data":"8a4910e0590e1aaf1b00a7404bf24e3ec05850816fcba6434b3ce5804428e9af"}
Jul 20 15:32:00.709592 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:32:00.709540    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jg2lz"
Jul 20 15:32:05.330570 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:32:05.330533    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:32:31.715860 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:32:31.715832    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jg2lz"
Jul 20 15:32:31.735750 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:32:31.735707    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jg2lz" podStartSLOduration=33.376786309 podStartE2EDuration="36.735691138s" totalImagesPullingTime="3.358904829s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:31:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:32:00.772784479 +0000 UTC m=+303.782406141" watchObservedRunningTime="2026-07-20 15:32:31.735691138 +0000 UTC m=+334.745312817"
Jul 20 15:32:43.440695 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:32:43.440662    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:32:43.540339 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:32:43.540296    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:32:43.634697 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:32:43.634669    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:33:10.967322 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:33:10.967210    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:33:13.076381 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:33:13.076356    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:33:17.306993 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:33:17.306961    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:33:19.696707 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:33:19.696676    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:33:19.849444 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:33:19.849413    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:33:21.133508 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:33:21.133476    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:33:24.269767 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:33:24.269733    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:33:24.658698 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:33:24.658624    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:33:25.029106 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:33:25.029073    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:33:27.706628 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:33:27.706584    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:33:29.052253 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:33:29.052202    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:33:29.156980 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:33:29.156952    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:33:32.850707 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:33:32.850670    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:33:32.967547 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:33:32.967516    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:33:33.535254 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:33:33.535225    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:33:33.636096 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:33:33.636068    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:33:33.916100 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:33:33.916019    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:33:34.217647 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:33:34.217573    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:33:36.267597 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:33:36.267568    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:33:36.352945 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:33:36.352900    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:33:44.722838 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:33:44.722808    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:33:44.820386 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:33:44.820351    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:33:44.924801 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:33:44.924770    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:33:47.176304 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:33:47.176248    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:33:47.278633 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:33:47.278593    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:33:49.998183 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:33:49.998136    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:33:50.101240 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:33:50.101199    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:33:53.584416 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:33:53.584373    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:33:53.685452 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:33:53.685423    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:33:58.153518 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:33:58.153478    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:33:58.218534 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:33:58.218509    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:34:02.406081 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:02.406038    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:34:02.506235 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:02.506201    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:34:05.007906 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:05.007864    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:34:05.111557 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:05.111506    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:34:08.226975 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:08.226942    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:34:08.328893 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:08.328864    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:34:09.025251 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:09.025222    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:34:09.115476 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:09.115440    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:34:09.220364 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:09.220327    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:34:11.818423 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:11.818392    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:34:11.923486 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:11.923447    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:34:21.814697 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:21.814652    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:34:21.933929 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:21.933891    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:34:22.315266 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:22.315234    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:34:22.422806 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:22.422779    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:34:25.582883 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:25.582845    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:34:25.697685 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:25.697653    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:34:42.457700 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:42.457655    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:34:42.480021 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:42.479994    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jzk5p"]
Jul 20 15:34:42.483096 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:42.483081    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-jzk5p"
Jul 20 15:34:42.487075 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:42.487057    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 20 15:34:42.487278 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:42.487267    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-x6bxm\""
Jul 20 15:34:42.487523 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:42.487509    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 20 15:34:42.488443 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:42.488422    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 20 15:34:42.502018 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:42.501993    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jzk5p"]
Jul 20 15:34:42.515211 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:42.515189    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6sk4\" (UniqueName: \"kubernetes.io/projected/e7677bbf-90b5-42dd-b9a8-43bf7983e072-kube-api-access-f6sk4\") pod \"limitador-limitador-64c94fd64f-jzk5p\" (UID: \"e7677bbf-90b5-42dd-b9a8-43bf7983e072\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jzk5p"
Jul 20 15:34:42.515333 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:42.515224    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e7677bbf-90b5-42dd-b9a8-43bf7983e072-config-file\") pod \"limitador-limitador-64c94fd64f-jzk5p\" (UID: \"e7677bbf-90b5-42dd-b9a8-43bf7983e072\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jzk5p"
Jul 20 15:34:42.522116 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:42.522091    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jzk5p"]
Jul 20 15:34:42.553450 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:42.553425    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:34:42.616316 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:42.616273    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6sk4\" (UniqueName: \"kubernetes.io/projected/e7677bbf-90b5-42dd-b9a8-43bf7983e072-kube-api-access-f6sk4\") pod \"limitador-limitador-64c94fd64f-jzk5p\" (UID: \"e7677bbf-90b5-42dd-b9a8-43bf7983e072\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jzk5p"
Jul 20 15:34:42.616429 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:42.616328    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e7677bbf-90b5-42dd-b9a8-43bf7983e072-config-file\") pod \"limitador-limitador-64c94fd64f-jzk5p\" (UID: \"e7677bbf-90b5-42dd-b9a8-43bf7983e072\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jzk5p"
Jul 20 15:34:42.616971 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:42.616954    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e7677bbf-90b5-42dd-b9a8-43bf7983e072-config-file\") pod \"limitador-limitador-64c94fd64f-jzk5p\" (UID: \"e7677bbf-90b5-42dd-b9a8-43bf7983e072\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jzk5p"
Jul 20 15:34:42.624562 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:42.624521    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6sk4\" (UniqueName: \"kubernetes.io/projected/e7677bbf-90b5-42dd-b9a8-43bf7983e072-kube-api-access-f6sk4\") pod \"limitador-limitador-64c94fd64f-jzk5p\" (UID: \"e7677bbf-90b5-42dd-b9a8-43bf7983e072\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jzk5p"
Jul 20 15:34:42.627559 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:42.627537    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:34:42.793432 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:42.793397    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-jzk5p"
Jul 20 15:34:42.917019 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:42.916997    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jzk5p"]
Jul 20 15:34:42.919070 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:34:42.919045    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7677bbf_90b5_42dd_b9a8_43bf7983e072.slice/crio-cd2af66d7ef417cc3fb91bf77aca39a98ff787da4ad61fa91004c644c786de8b WatchSource:0}: Error finding container cd2af66d7ef417cc3fb91bf77aca39a98ff787da4ad61fa91004c644c786de8b: Status 404 returned error can't find the container with id cd2af66d7ef417cc3fb91bf77aca39a98ff787da4ad61fa91004c644c786de8b
Jul 20 15:34:42.920937 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:42.920917    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 15:34:43.266082 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:43.266050    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-cxjb4"]
Jul 20 15:34:43.270530 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:43.270510    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-cxjb4"
Jul 20 15:34:43.273414 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:43.273394    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-bhtws\""
Jul 20 15:34:43.276632 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:43.276605    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-cxjb4"]
Jul 20 15:34:43.306842 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:43.306817    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:34:43.321855 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:43.321827    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjltl\" (UniqueName: \"kubernetes.io/projected/770b444b-2f79-4800-8def-88b92c016c6b-kube-api-access-rjltl\") pod \"authorino-58cd955675-cxjb4\" (UID: \"770b444b-2f79-4800-8def-88b92c016c6b\") " pod="kuadrant-system/authorino-58cd955675-cxjb4"
Jul 20 15:34:43.323759 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:43.323715    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-jzk5p" event={"ID":"e7677bbf-90b5-42dd-b9a8-43bf7983e072","Type":"ContainerStarted","Data":"cd2af66d7ef417cc3fb91bf77aca39a98ff787da4ad61fa91004c644c786de8b"}
Jul 20 15:34:43.406816 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:43.406783    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:34:43.423209 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:43.423178    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjltl\" (UniqueName: \"kubernetes.io/projected/770b444b-2f79-4800-8def-88b92c016c6b-kube-api-access-rjltl\") pod \"authorino-58cd955675-cxjb4\" (UID: \"770b444b-2f79-4800-8def-88b92c016c6b\") " pod="kuadrant-system/authorino-58cd955675-cxjb4"
Jul 20 15:34:43.431971 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:43.431938    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjltl\" (UniqueName: \"kubernetes.io/projected/770b444b-2f79-4800-8def-88b92c016c6b-kube-api-access-rjltl\") pod \"authorino-58cd955675-cxjb4\" (UID: \"770b444b-2f79-4800-8def-88b92c016c6b\") " pod="kuadrant-system/authorino-58cd955675-cxjb4"
Jul 20 15:34:43.580856 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:43.580786    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-rg2gf"]
Jul 20 15:34:43.584132 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:43.584116    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-rg2gf"
Jul 20 15:34:43.590523 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:43.590480    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-cxjb4"
Jul 20 15:34:43.591313 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:43.591081    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rg2gf"]
Jul 20 15:34:43.625111 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:43.625073    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6b2sg\" (UniqueName: \"kubernetes.io/projected/56eac2c2-791b-421d-9c76-422388418ac6-kube-api-access-6b2sg\") pod \"authorino-75d556bd77-rg2gf\" (UID: \"56eac2c2-791b-421d-9c76-422388418ac6\") " pod="kuadrant-system/authorino-75d556bd77-rg2gf"
Jul 20 15:34:43.726094 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:43.726057    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6b2sg\" (UniqueName: \"kubernetes.io/projected/56eac2c2-791b-421d-9c76-422388418ac6-kube-api-access-6b2sg\") pod \"authorino-75d556bd77-rg2gf\" (UID: \"56eac2c2-791b-421d-9c76-422388418ac6\") " pod="kuadrant-system/authorino-75d556bd77-rg2gf"
Jul 20 15:34:43.727002 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:43.726959    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-cxjb4"]
Jul 20 15:34:43.729953 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:34:43.729914    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod770b444b_2f79_4800_8def_88b92c016c6b.slice/crio-3ac390cec6353ee0d7551c581f01a6113b1c30d539ec63824bd0222318f1a7eb WatchSource:0}: Error finding container 3ac390cec6353ee0d7551c581f01a6113b1c30d539ec63824bd0222318f1a7eb: Status 404 returned error can't find the container with id 3ac390cec6353ee0d7551c581f01a6113b1c30d539ec63824bd0222318f1a7eb
Jul 20 15:34:43.736056 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:43.736007    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6b2sg\" (UniqueName: \"kubernetes.io/projected/56eac2c2-791b-421d-9c76-422388418ac6-kube-api-access-6b2sg\") pod \"authorino-75d556bd77-rg2gf\" (UID: \"56eac2c2-791b-421d-9c76-422388418ac6\") " pod="kuadrant-system/authorino-75d556bd77-rg2gf"
Jul 20 15:34:43.893990 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:43.893900    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-rg2gf"
Jul 20 15:34:44.053048 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:44.053022    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rg2gf"]
Jul 20 15:34:44.055822 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:34:44.055768    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod56eac2c2_791b_421d_9c76_422388418ac6.slice/crio-9bbfd84bb2648aca1df0fbcbd4d9ff24b689a98e6afe9c1040f0b260c03a52fc WatchSource:0}: Error finding container 9bbfd84bb2648aca1df0fbcbd4d9ff24b689a98e6afe9c1040f0b260c03a52fc: Status 404 returned error can't find the container with id 9bbfd84bb2648aca1df0fbcbd4d9ff24b689a98e6afe9c1040f0b260c03a52fc
Jul 20 15:34:44.330070 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:44.330033    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-rg2gf" event={"ID":"56eac2c2-791b-421d-9c76-422388418ac6","Type":"ContainerStarted","Data":"9bbfd84bb2648aca1df0fbcbd4d9ff24b689a98e6afe9c1040f0b260c03a52fc"}
Jul 20 15:34:44.331854 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:44.331817    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-cxjb4" event={"ID":"770b444b-2f79-4800-8def-88b92c016c6b","Type":"ContainerStarted","Data":"3ac390cec6353ee0d7551c581f01a6113b1c30d539ec63824bd0222318f1a7eb"}
Jul 20 15:34:48.352228 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:48.352185    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-jzk5p" event={"ID":"e7677bbf-90b5-42dd-b9a8-43bf7983e072","Type":"ContainerStarted","Data":"fd44aef1e1da8c2281d94c06e3abba9002c5ea89ac3654f2a4396c525c5e8f04"}
Jul 20 15:34:48.352713 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:48.352327    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-jzk5p"
Jul 20 15:34:48.353599 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:48.353575    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-rg2gf" event={"ID":"56eac2c2-791b-421d-9c76-422388418ac6","Type":"ContainerStarted","Data":"bad8754e5608dad49c8047192a686f05e4e7ea55a124c346ad05c8a7023dd4d8"}
Jul 20 15:34:48.354931 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:48.354910    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-cxjb4" event={"ID":"770b444b-2f79-4800-8def-88b92c016c6b","Type":"ContainerStarted","Data":"477f231f76e34be2af3f053799d9bdbe1537006cc8871330083c59169c172373"}
Jul 20 15:34:48.370295 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:48.370242    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-jzk5p" podStartSLOduration=1.692517138 podStartE2EDuration="6.370231002s" totalImagesPullingTime="4.677713864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:34:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:34:48.368043514 +0000 UTC m=+471.377665181" watchObservedRunningTime="2026-07-20 15:34:48.370231002 +0000 UTC m=+471.379852664"
Jul 20 15:34:48.383838 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:48.383796    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-cxjb4" podStartSLOduration=1.4482273509999999 podStartE2EDuration="5.383782868s" totalImagesPullingTime="3.935555517s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:34:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:34:48.382334983 +0000 UTC m=+471.391956645" watchObservedRunningTime="2026-07-20 15:34:48.383782868 +0000 UTC m=+471.393404557"
Jul 20 15:34:48.398398 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:48.398359    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-rg2gf" podStartSLOduration=1.784476905 podStartE2EDuration="5.398346715s" totalImagesPullingTime="3.61386981s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:34:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:34:48.396319966 +0000 UTC m=+471.405941629" watchObservedRunningTime="2026-07-20 15:34:48.398346715 +0000 UTC m=+471.407968377"
Jul 20 15:34:48.422881 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:48.422843    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-cxjb4"]
Jul 20 15:34:50.364122 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:50.364081    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-cxjb4" podUID="770b444b-2f79-4800-8def-88b92c016c6b" containerName="authorino" containerID="cri-o://477f231f76e34be2af3f053799d9bdbe1537006cc8871330083c59169c172373" gracePeriod=30
Jul 20 15:34:50.602673 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:50.602653    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-cxjb4"
Jul 20 15:34:50.685646 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:50.685578    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjltl\" (UniqueName: \"kubernetes.io/projected/770b444b-2f79-4800-8def-88b92c016c6b-kube-api-access-rjltl\") pod \"770b444b-2f79-4800-8def-88b92c016c6b\" (UID: \"770b444b-2f79-4800-8def-88b92c016c6b\") "
Jul 20 15:34:50.687654 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:50.687619    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/770b444b-2f79-4800-8def-88b92c016c6b-kube-api-access-rjltl" (OuterVolumeSpecName: "kube-api-access-rjltl") pod "770b444b-2f79-4800-8def-88b92c016c6b" (UID: "770b444b-2f79-4800-8def-88b92c016c6b"). InnerVolumeSpecName "kube-api-access-rjltl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 15:34:50.787067 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:50.787036    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjltl\" (UniqueName: \"kubernetes.io/projected/770b444b-2f79-4800-8def-88b92c016c6b-kube-api-access-rjltl\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:34:51.368855 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:51.368824    2559 generic.go:356] "Generic (PLEG): container finished" podID="770b444b-2f79-4800-8def-88b92c016c6b" containerID="477f231f76e34be2af3f053799d9bdbe1537006cc8871330083c59169c172373" exitCode=0
Jul 20 15:34:51.369249 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:51.368865    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-cxjb4"
Jul 20 15:34:51.369249 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:51.368893    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-cxjb4" event={"ID":"770b444b-2f79-4800-8def-88b92c016c6b","Type":"ContainerDied","Data":"477f231f76e34be2af3f053799d9bdbe1537006cc8871330083c59169c172373"}
Jul 20 15:34:51.369249 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:51.368923    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-cxjb4" event={"ID":"770b444b-2f79-4800-8def-88b92c016c6b","Type":"ContainerDied","Data":"3ac390cec6353ee0d7551c581f01a6113b1c30d539ec63824bd0222318f1a7eb"}
Jul 20 15:34:51.369249 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:51.368942    2559 scope.go:117] "RemoveContainer" containerID="477f231f76e34be2af3f053799d9bdbe1537006cc8871330083c59169c172373"
Jul 20 15:34:51.377128 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:51.377112    2559 scope.go:117] "RemoveContainer" containerID="477f231f76e34be2af3f053799d9bdbe1537006cc8871330083c59169c172373"
Jul 20 15:34:51.377449 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:34:51.377428    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"477f231f76e34be2af3f053799d9bdbe1537006cc8871330083c59169c172373\": container with ID starting with 477f231f76e34be2af3f053799d9bdbe1537006cc8871330083c59169c172373 not found: ID does not exist" containerID="477f231f76e34be2af3f053799d9bdbe1537006cc8871330083c59169c172373"
Jul 20 15:34:51.377517 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:51.377461    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"477f231f76e34be2af3f053799d9bdbe1537006cc8871330083c59169c172373"} err="failed to get container status \"477f231f76e34be2af3f053799d9bdbe1537006cc8871330083c59169c172373\": rpc error: code = NotFound desc = could not find container \"477f231f76e34be2af3f053799d9bdbe1537006cc8871330083c59169c172373\": container with ID starting with 477f231f76e34be2af3f053799d9bdbe1537006cc8871330083c59169c172373 not found: ID does not exist"
Jul 20 15:34:51.389875 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:51.389845    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-cxjb4"]
Jul 20 15:34:51.391486 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:51.391467    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-cxjb4"]
Jul 20 15:34:51.513804 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:51.513772    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="770b444b-2f79-4800-8def-88b92c016c6b" path="/var/lib/kubelet/pods/770b444b-2f79-4800-8def-88b92c016c6b/volumes"
Jul 20 15:34:58.105357 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:58.105323    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jzk5p"]
Jul 20 15:34:58.105772 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:58.105500    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-jzk5p" podUID="e7677bbf-90b5-42dd-b9a8-43bf7983e072" containerName="limitador" containerID="cri-o://fd44aef1e1da8c2281d94c06e3abba9002c5ea89ac3654f2a4396c525c5e8f04" gracePeriod=30
Jul 20 15:34:58.106153 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:58.106134    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-jzk5p"
Jul 20 15:34:58.434166 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:34:58.434132    2559 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7677bbf_90b5_42dd_b9a8_43bf7983e072.slice/crio-fd44aef1e1da8c2281d94c06e3abba9002c5ea89ac3654f2a4396c525c5e8f04.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7677bbf_90b5_42dd_b9a8_43bf7983e072.slice/crio-conmon-fd44aef1e1da8c2281d94c06e3abba9002c5ea89ac3654f2a4396c525c5e8f04.scope\": RecentStats: unable to find data in memory cache]"
Jul 20 15:34:58.651046 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:58.651024    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-jzk5p"
Jul 20 15:34:58.753274 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:58.753204    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6sk4\" (UniqueName: \"kubernetes.io/projected/e7677bbf-90b5-42dd-b9a8-43bf7983e072-kube-api-access-f6sk4\") pod \"e7677bbf-90b5-42dd-b9a8-43bf7983e072\" (UID: \"e7677bbf-90b5-42dd-b9a8-43bf7983e072\") "
Jul 20 15:34:58.753274 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:58.753241    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e7677bbf-90b5-42dd-b9a8-43bf7983e072-config-file\") pod \"e7677bbf-90b5-42dd-b9a8-43bf7983e072\" (UID: \"e7677bbf-90b5-42dd-b9a8-43bf7983e072\") "
Jul 20 15:34:58.753636 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:58.753612    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7677bbf-90b5-42dd-b9a8-43bf7983e072-config-file" (OuterVolumeSpecName: "config-file") pod "e7677bbf-90b5-42dd-b9a8-43bf7983e072" (UID: "e7677bbf-90b5-42dd-b9a8-43bf7983e072"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 15:34:58.755315 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:58.755274    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7677bbf-90b5-42dd-b9a8-43bf7983e072-kube-api-access-f6sk4" (OuterVolumeSpecName: "kube-api-access-f6sk4") pod "e7677bbf-90b5-42dd-b9a8-43bf7983e072" (UID: "e7677bbf-90b5-42dd-b9a8-43bf7983e072"). InnerVolumeSpecName "kube-api-access-f6sk4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 15:34:58.853976 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:58.853939    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6sk4\" (UniqueName: \"kubernetes.io/projected/e7677bbf-90b5-42dd-b9a8-43bf7983e072-kube-api-access-f6sk4\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:34:58.853976 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:58.853973    2559 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e7677bbf-90b5-42dd-b9a8-43bf7983e072-config-file\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:34:59.403226 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:59.403188    2559 generic.go:356] "Generic (PLEG): container finished" podID="e7677bbf-90b5-42dd-b9a8-43bf7983e072" containerID="fd44aef1e1da8c2281d94c06e3abba9002c5ea89ac3654f2a4396c525c5e8f04" exitCode=0
Jul 20 15:34:59.403709 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:59.403233    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-jzk5p" event={"ID":"e7677bbf-90b5-42dd-b9a8-43bf7983e072","Type":"ContainerDied","Data":"fd44aef1e1da8c2281d94c06e3abba9002c5ea89ac3654f2a4396c525c5e8f04"}
Jul 20 15:34:59.403709 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:59.403243    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-jzk5p"
Jul 20 15:34:59.403709 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:59.403350    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-jzk5p" event={"ID":"e7677bbf-90b5-42dd-b9a8-43bf7983e072","Type":"ContainerDied","Data":"cd2af66d7ef417cc3fb91bf77aca39a98ff787da4ad61fa91004c644c786de8b"}
Jul 20 15:34:59.403709 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:59.403387    2559 scope.go:117] "RemoveContainer" containerID="fd44aef1e1da8c2281d94c06e3abba9002c5ea89ac3654f2a4396c525c5e8f04"
Jul 20 15:34:59.411468 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:59.411452    2559 scope.go:117] "RemoveContainer" containerID="fd44aef1e1da8c2281d94c06e3abba9002c5ea89ac3654f2a4396c525c5e8f04"
Jul 20 15:34:59.411684 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:34:59.411665    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd44aef1e1da8c2281d94c06e3abba9002c5ea89ac3654f2a4396c525c5e8f04\": container with ID starting with fd44aef1e1da8c2281d94c06e3abba9002c5ea89ac3654f2a4396c525c5e8f04 not found: ID does not exist" containerID="fd44aef1e1da8c2281d94c06e3abba9002c5ea89ac3654f2a4396c525c5e8f04"
Jul 20 15:34:59.411735 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:59.411690    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd44aef1e1da8c2281d94c06e3abba9002c5ea89ac3654f2a4396c525c5e8f04"} err="failed to get container status \"fd44aef1e1da8c2281d94c06e3abba9002c5ea89ac3654f2a4396c525c5e8f04\": rpc error: code = NotFound desc = could not find container \"fd44aef1e1da8c2281d94c06e3abba9002c5ea89ac3654f2a4396c525c5e8f04\": container with ID starting with fd44aef1e1da8c2281d94c06e3abba9002c5ea89ac3654f2a4396c525c5e8f04 not found: ID does not exist"
Jul 20 15:34:59.424182 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:59.424164    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jzk5p"]
Jul 20 15:34:59.428552 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:59.428533    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jzk5p"]
Jul 20 15:34:59.513567 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:59.513544    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7677bbf-90b5-42dd-b9a8-43bf7983e072" path="/var/lib/kubelet/pods/e7677bbf-90b5-42dd-b9a8-43bf7983e072/volumes"
Jul 20 15:34:59.750749 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:59.750670    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:34:59.850936 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:34:59.850907    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:35:00.240941 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:00.240901    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-9bq72"]
Jul 20 15:35:00.241236 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:00.241224    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7677bbf-90b5-42dd-b9a8-43bf7983e072" containerName="limitador"
Jul 20 15:35:00.241279 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:00.241239    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7677bbf-90b5-42dd-b9a8-43bf7983e072" containerName="limitador"
Jul 20 15:35:00.241279 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:00.241249    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="770b444b-2f79-4800-8def-88b92c016c6b" containerName="authorino"
Jul 20 15:35:00.241279 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:00.241254    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="770b444b-2f79-4800-8def-88b92c016c6b" containerName="authorino"
Jul 20 15:35:00.241393 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:00.241329    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7677bbf-90b5-42dd-b9a8-43bf7983e072" containerName="limitador"
Jul 20 15:35:00.241393 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:00.241338    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="770b444b-2f79-4800-8def-88b92c016c6b" containerName="authorino"
Jul 20 15:35:00.245576 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:00.245554    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9bq72"
Jul 20 15:35:00.249731 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:00.249704    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 20 15:35:00.249849 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:00.249760    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 20 15:35:00.250812 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:00.250799    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-kcwhs\""
Jul 20 15:35:00.250812 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:00.250805    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 15:35:00.256003 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:00.255985    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-9bq72"]
Jul 20 15:35:00.293637 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:00.293609    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:35:00.366651 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:00.366619    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d3c36fb8-86ed-4c81-a11c-ca4ac80a2d23-data\") pod \"postgres-69fb4594cf-9bq72\" (UID: \"d3c36fb8-86ed-4c81-a11c-ca4ac80a2d23\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9bq72"
Jul 20 15:35:00.366783 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:00.366668    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cr8qj\" (UniqueName: \"kubernetes.io/projected/d3c36fb8-86ed-4c81-a11c-ca4ac80a2d23-kube-api-access-cr8qj\") pod \"postgres-69fb4594cf-9bq72\" (UID: \"d3c36fb8-86ed-4c81-a11c-ca4ac80a2d23\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9bq72"
Jul 20 15:35:00.468058 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:00.468028    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d3c36fb8-86ed-4c81-a11c-ca4ac80a2d23-data\") pod \"postgres-69fb4594cf-9bq72\" (UID: \"d3c36fb8-86ed-4c81-a11c-ca4ac80a2d23\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9bq72"
Jul 20 15:35:00.468390 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:00.468065    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cr8qj\" (UniqueName: \"kubernetes.io/projected/d3c36fb8-86ed-4c81-a11c-ca4ac80a2d23-kube-api-access-cr8qj\") pod \"postgres-69fb4594cf-9bq72\" (UID: \"d3c36fb8-86ed-4c81-a11c-ca4ac80a2d23\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9bq72"
Jul 20 15:35:00.468442 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:00.468406    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d3c36fb8-86ed-4c81-a11c-ca4ac80a2d23-data\") pod \"postgres-69fb4594cf-9bq72\" (UID: \"d3c36fb8-86ed-4c81-a11c-ca4ac80a2d23\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9bq72"
Jul 20 15:35:00.481570 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:00.481543    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cr8qj\" (UniqueName: \"kubernetes.io/projected/d3c36fb8-86ed-4c81-a11c-ca4ac80a2d23-kube-api-access-cr8qj\") pod \"postgres-69fb4594cf-9bq72\" (UID: \"d3c36fb8-86ed-4c81-a11c-ca4ac80a2d23\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9bq72"
Jul 20 15:35:00.555773 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:00.555750    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9bq72"
Jul 20 15:35:00.678593 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:00.678563    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-9bq72"]
Jul 20 15:35:00.681266 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:35:00.681240    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3c36fb8_86ed_4c81_a11c_ca4ac80a2d23.slice/crio-be97fff1621e6f0450c71c31aa84c3029991e18ba0cba1ca84d8a7b8c83e6c10 WatchSource:0}: Error finding container be97fff1621e6f0450c71c31aa84c3029991e18ba0cba1ca84d8a7b8c83e6c10: Status 404 returned error can't find the container with id be97fff1621e6f0450c71c31aa84c3029991e18ba0cba1ca84d8a7b8c83e6c10
Jul 20 15:35:01.412936 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:01.412905    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9bq72" event={"ID":"d3c36fb8-86ed-4c81-a11c-ca4ac80a2d23","Type":"ContainerStarted","Data":"be97fff1621e6f0450c71c31aa84c3029991e18ba0cba1ca84d8a7b8c83e6c10"}
Jul 20 15:35:08.448836 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:08.448807    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9bq72" event={"ID":"d3c36fb8-86ed-4c81-a11c-ca4ac80a2d23","Type":"ContainerStarted","Data":"8adf8d256be0f69ca184eefd2050f4fc2b6a085b4e3c49e976fe57bf72769ecd"}
Jul 20 15:35:08.449196 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:08.448875    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9bq72"
Jul 20 15:35:08.464386 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:08.464344    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9bq72" podStartSLOduration=0.778069084 podStartE2EDuration="8.464328702s" totalImagesPullingTime="7.686259618s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:35:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:35:08.46305595 +0000 UTC m=+491.472677633" watchObservedRunningTime="2026-07-20 15:35:08.464328702 +0000 UTC m=+491.473950363"
Jul 20 15:35:14.482539 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:14.482507    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9bq72"
Jul 20 15:35:14.809679 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:14.809648    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:35:15.080676 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:15.080612    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wgswp"]
Jul 20 15:35:15.086619 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:15.086598    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-wgswp"
Jul 20 15:35:15.088717 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:15.088695    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wgswp"]
Jul 20 15:35:15.191807 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:15.191777    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjcfw\" (UniqueName: \"kubernetes.io/projected/d64d6099-afb5-4ecb-bbd5-a5181eba72f8-kube-api-access-vjcfw\") pod \"authorino-5d4b67c446-wgswp\" (UID: \"d64d6099-afb5-4ecb-bbd5-a5181eba72f8\") " pod="kuadrant-system/authorino-5d4b67c446-wgswp"
Jul 20 15:35:15.194173 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:15.194143    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:35:15.293014 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:15.292988    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjcfw\" (UniqueName: \"kubernetes.io/projected/d64d6099-afb5-4ecb-bbd5-a5181eba72f8-kube-api-access-vjcfw\") pod \"authorino-5d4b67c446-wgswp\" (UID: \"d64d6099-afb5-4ecb-bbd5-a5181eba72f8\") " pod="kuadrant-system/authorino-5d4b67c446-wgswp"
Jul 20 15:35:15.300761 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:15.300742    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjcfw\" (UniqueName: \"kubernetes.io/projected/d64d6099-afb5-4ecb-bbd5-a5181eba72f8-kube-api-access-vjcfw\") pod \"authorino-5d4b67c446-wgswp\" (UID: \"d64d6099-afb5-4ecb-bbd5-a5181eba72f8\") " pod="kuadrant-system/authorino-5d4b67c446-wgswp"
Jul 20 15:35:15.362217 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:15.362162    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wgswp"]
Jul 20 15:35:15.362374 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:15.362362    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-wgswp"
Jul 20 15:35:15.387986 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:15.387965    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-57b576456b-j6q5x"]
Jul 20 15:35:15.392774 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:15.392756    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57b576456b-j6q5x"
Jul 20 15:35:15.398368 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:15.398350    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57b576456b-j6q5x"]
Jul 20 15:35:15.473972 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:15.473937    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57b576456b-j6q5x"]
Jul 20 15:35:15.474303 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:35:15.474259    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-j49hm], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-57b576456b-j6q5x" podUID="11f62016-cb1f-4a4b-9207-68c58534f30b"
Jul 20 15:35:15.487521 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:15.487494    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wgswp"]
Jul 20 15:35:15.490294 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:35:15.490265    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd64d6099_afb5_4ecb_bbd5_a5181eba72f8.slice/crio-cb03e8c862ac90b75c41e6a90a193757338aaeeed0016830f53655f85f4c3c7e WatchSource:0}: Error finding container cb03e8c862ac90b75c41e6a90a193757338aaeeed0016830f53655f85f4c3c7e: Status 404 returned error can't find the container with id cb03e8c862ac90b75c41e6a90a193757338aaeeed0016830f53655f85f4c3c7e
Jul 20 15:35:15.495164 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:15.495142    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j49hm\" (UniqueName: \"kubernetes.io/projected/11f62016-cb1f-4a4b-9207-68c58534f30b-kube-api-access-j49hm\") pod \"authorino-57b576456b-j6q5x\" (UID: \"11f62016-cb1f-4a4b-9207-68c58534f30b\") " pod="kuadrant-system/authorino-57b576456b-j6q5x"
Jul 20 15:35:15.507575 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:15.507554    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7679b6dbfd-hr46t"]
Jul 20 15:35:15.511392 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:15.511372    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7679b6dbfd-hr46t"
Jul 20 15:35:15.513730 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:15.513712    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 20 15:35:15.517249 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:15.517224    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7679b6dbfd-hr46t"]
Jul 20 15:35:15.596365 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:15.596345    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/48100c36-b8db-4f86-a728-170ae963e729-tls-cert\") pod \"authorino-7679b6dbfd-hr46t\" (UID: \"48100c36-b8db-4f86-a728-170ae963e729\") " pod="kuadrant-system/authorino-7679b6dbfd-hr46t"
Jul 20 15:35:15.596456 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:15.596380    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mft8q\" (UniqueName: \"kubernetes.io/projected/48100c36-b8db-4f86-a728-170ae963e729-kube-api-access-mft8q\") pod \"authorino-7679b6dbfd-hr46t\" (UID: \"48100c36-b8db-4f86-a728-170ae963e729\") " pod="kuadrant-system/authorino-7679b6dbfd-hr46t"
Jul 20 15:35:15.596456 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:15.596404    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j49hm\" (UniqueName: \"kubernetes.io/projected/11f62016-cb1f-4a4b-9207-68c58534f30b-kube-api-access-j49hm\") pod \"authorino-57b576456b-j6q5x\" (UID: \"11f62016-cb1f-4a4b-9207-68c58534f30b\") " pod="kuadrant-system/authorino-57b576456b-j6q5x"
Jul 20 15:35:15.604640 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:15.604620    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j49hm\" (UniqueName: \"kubernetes.io/projected/11f62016-cb1f-4a4b-9207-68c58534f30b-kube-api-access-j49hm\") pod \"authorino-57b576456b-j6q5x\" (UID: \"11f62016-cb1f-4a4b-9207-68c58534f30b\") " pod="kuadrant-system/authorino-57b576456b-j6q5x"
Jul 20 15:35:15.696873 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:15.696813    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/48100c36-b8db-4f86-a728-170ae963e729-tls-cert\") pod \"authorino-7679b6dbfd-hr46t\" (UID: \"48100c36-b8db-4f86-a728-170ae963e729\") " pod="kuadrant-system/authorino-7679b6dbfd-hr46t"
Jul 20 15:35:15.696873 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:15.696864    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mft8q\" (UniqueName: \"kubernetes.io/projected/48100c36-b8db-4f86-a728-170ae963e729-kube-api-access-mft8q\") pod \"authorino-7679b6dbfd-hr46t\" (UID: \"48100c36-b8db-4f86-a728-170ae963e729\") " pod="kuadrant-system/authorino-7679b6dbfd-hr46t"
Jul 20 15:35:15.698854 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:15.698836    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/48100c36-b8db-4f86-a728-170ae963e729-tls-cert\") pod \"authorino-7679b6dbfd-hr46t\" (UID: \"48100c36-b8db-4f86-a728-170ae963e729\") " pod="kuadrant-system/authorino-7679b6dbfd-hr46t"
Jul 20 15:35:15.704553 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:15.704534    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mft8q\" (UniqueName: \"kubernetes.io/projected/48100c36-b8db-4f86-a728-170ae963e729-kube-api-access-mft8q\") pod \"authorino-7679b6dbfd-hr46t\" (UID: \"48100c36-b8db-4f86-a728-170ae963e729\") " pod="kuadrant-system/authorino-7679b6dbfd-hr46t"
Jul 20 15:35:15.821069 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:15.821045    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7679b6dbfd-hr46t"
Jul 20 15:35:15.942407 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:15.942386    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7679b6dbfd-hr46t"]
Jul 20 15:35:15.945370 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:35:15.945344    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod48100c36_b8db_4f86_a728_170ae963e729.slice/crio-4f99a9744e27ab552910a490b874652be1b6868384995e962f25e4e2cf15463d WatchSource:0}: Error finding container 4f99a9744e27ab552910a490b874652be1b6868384995e962f25e4e2cf15463d: Status 404 returned error can't find the container with id 4f99a9744e27ab552910a490b874652be1b6868384995e962f25e4e2cf15463d
Jul 20 15:35:16.482140 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:16.482108    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7679b6dbfd-hr46t" event={"ID":"48100c36-b8db-4f86-a728-170ae963e729","Type":"ContainerStarted","Data":"214fd2430d81687708399b092c75b5e951d9656c3744a155b2996166c9fe5c87"}
Jul 20 15:35:16.482140 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:16.482146    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7679b6dbfd-hr46t" event={"ID":"48100c36-b8db-4f86-a728-170ae963e729","Type":"ContainerStarted","Data":"4f99a9744e27ab552910a490b874652be1b6868384995e962f25e4e2cf15463d"}
Jul 20 15:35:16.483566 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:16.483545    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57b576456b-j6q5x"
Jul 20 15:35:16.483698 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:16.483603    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-wgswp" podUID="d64d6099-afb5-4ecb-bbd5-a5181eba72f8" containerName="authorino" containerID="cri-o://c8cb90f4d3811653e49e39b26884315ed0804224869c1c2b5f0ba32ed1e252e0" gracePeriod=30
Jul 20 15:35:16.483698 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:16.483544    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-wgswp" event={"ID":"d64d6099-afb5-4ecb-bbd5-a5181eba72f8","Type":"ContainerStarted","Data":"c8cb90f4d3811653e49e39b26884315ed0804224869c1c2b5f0ba32ed1e252e0"}
Jul 20 15:35:16.483698 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:16.483668    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-wgswp" event={"ID":"d64d6099-afb5-4ecb-bbd5-a5181eba72f8","Type":"ContainerStarted","Data":"cb03e8c862ac90b75c41e6a90a193757338aaeeed0016830f53655f85f4c3c7e"}
Jul 20 15:35:16.490693 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:16.490676    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57b576456b-j6q5x"
Jul 20 15:35:16.499436 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:16.499398    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7679b6dbfd-hr46t" podStartSLOduration=1.160011135 podStartE2EDuration="1.49938488s" totalImagesPullingTime="339.373745ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:35:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:35:16.498229816 +0000 UTC m=+499.507851479" watchObservedRunningTime="2026-07-20 15:35:16.49938488 +0000 UTC m=+499.509006519"
Jul 20 15:35:16.514429 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:16.514391    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-wgswp" podStartSLOduration=1.125664196 podStartE2EDuration="1.514379642s" totalImagesPullingTime="388.715446ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:35:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:35:16.512866472 +0000 UTC m=+499.522488137" watchObservedRunningTime="2026-07-20 15:35:16.514379642 +0000 UTC m=+499.524001304"
Jul 20 15:35:16.529496 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:16.529475    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rg2gf"]
Jul 20 15:35:16.529652 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:16.529634    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-rg2gf" podUID="56eac2c2-791b-421d-9c76-422388418ac6" containerName="authorino" containerID="cri-o://bad8754e5608dad49c8047192a686f05e4e7ea55a124c346ad05c8a7023dd4d8" gracePeriod=30
Jul 20 15:35:16.603484 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:16.603457    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j49hm\" (UniqueName: \"kubernetes.io/projected/11f62016-cb1f-4a4b-9207-68c58534f30b-kube-api-access-j49hm\") pod \"11f62016-cb1f-4a4b-9207-68c58534f30b\" (UID: \"11f62016-cb1f-4a4b-9207-68c58534f30b\") "
Jul 20 15:35:16.605487 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:16.605465    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11f62016-cb1f-4a4b-9207-68c58534f30b-kube-api-access-j49hm" (OuterVolumeSpecName: "kube-api-access-j49hm") pod "11f62016-cb1f-4a4b-9207-68c58534f30b" (UID: "11f62016-cb1f-4a4b-9207-68c58534f30b"). InnerVolumeSpecName "kube-api-access-j49hm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 15:35:16.704926 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:16.704901    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j49hm\" (UniqueName: \"kubernetes.io/projected/11f62016-cb1f-4a4b-9207-68c58534f30b-kube-api-access-j49hm\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:35:16.764709 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:16.764686    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-wgswp"
Jul 20 15:35:16.771996 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:16.771975    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-rg2gf"
Jul 20 15:35:16.906959 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:16.906885    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6b2sg\" (UniqueName: \"kubernetes.io/projected/56eac2c2-791b-421d-9c76-422388418ac6-kube-api-access-6b2sg\") pod \"56eac2c2-791b-421d-9c76-422388418ac6\" (UID: \"56eac2c2-791b-421d-9c76-422388418ac6\") "
Jul 20 15:35:16.907085 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:16.906964    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vjcfw\" (UniqueName: \"kubernetes.io/projected/d64d6099-afb5-4ecb-bbd5-a5181eba72f8-kube-api-access-vjcfw\") pod \"d64d6099-afb5-4ecb-bbd5-a5181eba72f8\" (UID: \"d64d6099-afb5-4ecb-bbd5-a5181eba72f8\") "
Jul 20 15:35:16.908989 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:16.908962    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56eac2c2-791b-421d-9c76-422388418ac6-kube-api-access-6b2sg" (OuterVolumeSpecName: "kube-api-access-6b2sg") pod "56eac2c2-791b-421d-9c76-422388418ac6" (UID: "56eac2c2-791b-421d-9c76-422388418ac6"). InnerVolumeSpecName "kube-api-access-6b2sg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 15:35:16.908989 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:16.908971    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d64d6099-afb5-4ecb-bbd5-a5181eba72f8-kube-api-access-vjcfw" (OuterVolumeSpecName: "kube-api-access-vjcfw") pod "d64d6099-afb5-4ecb-bbd5-a5181eba72f8" (UID: "d64d6099-afb5-4ecb-bbd5-a5181eba72f8"). InnerVolumeSpecName "kube-api-access-vjcfw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 15:35:17.008545 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:17.008513    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6b2sg\" (UniqueName: \"kubernetes.io/projected/56eac2c2-791b-421d-9c76-422388418ac6-kube-api-access-6b2sg\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:35:17.008545 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:17.008539    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjcfw\" (UniqueName: \"kubernetes.io/projected/d64d6099-afb5-4ecb-bbd5-a5181eba72f8-kube-api-access-vjcfw\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:35:17.490331 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:17.490282    2559 generic.go:356] "Generic (PLEG): container finished" podID="d64d6099-afb5-4ecb-bbd5-a5181eba72f8" containerID="c8cb90f4d3811653e49e39b26884315ed0804224869c1c2b5f0ba32ed1e252e0" exitCode=0
Jul 20 15:35:17.490504 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:17.490347    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-wgswp"
Jul 20 15:35:17.490504 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:17.490391    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-wgswp" event={"ID":"d64d6099-afb5-4ecb-bbd5-a5181eba72f8","Type":"ContainerDied","Data":"c8cb90f4d3811653e49e39b26884315ed0804224869c1c2b5f0ba32ed1e252e0"}
Jul 20 15:35:17.490504 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:17.490441    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-wgswp" event={"ID":"d64d6099-afb5-4ecb-bbd5-a5181eba72f8","Type":"ContainerDied","Data":"cb03e8c862ac90b75c41e6a90a193757338aaeeed0016830f53655f85f4c3c7e"}
Jul 20 15:35:17.490504 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:17.490466    2559 scope.go:117] "RemoveContainer" containerID="c8cb90f4d3811653e49e39b26884315ed0804224869c1c2b5f0ba32ed1e252e0"
Jul 20 15:35:17.491741 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:17.491639    2559 generic.go:356] "Generic (PLEG): container finished" podID="56eac2c2-791b-421d-9c76-422388418ac6" containerID="bad8754e5608dad49c8047192a686f05e4e7ea55a124c346ad05c8a7023dd4d8" exitCode=0
Jul 20 15:35:17.491741 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:17.491698    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-rg2gf"
Jul 20 15:35:17.491741 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:17.491714    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-rg2gf" event={"ID":"56eac2c2-791b-421d-9c76-422388418ac6","Type":"ContainerDied","Data":"bad8754e5608dad49c8047192a686f05e4e7ea55a124c346ad05c8a7023dd4d8"}
Jul 20 15:35:17.492108 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:17.491748    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-rg2gf" event={"ID":"56eac2c2-791b-421d-9c76-422388418ac6","Type":"ContainerDied","Data":"9bbfd84bb2648aca1df0fbcbd4d9ff24b689a98e6afe9c1040f0b260c03a52fc"}
Jul 20 15:35:17.492108 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:17.491892    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57b576456b-j6q5x"
Jul 20 15:35:17.499147 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:17.499128    2559 scope.go:117] "RemoveContainer" containerID="c8cb90f4d3811653e49e39b26884315ed0804224869c1c2b5f0ba32ed1e252e0"
Jul 20 15:35:17.499426 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:35:17.499405    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8cb90f4d3811653e49e39b26884315ed0804224869c1c2b5f0ba32ed1e252e0\": container with ID starting with c8cb90f4d3811653e49e39b26884315ed0804224869c1c2b5f0ba32ed1e252e0 not found: ID does not exist" containerID="c8cb90f4d3811653e49e39b26884315ed0804224869c1c2b5f0ba32ed1e252e0"
Jul 20 15:35:17.499476 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:17.499437    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8cb90f4d3811653e49e39b26884315ed0804224869c1c2b5f0ba32ed1e252e0"} err="failed to get container status \"c8cb90f4d3811653e49e39b26884315ed0804224869c1c2b5f0ba32ed1e252e0\": rpc error: code = NotFound desc = could not find container \"c8cb90f4d3811653e49e39b26884315ed0804224869c1c2b5f0ba32ed1e252e0\": container with ID starting with c8cb90f4d3811653e49e39b26884315ed0804224869c1c2b5f0ba32ed1e252e0 not found: ID does not exist"
Jul 20 15:35:17.499476 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:17.499456    2559 scope.go:117] "RemoveContainer" containerID="bad8754e5608dad49c8047192a686f05e4e7ea55a124c346ad05c8a7023dd4d8"
Jul 20 15:35:17.507198 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:17.507181    2559 scope.go:117] "RemoveContainer" containerID="bad8754e5608dad49c8047192a686f05e4e7ea55a124c346ad05c8a7023dd4d8"
Jul 20 15:35:17.507436 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:35:17.507419    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bad8754e5608dad49c8047192a686f05e4e7ea55a124c346ad05c8a7023dd4d8\": container with ID starting with bad8754e5608dad49c8047192a686f05e4e7ea55a124c346ad05c8a7023dd4d8 not found: ID does not exist" containerID="bad8754e5608dad49c8047192a686f05e4e7ea55a124c346ad05c8a7023dd4d8"
Jul 20 15:35:17.507487 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:17.507442    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bad8754e5608dad49c8047192a686f05e4e7ea55a124c346ad05c8a7023dd4d8"} err="failed to get container status \"bad8754e5608dad49c8047192a686f05e4e7ea55a124c346ad05c8a7023dd4d8\": rpc error: code = NotFound desc = could not find container \"bad8754e5608dad49c8047192a686f05e4e7ea55a124c346ad05c8a7023dd4d8\": container with ID starting with bad8754e5608dad49c8047192a686f05e4e7ea55a124c346ad05c8a7023dd4d8 not found: ID does not exist"
Jul 20 15:35:17.516521 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:17.516502    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wgswp"]
Jul 20 15:35:17.523960 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:17.523942    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wgswp"]
Jul 20 15:35:17.535078 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:17.535059    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rg2gf"]
Jul 20 15:35:17.538832 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:17.538807    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rg2gf"]
Jul 20 15:35:17.561882 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:17.561823    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57b576456b-j6q5x"]
Jul 20 15:35:17.565688 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:17.565668    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-57b576456b-j6q5x"]
Jul 20 15:35:19.513944 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:19.513912    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11f62016-cb1f-4a4b-9207-68c58534f30b" path="/var/lib/kubelet/pods/11f62016-cb1f-4a4b-9207-68c58534f30b/volumes"
Jul 20 15:35:19.514319 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:19.514118    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56eac2c2-791b-421d-9c76-422388418ac6" path="/var/lib/kubelet/pods/56eac2c2-791b-421d-9c76-422388418ac6/volumes"
Jul 20 15:35:19.514408 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:19.514396    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d64d6099-afb5-4ecb-bbd5-a5181eba72f8" path="/var/lib/kubelet/pods/d64d6099-afb5-4ecb-bbd5-a5181eba72f8/volumes"
Jul 20 15:35:21.790671 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:21.790638    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:35:21.796770 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:21.796742    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7c44cd9987-pqz9v"]
Jul 20 15:35:21.797111 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:21.797083    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d64d6099-afb5-4ecb-bbd5-a5181eba72f8" containerName="authorino"
Jul 20 15:35:21.797111 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:21.797110    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d64d6099-afb5-4ecb-bbd5-a5181eba72f8" containerName="authorino"
Jul 20 15:35:21.797313 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:21.797125    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56eac2c2-791b-421d-9c76-422388418ac6" containerName="authorino"
Jul 20 15:35:21.797313 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:21.797131    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="56eac2c2-791b-421d-9c76-422388418ac6" containerName="authorino"
Jul 20 15:35:21.797313 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:21.797211    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="56eac2c2-791b-421d-9c76-422388418ac6" containerName="authorino"
Jul 20 15:35:21.797313 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:21.797225    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d64d6099-afb5-4ecb-bbd5-a5181eba72f8" containerName="authorino"
Jul 20 15:35:21.891001 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:21.890965    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:35:21.894267 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:21.894249    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7c44cd9987-pqz9v"]
Jul 20 15:35:21.894364 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:21.894344    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7c44cd9987-pqz9v"
Jul 20 15:35:21.903126 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:21.903096    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 20 15:35:21.903237 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:21.903112    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-nvkzc\""
Jul 20 15:35:21.903237 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:21.903097    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 20 15:35:21.991703 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:21.991676    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:35:22.005584 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:22.005560    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-854b67b788-nh8m2"]
Jul 20 15:35:22.038201 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:22.038177    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-854b67b788-nh8m2"]
Jul 20 15:35:22.038281 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:22.038259    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-854b67b788-nh8m2"
Jul 20 15:35:22.049900 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:22.049880    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cf37448e-4647-4702-9682-f6daf2c563ee-webhook-cert\") pod \"maas-controller-7c44cd9987-pqz9v\" (UID: \"cf37448e-4647-4702-9682-f6daf2c563ee\") " pod="opendatahub/maas-controller-7c44cd9987-pqz9v"
Jul 20 15:35:22.049987 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:22.049917    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mlvx\" (UniqueName: \"kubernetes.io/projected/cf37448e-4647-4702-9682-f6daf2c563ee-kube-api-access-4mlvx\") pod \"maas-controller-7c44cd9987-pqz9v\" (UID: \"cf37448e-4647-4702-9682-f6daf2c563ee\") " pod="opendatahub/maas-controller-7c44cd9987-pqz9v"
Jul 20 15:35:22.151149 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:22.151120    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mlvx\" (UniqueName: \"kubernetes.io/projected/cf37448e-4647-4702-9682-f6daf2c563ee-kube-api-access-4mlvx\") pod \"maas-controller-7c44cd9987-pqz9v\" (UID: \"cf37448e-4647-4702-9682-f6daf2c563ee\") " pod="opendatahub/maas-controller-7c44cd9987-pqz9v"
Jul 20 15:35:22.151226 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:22.151158    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpt8g\" (UniqueName: \"kubernetes.io/projected/f63a30bf-ce64-4ea5-ae1a-4331dc7f6cf3-kube-api-access-xpt8g\") pod \"maas-controller-854b67b788-nh8m2\" (UID: \"f63a30bf-ce64-4ea5-ae1a-4331dc7f6cf3\") " pod="opendatahub/maas-controller-854b67b788-nh8m2"
Jul 20 15:35:22.151274 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:22.151234    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f63a30bf-ce64-4ea5-ae1a-4331dc7f6cf3-webhook-cert\") pod \"maas-controller-854b67b788-nh8m2\" (UID: \"f63a30bf-ce64-4ea5-ae1a-4331dc7f6cf3\") " pod="opendatahub/maas-controller-854b67b788-nh8m2"
Jul 20 15:35:22.151342 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:22.151277    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cf37448e-4647-4702-9682-f6daf2c563ee-webhook-cert\") pod \"maas-controller-7c44cd9987-pqz9v\" (UID: \"cf37448e-4647-4702-9682-f6daf2c563ee\") " pod="opendatahub/maas-controller-7c44cd9987-pqz9v"
Jul 20 15:35:22.153576 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:22.153556    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cf37448e-4647-4702-9682-f6daf2c563ee-webhook-cert\") pod \"maas-controller-7c44cd9987-pqz9v\" (UID: \"cf37448e-4647-4702-9682-f6daf2c563ee\") " pod="opendatahub/maas-controller-7c44cd9987-pqz9v"
Jul 20 15:35:22.159777 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:22.159759    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mlvx\" (UniqueName: \"kubernetes.io/projected/cf37448e-4647-4702-9682-f6daf2c563ee-kube-api-access-4mlvx\") pod \"maas-controller-7c44cd9987-pqz9v\" (UID: \"cf37448e-4647-4702-9682-f6daf2c563ee\") " pod="opendatahub/maas-controller-7c44cd9987-pqz9v"
Jul 20 15:35:22.204571 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:22.204542    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7c44cd9987-pqz9v"
Jul 20 15:35:22.252386 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:22.252350    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpt8g\" (UniqueName: \"kubernetes.io/projected/f63a30bf-ce64-4ea5-ae1a-4331dc7f6cf3-kube-api-access-xpt8g\") pod \"maas-controller-854b67b788-nh8m2\" (UID: \"f63a30bf-ce64-4ea5-ae1a-4331dc7f6cf3\") " pod="opendatahub/maas-controller-854b67b788-nh8m2"
Jul 20 15:35:22.252484 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:22.252437    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f63a30bf-ce64-4ea5-ae1a-4331dc7f6cf3-webhook-cert\") pod \"maas-controller-854b67b788-nh8m2\" (UID: \"f63a30bf-ce64-4ea5-ae1a-4331dc7f6cf3\") " pod="opendatahub/maas-controller-854b67b788-nh8m2"
Jul 20 15:35:22.254770 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:22.254742    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f63a30bf-ce64-4ea5-ae1a-4331dc7f6cf3-webhook-cert\") pod \"maas-controller-854b67b788-nh8m2\" (UID: \"f63a30bf-ce64-4ea5-ae1a-4331dc7f6cf3\") " pod="opendatahub/maas-controller-854b67b788-nh8m2"
Jul 20 15:35:22.260599 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:22.260581    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpt8g\" (UniqueName: \"kubernetes.io/projected/f63a30bf-ce64-4ea5-ae1a-4331dc7f6cf3-kube-api-access-xpt8g\") pod \"maas-controller-854b67b788-nh8m2\" (UID: \"f63a30bf-ce64-4ea5-ae1a-4331dc7f6cf3\") " pod="opendatahub/maas-controller-854b67b788-nh8m2"
Jul 20 15:35:22.343034 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:22.343009    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7c44cd9987-pqz9v"]
Jul 20 15:35:22.344878 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:35:22.344848    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf37448e_4647_4702_9682_f6daf2c563ee.slice/crio-e46d358fa03e1ca2980308d24675ad73fc98ec0dc566c2e46c52dd3c3b0825ff WatchSource:0}: Error finding container e46d358fa03e1ca2980308d24675ad73fc98ec0dc566c2e46c52dd3c3b0825ff: Status 404 returned error can't find the container with id e46d358fa03e1ca2980308d24675ad73fc98ec0dc566c2e46c52dd3c3b0825ff
Jul 20 15:35:22.347640 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:22.347619    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-854b67b788-nh8m2"
Jul 20 15:35:22.507457 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:22.507435    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-854b67b788-nh8m2"]
Jul 20 15:35:22.509186 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:35:22.509159    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf63a30bf_ce64_4ea5_ae1a_4331dc7f6cf3.slice/crio-6a3c596ab631e4a50e8671898dae52e2e471bb78a540f61ce151825b17637b29 WatchSource:0}: Error finding container 6a3c596ab631e4a50e8671898dae52e2e471bb78a540f61ce151825b17637b29: Status 404 returned error can't find the container with id 6a3c596ab631e4a50e8671898dae52e2e471bb78a540f61ce151825b17637b29
Jul 20 15:35:22.521964 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:22.521939    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7c44cd9987-pqz9v" event={"ID":"cf37448e-4647-4702-9682-f6daf2c563ee","Type":"ContainerStarted","Data":"e46d358fa03e1ca2980308d24675ad73fc98ec0dc566c2e46c52dd3c3b0825ff"}
Jul 20 15:35:22.522985 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:22.522964    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-854b67b788-nh8m2" event={"ID":"f63a30bf-ce64-4ea5-ae1a-4331dc7f6cf3","Type":"ContainerStarted","Data":"6a3c596ab631e4a50e8671898dae52e2e471bb78a540f61ce151825b17637b29"}
Jul 20 15:35:25.538132 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:25.538096    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:35:25.539837 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:25.539812    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7c44cd9987-pqz9v" event={"ID":"cf37448e-4647-4702-9682-f6daf2c563ee","Type":"ContainerStarted","Data":"477bc3aa72b25d75415af0b717e59c00952a21e05c4c4bacae568f61ad160e82"}
Jul 20 15:35:25.539971 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:25.539871    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7c44cd9987-pqz9v"
Jul 20 15:35:25.541235 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:25.541218    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-854b67b788-nh8m2" event={"ID":"f63a30bf-ce64-4ea5-ae1a-4331dc7f6cf3","Type":"ContainerStarted","Data":"9ea5b1d8b25d39d8bfb29090067dcdff7683e35c86ab8950775685151a7a0ee8"}
Jul 20 15:35:25.541364 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:25.541344    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-854b67b788-nh8m2"
Jul 20 15:35:25.568836 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:25.568791    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7c44cd9987-pqz9v" podStartSLOduration=1.559861996 podStartE2EDuration="4.568778916s" totalImagesPullingTime="3.00891692s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:35:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:35:25.566576511 +0000 UTC m=+508.576198174" watchObservedRunningTime="2026-07-20 15:35:25.568778916 +0000 UTC m=+508.578400613"
Jul 20 15:35:25.612039 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:25.611966    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-854b67b788-nh8m2" podStartSLOduration=1.771231447 podStartE2EDuration="4.611953515s" totalImagesPullingTime="2.840722068s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:35:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:35:25.610047969 +0000 UTC m=+508.619669656" watchObservedRunningTime="2026-07-20 15:35:25.611953515 +0000 UTC m=+508.621575177"
Jul 20 15:35:25.734909 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:25.734870    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:35:36.547364 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:36.547329    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7c44cd9987-pqz9v"
Jul 20 15:35:36.547758 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:36.547500    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-854b67b788-nh8m2"
Jul 20 15:35:36.615996 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:36.615965    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7c44cd9987-pqz9v"]
Jul 20 15:35:36.616176 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:36.616150    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7c44cd9987-pqz9v" podUID="cf37448e-4647-4702-9682-f6daf2c563ee" containerName="manager" containerID="cri-o://477bc3aa72b25d75415af0b717e59c00952a21e05c4c4bacae568f61ad160e82" gracePeriod=30
Jul 20 15:35:36.856121 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:36.856096    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7c44cd9987-pqz9v"
Jul 20 15:35:36.963917 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:36.963889    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4mlvx\" (UniqueName: \"kubernetes.io/projected/cf37448e-4647-4702-9682-f6daf2c563ee-kube-api-access-4mlvx\") pod \"cf37448e-4647-4702-9682-f6daf2c563ee\" (UID: \"cf37448e-4647-4702-9682-f6daf2c563ee\") "
Jul 20 15:35:36.964063 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:36.963944    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cf37448e-4647-4702-9682-f6daf2c563ee-webhook-cert\") pod \"cf37448e-4647-4702-9682-f6daf2c563ee\" (UID: \"cf37448e-4647-4702-9682-f6daf2c563ee\") "
Jul 20 15:35:36.965959 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:36.965937    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf37448e-4647-4702-9682-f6daf2c563ee-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "cf37448e-4647-4702-9682-f6daf2c563ee" (UID: "cf37448e-4647-4702-9682-f6daf2c563ee"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 15:35:36.966083 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:36.966062    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf37448e-4647-4702-9682-f6daf2c563ee-kube-api-access-4mlvx" (OuterVolumeSpecName: "kube-api-access-4mlvx") pod "cf37448e-4647-4702-9682-f6daf2c563ee" (UID: "cf37448e-4647-4702-9682-f6daf2c563ee"). InnerVolumeSpecName "kube-api-access-4mlvx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 15:35:37.065052 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:37.065025    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4mlvx\" (UniqueName: \"kubernetes.io/projected/cf37448e-4647-4702-9682-f6daf2c563ee-kube-api-access-4mlvx\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:35:37.065052 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:37.065050    2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cf37448e-4647-4702-9682-f6daf2c563ee-webhook-cert\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:35:37.592449 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:37.592403    2559 generic.go:356] "Generic (PLEG): container finished" podID="cf37448e-4647-4702-9682-f6daf2c563ee" containerID="477bc3aa72b25d75415af0b717e59c00952a21e05c4c4bacae568f61ad160e82" exitCode=0
Jul 20 15:35:37.592870 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:37.592462    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7c44cd9987-pqz9v"
Jul 20 15:35:37.592870 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:37.592487    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7c44cd9987-pqz9v" event={"ID":"cf37448e-4647-4702-9682-f6daf2c563ee","Type":"ContainerDied","Data":"477bc3aa72b25d75415af0b717e59c00952a21e05c4c4bacae568f61ad160e82"}
Jul 20 15:35:37.592870 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:37.592548    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7c44cd9987-pqz9v" event={"ID":"cf37448e-4647-4702-9682-f6daf2c563ee","Type":"ContainerDied","Data":"e46d358fa03e1ca2980308d24675ad73fc98ec0dc566c2e46c52dd3c3b0825ff"}
Jul 20 15:35:37.592870 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:37.592583    2559 scope.go:117] "RemoveContainer" containerID="477bc3aa72b25d75415af0b717e59c00952a21e05c4c4bacae568f61ad160e82"
Jul 20 15:35:37.600627 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:37.600602    2559 scope.go:117] "RemoveContainer" containerID="477bc3aa72b25d75415af0b717e59c00952a21e05c4c4bacae568f61ad160e82"
Jul 20 15:35:37.600880 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:35:37.600863    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"477bc3aa72b25d75415af0b717e59c00952a21e05c4c4bacae568f61ad160e82\": container with ID starting with 477bc3aa72b25d75415af0b717e59c00952a21e05c4c4bacae568f61ad160e82 not found: ID does not exist" containerID="477bc3aa72b25d75415af0b717e59c00952a21e05c4c4bacae568f61ad160e82"
Jul 20 15:35:37.600947 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:37.600889    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"477bc3aa72b25d75415af0b717e59c00952a21e05c4c4bacae568f61ad160e82"} err="failed to get container status \"477bc3aa72b25d75415af0b717e59c00952a21e05c4c4bacae568f61ad160e82\": rpc error: code = NotFound desc = could not find container \"477bc3aa72b25d75415af0b717e59c00952a21e05c4c4bacae568f61ad160e82\": container with ID starting with 477bc3aa72b25d75415af0b717e59c00952a21e05c4c4bacae568f61ad160e82 not found: ID does not exist"
Jul 20 15:35:37.611715 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:37.611689    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7c44cd9987-pqz9v"]
Jul 20 15:35:37.615696 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:37.615667    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7c44cd9987-pqz9v"]
Jul 20 15:35:39.513777 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:39.513737    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf37448e-4647-4702-9682-f6daf2c563ee" path="/var/lib/kubelet/pods/cf37448e-4647-4702-9682-f6daf2c563ee/volumes"
Jul 20 15:35:57.311097 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:57.311008    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:35:57.824395 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:57.824363    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-597f477cb8-sdsxn"]
Jul 20 15:35:57.824689 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:57.824677    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf37448e-4647-4702-9682-f6daf2c563ee" containerName="manager"
Jul 20 15:35:57.824737 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:57.824691    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf37448e-4647-4702-9682-f6daf2c563ee" containerName="manager"
Jul 20 15:35:57.824776 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:57.824744    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf37448e-4647-4702-9682-f6daf2c563ee" containerName="manager"
Jul 20 15:35:57.829977 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:57.829957    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-597f477cb8-sdsxn"
Jul 20 15:35:57.832542 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:57.832522    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 20 15:35:57.842149 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:57.842127    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-597f477cb8-sdsxn"]
Jul 20 15:35:57.929663 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:57.929635    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7baf8881-13bf-441c-8417-4648a2efe781-openshift-service-ca\") pod \"maas-api-597f477cb8-sdsxn\" (UID: \"7baf8881-13bf-441c-8417-4648a2efe781\") " pod="odh-ai-gateway-infra/maas-api-597f477cb8-sdsxn"
Jul 20 15:35:57.929818 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:57.929679    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g96r9\" (UniqueName: \"kubernetes.io/projected/7baf8881-13bf-441c-8417-4648a2efe781-kube-api-access-g96r9\") pod \"maas-api-597f477cb8-sdsxn\" (UID: \"7baf8881-13bf-441c-8417-4648a2efe781\") " pod="odh-ai-gateway-infra/maas-api-597f477cb8-sdsxn"
Jul 20 15:35:57.929818 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:57.929700    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7baf8881-13bf-441c-8417-4648a2efe781-maas-api-tls\") pod \"maas-api-597f477cb8-sdsxn\" (UID: \"7baf8881-13bf-441c-8417-4648a2efe781\") " pod="odh-ai-gateway-infra/maas-api-597f477cb8-sdsxn"
Jul 20 15:35:58.030447 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:58.030419    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7baf8881-13bf-441c-8417-4648a2efe781-openshift-service-ca\") pod \"maas-api-597f477cb8-sdsxn\" (UID: \"7baf8881-13bf-441c-8417-4648a2efe781\") " pod="odh-ai-gateway-infra/maas-api-597f477cb8-sdsxn"
Jul 20 15:35:58.030586 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:58.030470    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g96r9\" (UniqueName: \"kubernetes.io/projected/7baf8881-13bf-441c-8417-4648a2efe781-kube-api-access-g96r9\") pod \"maas-api-597f477cb8-sdsxn\" (UID: \"7baf8881-13bf-441c-8417-4648a2efe781\") " pod="odh-ai-gateway-infra/maas-api-597f477cb8-sdsxn"
Jul 20 15:35:58.030586 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:58.030491    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7baf8881-13bf-441c-8417-4648a2efe781-maas-api-tls\") pod \"maas-api-597f477cb8-sdsxn\" (UID: \"7baf8881-13bf-441c-8417-4648a2efe781\") " pod="odh-ai-gateway-infra/maas-api-597f477cb8-sdsxn"
Jul 20 15:35:58.030694 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:35:58.030588    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Jul 20 15:35:58.030694 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:35:58.030646    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7baf8881-13bf-441c-8417-4648a2efe781-maas-api-tls podName:7baf8881-13bf-441c-8417-4648a2efe781 nodeName:}" failed. No retries permitted until 2026-07-20 15:35:58.530629148 +0000 UTC m=+541.540250788 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7baf8881-13bf-441c-8417-4648a2efe781-maas-api-tls") pod "maas-api-597f477cb8-sdsxn" (UID: "7baf8881-13bf-441c-8417-4648a2efe781") : secret "maas-api-serving-cert" not found
Jul 20 15:35:58.031108 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:58.031073    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7baf8881-13bf-441c-8417-4648a2efe781-openshift-service-ca\") pod \"maas-api-597f477cb8-sdsxn\" (UID: \"7baf8881-13bf-441c-8417-4648a2efe781\") " pod="odh-ai-gateway-infra/maas-api-597f477cb8-sdsxn"
Jul 20 15:35:58.051311 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:58.051271    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g96r9\" (UniqueName: \"kubernetes.io/projected/7baf8881-13bf-441c-8417-4648a2efe781-kube-api-access-g96r9\") pod \"maas-api-597f477cb8-sdsxn\" (UID: \"7baf8881-13bf-441c-8417-4648a2efe781\") " pod="odh-ai-gateway-infra/maas-api-597f477cb8-sdsxn"
Jul 20 15:35:58.528424 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:58.528391    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-zj7qr"]
Jul 20 15:35:58.530596 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:58.530579    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-zj7qr"
Jul 20 15:35:58.532995 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:58.532974    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 20 15:35:58.533119 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:58.533037    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-p6x2q\""
Jul 20 15:35:58.533974 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:58.533954    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7baf8881-13bf-441c-8417-4648a2efe781-maas-api-tls\") pod \"maas-api-597f477cb8-sdsxn\" (UID: \"7baf8881-13bf-441c-8417-4648a2efe781\") " pod="odh-ai-gateway-infra/maas-api-597f477cb8-sdsxn"
Jul 20 15:35:58.536732 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:58.536690    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7baf8881-13bf-441c-8417-4648a2efe781-maas-api-tls\") pod \"maas-api-597f477cb8-sdsxn\" (UID: \"7baf8881-13bf-441c-8417-4648a2efe781\") " pod="odh-ai-gateway-infra/maas-api-597f477cb8-sdsxn"
Jul 20 15:35:58.542888 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:58.542864    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-zj7qr"]
Jul 20 15:35:58.634710 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:58.634682    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6sn5\" (UniqueName: \"kubernetes.io/projected/f8fdc8b6-4243-4136-8c59-138c45519065-kube-api-access-j6sn5\") pod \"payload-pre-processing-7677b774bb-zj7qr\" (UID: \"f8fdc8b6-4243-4136-8c59-138c45519065\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-zj7qr"
Jul 20 15:35:58.634988 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:58.634729    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8fdc8b6-4243-4136-8c59-138c45519065-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-zj7qr\" (UID: \"f8fdc8b6-4243-4136-8c59-138c45519065\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-zj7qr"
Jul 20 15:35:58.735303 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:58.735254    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6sn5\" (UniqueName: \"kubernetes.io/projected/f8fdc8b6-4243-4136-8c59-138c45519065-kube-api-access-j6sn5\") pod \"payload-pre-processing-7677b774bb-zj7qr\" (UID: \"f8fdc8b6-4243-4136-8c59-138c45519065\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-zj7qr"
Jul 20 15:35:58.735303 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:58.735306    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8fdc8b6-4243-4136-8c59-138c45519065-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-zj7qr\" (UID: \"f8fdc8b6-4243-4136-8c59-138c45519065\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-zj7qr"
Jul 20 15:35:58.735902 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:58.735881    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8fdc8b6-4243-4136-8c59-138c45519065-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-zj7qr\" (UID: \"f8fdc8b6-4243-4136-8c59-138c45519065\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-zj7qr"
Jul 20 15:35:58.740188 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:58.740167    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-597f477cb8-sdsxn"
Jul 20 15:35:58.744117 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:58.744098    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6sn5\" (UniqueName: \"kubernetes.io/projected/f8fdc8b6-4243-4136-8c59-138c45519065-kube-api-access-j6sn5\") pod \"payload-pre-processing-7677b774bb-zj7qr\" (UID: \"f8fdc8b6-4243-4136-8c59-138c45519065\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-zj7qr"
Jul 20 15:35:58.850886 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:58.850860    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-zj7qr"
Jul 20 15:35:58.863740 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:58.863638    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-597f477cb8-sdsxn"]
Jul 20 15:35:58.865808 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:35:58.865780    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7baf8881_13bf_441c_8417_4648a2efe781.slice/crio-dab6fc20c4b1f6fc7c4632b77f45f179de557e5d32a17b9d7301435e053642dc WatchSource:0}: Error finding container dab6fc20c4b1f6fc7c4632b77f45f179de557e5d32a17b9d7301435e053642dc: Status 404 returned error can't find the container with id dab6fc20c4b1f6fc7c4632b77f45f179de557e5d32a17b9d7301435e053642dc
Jul 20 15:35:58.980510 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:58.980448    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-zj7qr"]
Jul 20 15:35:58.982461 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:35:58.982436    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8fdc8b6_4243_4136_8c59_138c45519065.slice/crio-46bd7db9c5830fe978a1b9b104012abbe786f96b2fc796b25dec44e79461e979 WatchSource:0}: Error finding container 46bd7db9c5830fe978a1b9b104012abbe786f96b2fc796b25dec44e79461e979: Status 404 returned error can't find the container with id 46bd7db9c5830fe978a1b9b104012abbe786f96b2fc796b25dec44e79461e979
Jul 20 15:35:58.985798 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:58.985765    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:35:59.693569 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:59.693528    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-zj7qr" event={"ID":"f8fdc8b6-4243-4136-8c59-138c45519065","Type":"ContainerStarted","Data":"46bd7db9c5830fe978a1b9b104012abbe786f96b2fc796b25dec44e79461e979"}
Jul 20 15:35:59.695053 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:35:59.695023    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-597f477cb8-sdsxn" event={"ID":"7baf8881-13bf-441c-8417-4648a2efe781","Type":"ContainerStarted","Data":"dab6fc20c4b1f6fc7c4632b77f45f179de557e5d32a17b9d7301435e053642dc"}
Jul 20 15:36:01.705104 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:01.705005    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-zj7qr" event={"ID":"f8fdc8b6-4243-4136-8c59-138c45519065","Type":"ContainerStarted","Data":"6dc3facf50b2b0ecc075c5c54b5b4a3d444ae1a3cddfb31999d259eecb18ee8a"}
Jul 20 15:36:01.705104 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:01.705078    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-zj7qr"
Jul 20 15:36:01.706346 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:01.706323    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-597f477cb8-sdsxn" event={"ID":"7baf8881-13bf-441c-8417-4648a2efe781","Type":"ContainerStarted","Data":"473bcfaa7a44c8033a516141d4cc72c73fe3613a9de9b52eac1f39800d7b73be"}
Jul 20 15:36:01.706517 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:01.706501    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-597f477cb8-sdsxn"
Jul 20 15:36:01.723055 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:01.723013    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-zj7qr" podStartSLOduration=1.225313017 podStartE2EDuration="3.722997544s" totalImagesPullingTime="2.497684527s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:35:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:36:01.721534662 +0000 UTC m=+544.731156323" watchObservedRunningTime="2026-07-20 15:36:01.722997544 +0000 UTC m=+544.732619207"
Jul 20 15:36:01.738672 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:01.738623    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-597f477cb8-sdsxn" podStartSLOduration=2.151064384 podStartE2EDuration="4.738607566s" totalImagesPullingTime="2.587543182s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:35:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:36:01.736542797 +0000 UTC m=+544.746164459" watchObservedRunningTime="2026-07-20 15:36:01.738607566 +0000 UTC m=+544.748229227"
Jul 20 15:36:07.715583 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:07.715554    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-597f477cb8-sdsxn"
Jul 20 15:36:08.806412 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:08.806380    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:36:08.933353 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:08.933322    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:36:12.712319 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:12.712273    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-zj7qr"
Jul 20 15:36:14.989426 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:14.989389    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq"]
Jul 20 15:36:14.992205 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:14.992182    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq"
Jul 20 15:36:14.994629 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:14.994606    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 20 15:36:14.994749 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:14.994649    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 20 15:36:14.995477 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:14.995459    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-wqs5r\""
Jul 20 15:36:14.995586 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:14.995460    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 20 15:36:15.001477 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:15.001457    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq"]
Jul 20 15:36:15.011414 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:15.011378    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:36:15.068085 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:15.068053    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/593d4ab2-3745-45d5-8f37-d9d1702e82f8-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq\" (UID: \"593d4ab2-3745-45d5-8f37-d9d1702e82f8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq"
Jul 20 15:36:15.068229 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:15.068094    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/593d4ab2-3745-45d5-8f37-d9d1702e82f8-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq\" (UID: \"593d4ab2-3745-45d5-8f37-d9d1702e82f8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq"
Jul 20 15:36:15.068229 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:15.068124    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/593d4ab2-3745-45d5-8f37-d9d1702e82f8-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq\" (UID: \"593d4ab2-3745-45d5-8f37-d9d1702e82f8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq"
Jul 20 15:36:15.068229 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:15.068159    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/593d4ab2-3745-45d5-8f37-d9d1702e82f8-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq\" (UID: \"593d4ab2-3745-45d5-8f37-d9d1702e82f8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq"
Jul 20 15:36:15.068229 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:15.068188    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/593d4ab2-3745-45d5-8f37-d9d1702e82f8-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq\" (UID: \"593d4ab2-3745-45d5-8f37-d9d1702e82f8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq"
Jul 20 15:36:15.068229 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:15.068212    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndlbs\" (UniqueName: \"kubernetes.io/projected/593d4ab2-3745-45d5-8f37-d9d1702e82f8-kube-api-access-ndlbs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq\" (UID: \"593d4ab2-3745-45d5-8f37-d9d1702e82f8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq"
Jul 20 15:36:15.168661 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:15.168627    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/593d4ab2-3745-45d5-8f37-d9d1702e82f8-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq\" (UID: \"593d4ab2-3745-45d5-8f37-d9d1702e82f8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq"
Jul 20 15:36:15.168814 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:15.168668    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndlbs\" (UniqueName: \"kubernetes.io/projected/593d4ab2-3745-45d5-8f37-d9d1702e82f8-kube-api-access-ndlbs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq\" (UID: \"593d4ab2-3745-45d5-8f37-d9d1702e82f8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq"
Jul 20 15:36:15.168814 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:15.168715    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/593d4ab2-3745-45d5-8f37-d9d1702e82f8-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq\" (UID: \"593d4ab2-3745-45d5-8f37-d9d1702e82f8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq"
Jul 20 15:36:15.168814 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:15.168737    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/593d4ab2-3745-45d5-8f37-d9d1702e82f8-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq\" (UID: \"593d4ab2-3745-45d5-8f37-d9d1702e82f8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq"
Jul 20 15:36:15.168814 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:15.168754    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/593d4ab2-3745-45d5-8f37-d9d1702e82f8-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq\" (UID: \"593d4ab2-3745-45d5-8f37-d9d1702e82f8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq"
Jul 20 15:36:15.168814 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:15.168779    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/593d4ab2-3745-45d5-8f37-d9d1702e82f8-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq\" (UID: \"593d4ab2-3745-45d5-8f37-d9d1702e82f8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq"
Jul 20 15:36:15.169112 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:15.169090    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/593d4ab2-3745-45d5-8f37-d9d1702e82f8-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq\" (UID: \"593d4ab2-3745-45d5-8f37-d9d1702e82f8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq"
Jul 20 15:36:15.169149 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:15.169098    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/593d4ab2-3745-45d5-8f37-d9d1702e82f8-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq\" (UID: \"593d4ab2-3745-45d5-8f37-d9d1702e82f8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq"
Jul 20 15:36:15.169182 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:15.169142    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/593d4ab2-3745-45d5-8f37-d9d1702e82f8-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq\" (UID: \"593d4ab2-3745-45d5-8f37-d9d1702e82f8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq"
Jul 20 15:36:15.171018 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:15.170999    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/593d4ab2-3745-45d5-8f37-d9d1702e82f8-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq\" (UID: \"593d4ab2-3745-45d5-8f37-d9d1702e82f8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq"
Jul 20 15:36:15.171169 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:15.171150    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/593d4ab2-3745-45d5-8f37-d9d1702e82f8-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq\" (UID: \"593d4ab2-3745-45d5-8f37-d9d1702e82f8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq"
Jul 20 15:36:15.176629 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:15.176598    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndlbs\" (UniqueName: \"kubernetes.io/projected/593d4ab2-3745-45d5-8f37-d9d1702e82f8-kube-api-access-ndlbs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq\" (UID: \"593d4ab2-3745-45d5-8f37-d9d1702e82f8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq"
Jul 20 15:36:15.302986 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:15.302954    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq"
Jul 20 15:36:15.430151 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:15.430123    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq"]
Jul 20 15:36:15.432607 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:36:15.432575    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod593d4ab2_3745_45d5_8f37_d9d1702e82f8.slice/crio-d9e21d20e67a2e05edbb04ed888a12283f875b01c8fa2379cf6f0291882c5780 WatchSource:0}: Error finding container d9e21d20e67a2e05edbb04ed888a12283f875b01c8fa2379cf6f0291882c5780: Status 404 returned error can't find the container with id d9e21d20e67a2e05edbb04ed888a12283f875b01c8fa2379cf6f0291882c5780
Jul 20 15:36:15.768035 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:15.768000    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq" event={"ID":"593d4ab2-3745-45d5-8f37-d9d1702e82f8","Type":"ContainerStarted","Data":"d9e21d20e67a2e05edbb04ed888a12283f875b01c8fa2379cf6f0291882c5780"}
Jul 20 15:36:16.774458 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:16.774426    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq" event={"ID":"593d4ab2-3745-45d5-8f37-d9d1702e82f8","Type":"ContainerStarted","Data":"31e25dfe993a01c7e3bd60d647093593099701e9415f1fc03d7435f94f48bcbd"}
Jul 20 15:36:16.792784 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:16.792746    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq" podStartSLOduration=1.610122286 podStartE2EDuration="2.792731746s" totalImagesPullingTime="1.18260946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:36:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:36:16.791407774 +0000 UTC m=+559.801029438" watchObservedRunningTime="2026-07-20 15:36:16.792731746 +0000 UTC m=+559.802353439"
Jul 20 15:36:25.303993 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:25.303963    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq"
Jul 20 15:36:25.304398 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:25.304053    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq"
Jul 20 15:36:25.316664 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:25.316641    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq"
Jul 20 15:36:25.822813 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:25.822786    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq"
Jul 20 15:36:30.931095 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:30.931059    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:36:33.685331 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:33.685282    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:36:33.734565 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:33.734541    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:36:37.484541 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:37.484506    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-56jp7"]
Jul 20 15:36:37.489195 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:37.489174    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56jp7"
Jul 20 15:36:37.495042 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:37.495023    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 20 15:36:37.504671 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:37.504649    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-56jp7"]
Jul 20 15:36:37.527689 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:37.527661    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:36:37.552956 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:37.552924    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e25304b-f3b6-4168-b67e-c1c153bb0f5a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-56jp7\" (UID: \"3e25304b-f3b6-4168-b67e-c1c153bb0f5a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56jp7"
Jul 20 15:36:37.553096 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:37.552980    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rn5wh\" (UniqueName: \"kubernetes.io/projected/3e25304b-f3b6-4168-b67e-c1c153bb0f5a-kube-api-access-rn5wh\") pod \"e2e-trlp-test-simulated-kserve-598466548d-56jp7\" (UID: \"3e25304b-f3b6-4168-b67e-c1c153bb0f5a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56jp7"
Jul 20 15:36:37.553096 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:37.553053    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e25304b-f3b6-4168-b67e-c1c153bb0f5a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-56jp7\" (UID: \"3e25304b-f3b6-4168-b67e-c1c153bb0f5a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56jp7"
Jul 20 15:36:37.553215 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:37.553087    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e25304b-f3b6-4168-b67e-c1c153bb0f5a-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-56jp7\" (UID: \"3e25304b-f3b6-4168-b67e-c1c153bb0f5a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56jp7"
Jul 20 15:36:37.553215 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:37.553132    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e25304b-f3b6-4168-b67e-c1c153bb0f5a-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-56jp7\" (UID: \"3e25304b-f3b6-4168-b67e-c1c153bb0f5a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56jp7"
Jul 20 15:36:37.553215 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:37.553171    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e25304b-f3b6-4168-b67e-c1c153bb0f5a-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-56jp7\" (UID: \"3e25304b-f3b6-4168-b67e-c1c153bb0f5a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56jp7"
Jul 20 15:36:37.654096 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:37.654066    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e25304b-f3b6-4168-b67e-c1c153bb0f5a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-56jp7\" (UID: \"3e25304b-f3b6-4168-b67e-c1c153bb0f5a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56jp7"
Jul 20 15:36:37.654245 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:37.654108    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rn5wh\" (UniqueName: \"kubernetes.io/projected/3e25304b-f3b6-4168-b67e-c1c153bb0f5a-kube-api-access-rn5wh\") pod \"e2e-trlp-test-simulated-kserve-598466548d-56jp7\" (UID: \"3e25304b-f3b6-4168-b67e-c1c153bb0f5a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56jp7"
Jul 20 15:36:37.654245 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:37.654151    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e25304b-f3b6-4168-b67e-c1c153bb0f5a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-56jp7\" (UID: \"3e25304b-f3b6-4168-b67e-c1c153bb0f5a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56jp7"
Jul 20 15:36:37.654245 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:37.654189    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e25304b-f3b6-4168-b67e-c1c153bb0f5a-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-56jp7\" (UID: \"3e25304b-f3b6-4168-b67e-c1c153bb0f5a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56jp7"
Jul 20 15:36:37.654245 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:37.654209    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e25304b-f3b6-4168-b67e-c1c153bb0f5a-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-56jp7\" (UID: \"3e25304b-f3b6-4168-b67e-c1c153bb0f5a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56jp7"
Jul 20 15:36:37.654432 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:37.654263    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e25304b-f3b6-4168-b67e-c1c153bb0f5a-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-56jp7\" (UID: \"3e25304b-f3b6-4168-b67e-c1c153bb0f5a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56jp7"
Jul 20 15:36:37.654655 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:37.654634    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e25304b-f3b6-4168-b67e-c1c153bb0f5a-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-56jp7\" (UID: \"3e25304b-f3b6-4168-b67e-c1c153bb0f5a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56jp7"
Jul 20 15:36:37.654711 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:37.654642    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e25304b-f3b6-4168-b67e-c1c153bb0f5a-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-56jp7\" (UID: \"3e25304b-f3b6-4168-b67e-c1c153bb0f5a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56jp7"
Jul 20 15:36:37.654711 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:37.654686    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e25304b-f3b6-4168-b67e-c1c153bb0f5a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-56jp7\" (UID: \"3e25304b-f3b6-4168-b67e-c1c153bb0f5a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56jp7"
Jul 20 15:36:37.656407 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:37.656386    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e25304b-f3b6-4168-b67e-c1c153bb0f5a-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-56jp7\" (UID: \"3e25304b-f3b6-4168-b67e-c1c153bb0f5a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56jp7"
Jul 20 15:36:37.656661 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:37.656643    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e25304b-f3b6-4168-b67e-c1c153bb0f5a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-56jp7\" (UID: \"3e25304b-f3b6-4168-b67e-c1c153bb0f5a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56jp7"
Jul 20 15:36:37.663598 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:37.663576    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rn5wh\" (UniqueName: \"kubernetes.io/projected/3e25304b-f3b6-4168-b67e-c1c153bb0f5a-kube-api-access-rn5wh\") pod \"e2e-trlp-test-simulated-kserve-598466548d-56jp7\" (UID: \"3e25304b-f3b6-4168-b67e-c1c153bb0f5a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56jp7"
Jul 20 15:36:37.801977 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:37.801951    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56jp7"
Jul 20 15:36:37.930258 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:37.930233    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-56jp7"]
Jul 20 15:36:37.932819 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:36:37.932791    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e25304b_f3b6_4168_b67e_c1c153bb0f5a.slice/crio-df4f155f4c80a3753b1a7c4cae6b189b16482cc4b5e07349876852a7ca12d3db WatchSource:0}: Error finding container df4f155f4c80a3753b1a7c4cae6b189b16482cc4b5e07349876852a7ca12d3db: Status 404 returned error can't find the container with id df4f155f4c80a3753b1a7c4cae6b189b16482cc4b5e07349876852a7ca12d3db
Jul 20 15:36:38.867805 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:38.867768    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56jp7" event={"ID":"3e25304b-f3b6-4168-b67e-c1c153bb0f5a","Type":"ContainerStarted","Data":"0b9cc4880288762c3a8b3e0a85dc04bfd692bf674a975fb8fccf55a19bd2c524"}
Jul 20 15:36:38.868251 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:38.867818    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56jp7" event={"ID":"3e25304b-f3b6-4168-b67e-c1c153bb0f5a","Type":"ContainerStarted","Data":"df4f155f4c80a3753b1a7c4cae6b189b16482cc4b5e07349876852a7ca12d3db"}
Jul 20 15:36:38.886386 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:38.886344    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56jp7" podStartSLOduration=1.7019360350000001 podStartE2EDuration="1.886329915s" totalImagesPullingTime="184.39388ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:36:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:36:38.885114882 +0000 UTC m=+581.894736555" watchObservedRunningTime="2026-07-20 15:36:38.886329915 +0000 UTC m=+581.895951576"
Jul 20 15:36:46.565004 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:46.564972    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:36:46.899509 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:46.899423    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:36:47.802338 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:47.802310    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56jp7"
Jul 20 15:36:47.802688 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:47.802360    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56jp7"
Jul 20 15:36:47.814586 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:47.814563    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56jp7"
Jul 20 15:36:47.916703 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:47.916679    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56jp7"
Jul 20 15:36:49.915754 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:49.915723    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:36:53.126343 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:53.126310    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:36:57.332381 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:36:57.332351    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:37:03.833087 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:37:03.833050    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:38:04.586281 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:04.586250    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-959fcf8ff-7fq7c"]
Jul 20 15:38:04.589560 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:04.589543    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-959fcf8ff-7fq7c"
Jul 20 15:38:04.631186 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:04.631165    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-959fcf8ff-7fq7c"]
Jul 20 15:38:04.635882 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:04.635854    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7j8z7\" (UniqueName: \"kubernetes.io/projected/ad3725af-f98b-40ef-a66f-8949d3e21236-kube-api-access-7j8z7\") pod \"authorino-959fcf8ff-7fq7c\" (UID: \"ad3725af-f98b-40ef-a66f-8949d3e21236\") " pod="kuadrant-system/authorino-959fcf8ff-7fq7c"
Jul 20 15:38:04.635967 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:04.635906    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ad3725af-f98b-40ef-a66f-8949d3e21236-tls-cert\") pod \"authorino-959fcf8ff-7fq7c\" (UID: \"ad3725af-f98b-40ef-a66f-8949d3e21236\") " pod="kuadrant-system/authorino-959fcf8ff-7fq7c"
Jul 20 15:38:04.736743 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:04.736716    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ad3725af-f98b-40ef-a66f-8949d3e21236-tls-cert\") pod \"authorino-959fcf8ff-7fq7c\" (UID: \"ad3725af-f98b-40ef-a66f-8949d3e21236\") " pod="kuadrant-system/authorino-959fcf8ff-7fq7c"
Jul 20 15:38:04.736856 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:04.736844    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7j8z7\" (UniqueName: \"kubernetes.io/projected/ad3725af-f98b-40ef-a66f-8949d3e21236-kube-api-access-7j8z7\") pod \"authorino-959fcf8ff-7fq7c\" (UID: \"ad3725af-f98b-40ef-a66f-8949d3e21236\") " pod="kuadrant-system/authorino-959fcf8ff-7fq7c"
Jul 20 15:38:04.739253 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:04.739229    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ad3725af-f98b-40ef-a66f-8949d3e21236-tls-cert\") pod \"authorino-959fcf8ff-7fq7c\" (UID: \"ad3725af-f98b-40ef-a66f-8949d3e21236\") " pod="kuadrant-system/authorino-959fcf8ff-7fq7c"
Jul 20 15:38:04.749030 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:04.749008    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j8z7\" (UniqueName: \"kubernetes.io/projected/ad3725af-f98b-40ef-a66f-8949d3e21236-kube-api-access-7j8z7\") pod \"authorino-959fcf8ff-7fq7c\" (UID: \"ad3725af-f98b-40ef-a66f-8949d3e21236\") " pod="kuadrant-system/authorino-959fcf8ff-7fq7c"
Jul 20 15:38:04.898909 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:04.898841    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-959fcf8ff-7fq7c"
Jul 20 15:38:05.048110 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:05.048077    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-959fcf8ff-7fq7c"]
Jul 20 15:38:05.077886 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:05.077863    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:38:05.255017 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:05.254948    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-959fcf8ff-7fq7c" event={"ID":"ad3725af-f98b-40ef-a66f-8949d3e21236","Type":"ContainerStarted","Data":"6fafaaa00b2740e95bc75dc57feb0cd4bf2d442d07bf170001de03c7cdb9ee5c"}
Jul 20 15:38:05.310741 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:05.310709    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:38:06.260832 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:06.260801    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-959fcf8ff-7fq7c" event={"ID":"ad3725af-f98b-40ef-a66f-8949d3e21236","Type":"ContainerStarted","Data":"6ab92d8736df7be22d73860b935094fd71238aff43967b5a2747a633be1b5997"}
Jul 20 15:38:06.296217 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:06.296149    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-959fcf8ff-7fq7c" podStartSLOduration=1.818075165 podStartE2EDuration="2.296126584s" totalImagesPullingTime="478.051419ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:38:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:38:06.294369979 +0000 UTC m=+669.303991641" watchObservedRunningTime="2026-07-20 15:38:06.296126584 +0000 UTC m=+669.305748247"
Jul 20 15:38:06.358143 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:06.358105    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7679b6dbfd-hr46t"]
Jul 20 15:38:06.358335 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:06.358311    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7679b6dbfd-hr46t" podUID="48100c36-b8db-4f86-a728-170ae963e729" containerName="authorino" containerID="cri-o://214fd2430d81687708399b092c75b5e951d9656c3744a155b2996166c9fe5c87" gracePeriod=30
Jul 20 15:38:06.400399 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:06.400344    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:38:06.566692 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:06.566665    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:38:06.612454 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:06.612423    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7679b6dbfd-hr46t"
Jul 20 15:38:06.654425 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:06.654391    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/48100c36-b8db-4f86-a728-170ae963e729-tls-cert\") pod \"48100c36-b8db-4f86-a728-170ae963e729\" (UID: \"48100c36-b8db-4f86-a728-170ae963e729\") "
Jul 20 15:38:06.654606 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:06.654450    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mft8q\" (UniqueName: \"kubernetes.io/projected/48100c36-b8db-4f86-a728-170ae963e729-kube-api-access-mft8q\") pod \"48100c36-b8db-4f86-a728-170ae963e729\" (UID: \"48100c36-b8db-4f86-a728-170ae963e729\") "
Jul 20 15:38:06.657898 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:06.656967    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48100c36-b8db-4f86-a728-170ae963e729-kube-api-access-mft8q" (OuterVolumeSpecName: "kube-api-access-mft8q") pod "48100c36-b8db-4f86-a728-170ae963e729" (UID: "48100c36-b8db-4f86-a728-170ae963e729"). InnerVolumeSpecName "kube-api-access-mft8q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 15:38:06.667655 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:06.667630    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48100c36-b8db-4f86-a728-170ae963e729-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "48100c36-b8db-4f86-a728-170ae963e729" (UID: "48100c36-b8db-4f86-a728-170ae963e729"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 15:38:06.755897 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:06.755866    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mft8q\" (UniqueName: \"kubernetes.io/projected/48100c36-b8db-4f86-a728-170ae963e729-kube-api-access-mft8q\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:38:06.755897 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:06.755895    2559 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/48100c36-b8db-4f86-a728-170ae963e729-tls-cert\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:38:07.266674 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:07.266638    2559 generic.go:356] "Generic (PLEG): container finished" podID="48100c36-b8db-4f86-a728-170ae963e729" containerID="214fd2430d81687708399b092c75b5e951d9656c3744a155b2996166c9fe5c87" exitCode=0
Jul 20 15:38:07.267068 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:07.266692    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7679b6dbfd-hr46t"
Jul 20 15:38:07.267068 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:07.266729    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7679b6dbfd-hr46t" event={"ID":"48100c36-b8db-4f86-a728-170ae963e729","Type":"ContainerDied","Data":"214fd2430d81687708399b092c75b5e951d9656c3744a155b2996166c9fe5c87"}
Jul 20 15:38:07.267068 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:07.266768    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7679b6dbfd-hr46t" event={"ID":"48100c36-b8db-4f86-a728-170ae963e729","Type":"ContainerDied","Data":"4f99a9744e27ab552910a490b874652be1b6868384995e962f25e4e2cf15463d"}
Jul 20 15:38:07.267068 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:07.266789    2559 scope.go:117] "RemoveContainer" containerID="214fd2430d81687708399b092c75b5e951d9656c3744a155b2996166c9fe5c87"
Jul 20 15:38:07.276345 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:07.276321    2559 scope.go:117] "RemoveContainer" containerID="214fd2430d81687708399b092c75b5e951d9656c3744a155b2996166c9fe5c87"
Jul 20 15:38:07.276643 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:38:07.276623    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"214fd2430d81687708399b092c75b5e951d9656c3744a155b2996166c9fe5c87\": container with ID starting with 214fd2430d81687708399b092c75b5e951d9656c3744a155b2996166c9fe5c87 not found: ID does not exist" containerID="214fd2430d81687708399b092c75b5e951d9656c3744a155b2996166c9fe5c87"
Jul 20 15:38:07.276729 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:07.276654    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"214fd2430d81687708399b092c75b5e951d9656c3744a155b2996166c9fe5c87"} err="failed to get container status \"214fd2430d81687708399b092c75b5e951d9656c3744a155b2996166c9fe5c87\": rpc error: code = NotFound desc = could not find container \"214fd2430d81687708399b092c75b5e951d9656c3744a155b2996166c9fe5c87\": container with ID starting with 214fd2430d81687708399b092c75b5e951d9656c3744a155b2996166c9fe5c87 not found: ID does not exist"
Jul 20 15:38:07.301489 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:07.301465    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7679b6dbfd-hr46t"]
Jul 20 15:38:07.319537 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:07.319505    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7679b6dbfd-hr46t"]
Jul 20 15:38:07.521510 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:07.521436    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48100c36-b8db-4f86-a728-170ae963e729" path="/var/lib/kubelet/pods/48100c36-b8db-4f86-a728-170ae963e729/volumes"
Jul 20 15:38:17.629776 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:17.629696    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:38:17.714599 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:17.714567    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:38:48.750192 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:48.750156    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:38:48.850770 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:48.850740    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:38:53.634249 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:53.634220    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-854b67b788-nh8m2"]
Jul 20 15:38:53.634642 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:53.634433    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-854b67b788-nh8m2" podUID="f63a30bf-ce64-4ea5-ae1a-4331dc7f6cf3" containerName="manager" containerID="cri-o://9ea5b1d8b25d39d8bfb29090067dcdff7683e35c86ab8950775685151a7a0ee8" gracePeriod=30
Jul 20 15:38:53.875969 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:53.875947    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-854b67b788-nh8m2"
Jul 20 15:38:53.949450 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:53.949384    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xpt8g\" (UniqueName: \"kubernetes.io/projected/f63a30bf-ce64-4ea5-ae1a-4331dc7f6cf3-kube-api-access-xpt8g\") pod \"f63a30bf-ce64-4ea5-ae1a-4331dc7f6cf3\" (UID: \"f63a30bf-ce64-4ea5-ae1a-4331dc7f6cf3\") "
Jul 20 15:38:53.949450 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:53.949417    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f63a30bf-ce64-4ea5-ae1a-4331dc7f6cf3-webhook-cert\") pod \"f63a30bf-ce64-4ea5-ae1a-4331dc7f6cf3\" (UID: \"f63a30bf-ce64-4ea5-ae1a-4331dc7f6cf3\") "
Jul 20 15:38:53.951277 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:53.951251    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f63a30bf-ce64-4ea5-ae1a-4331dc7f6cf3-kube-api-access-xpt8g" (OuterVolumeSpecName: "kube-api-access-xpt8g") pod "f63a30bf-ce64-4ea5-ae1a-4331dc7f6cf3" (UID: "f63a30bf-ce64-4ea5-ae1a-4331dc7f6cf3"). InnerVolumeSpecName "kube-api-access-xpt8g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 15:38:53.951405 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:53.951354    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f63a30bf-ce64-4ea5-ae1a-4331dc7f6cf3-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "f63a30bf-ce64-4ea5-ae1a-4331dc7f6cf3" (UID: "f63a30bf-ce64-4ea5-ae1a-4331dc7f6cf3"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 15:38:54.050544 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:54.050516    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xpt8g\" (UniqueName: \"kubernetes.io/projected/f63a30bf-ce64-4ea5-ae1a-4331dc7f6cf3-kube-api-access-xpt8g\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:38:54.050544 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:54.050542    2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f63a30bf-ce64-4ea5-ae1a-4331dc7f6cf3-webhook-cert\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:38:54.481126 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:54.481102    2559 generic.go:356] "Generic (PLEG): container finished" podID="f63a30bf-ce64-4ea5-ae1a-4331dc7f6cf3" containerID="9ea5b1d8b25d39d8bfb29090067dcdff7683e35c86ab8950775685151a7a0ee8" exitCode=0
Jul 20 15:38:54.481275 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:54.481148    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-854b67b788-nh8m2" event={"ID":"f63a30bf-ce64-4ea5-ae1a-4331dc7f6cf3","Type":"ContainerDied","Data":"9ea5b1d8b25d39d8bfb29090067dcdff7683e35c86ab8950775685151a7a0ee8"}
Jul 20 15:38:54.481275 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:54.481154    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-854b67b788-nh8m2"
Jul 20 15:38:54.481275 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:54.481188    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-854b67b788-nh8m2" event={"ID":"f63a30bf-ce64-4ea5-ae1a-4331dc7f6cf3","Type":"ContainerDied","Data":"6a3c596ab631e4a50e8671898dae52e2e471bb78a540f61ce151825b17637b29"}
Jul 20 15:38:54.481275 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:54.481209    2559 scope.go:117] "RemoveContainer" containerID="9ea5b1d8b25d39d8bfb29090067dcdff7683e35c86ab8950775685151a7a0ee8"
Jul 20 15:38:54.494638 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:54.494621    2559 scope.go:117] "RemoveContainer" containerID="9ea5b1d8b25d39d8bfb29090067dcdff7683e35c86ab8950775685151a7a0ee8"
Jul 20 15:38:54.494864 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:38:54.494846    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ea5b1d8b25d39d8bfb29090067dcdff7683e35c86ab8950775685151a7a0ee8\": container with ID starting with 9ea5b1d8b25d39d8bfb29090067dcdff7683e35c86ab8950775685151a7a0ee8 not found: ID does not exist" containerID="9ea5b1d8b25d39d8bfb29090067dcdff7683e35c86ab8950775685151a7a0ee8"
Jul 20 15:38:54.494905 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:54.494872    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ea5b1d8b25d39d8bfb29090067dcdff7683e35c86ab8950775685151a7a0ee8"} err="failed to get container status \"9ea5b1d8b25d39d8bfb29090067dcdff7683e35c86ab8950775685151a7a0ee8\": rpc error: code = NotFound desc = could not find container \"9ea5b1d8b25d39d8bfb29090067dcdff7683e35c86ab8950775685151a7a0ee8\": container with ID starting with 9ea5b1d8b25d39d8bfb29090067dcdff7683e35c86ab8950775685151a7a0ee8 not found: ID does not exist"
Jul 20 15:38:54.521081 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:54.521063    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-854b67b788-nh8m2"]
Jul 20 15:38:54.533616 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:54.533595    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-854b67b788-nh8m2"]
Jul 20 15:38:55.513820 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:38:55.513791    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f63a30bf-ce64-4ea5-ae1a-4331dc7f6cf3" path="/var/lib/kubelet/pods/f63a30bf-ce64-4ea5-ae1a-4331dc7f6cf3/volumes"
Jul 20 15:39:55.363999 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:39:55.363964    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-854b67b788-zbmlc"]
Jul 20 15:39:55.364506 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:39:55.364365    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48100c36-b8db-4f86-a728-170ae963e729" containerName="authorino"
Jul 20 15:39:55.364506 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:39:55.364379    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="48100c36-b8db-4f86-a728-170ae963e729" containerName="authorino"
Jul 20 15:39:55.364506 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:39:55.364402    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f63a30bf-ce64-4ea5-ae1a-4331dc7f6cf3" containerName="manager"
Jul 20 15:39:55.364506 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:39:55.364408    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f63a30bf-ce64-4ea5-ae1a-4331dc7f6cf3" containerName="manager"
Jul 20 15:39:55.364506 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:39:55.364466    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="48100c36-b8db-4f86-a728-170ae963e729" containerName="authorino"
Jul 20 15:39:55.364506 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:39:55.364473    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f63a30bf-ce64-4ea5-ae1a-4331dc7f6cf3" containerName="manager"
Jul 20 15:39:55.367504 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:39:55.367487    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-854b67b788-zbmlc"
Jul 20 15:39:55.371029 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:39:55.371005    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-nvkzc\""
Jul 20 15:39:55.371164 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:39:55.371060    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 20 15:39:55.371164 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:39:55.371009    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 20 15:39:55.378185 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:39:55.378165    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-854b67b788-zbmlc"]
Jul 20 15:39:55.451616 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:39:55.451593    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8fddb36b-00da-4988-b6b1-facf4c44ec22-webhook-cert\") pod \"maas-controller-854b67b788-zbmlc\" (UID: \"8fddb36b-00da-4988-b6b1-facf4c44ec22\") " pod="opendatahub/maas-controller-854b67b788-zbmlc"
Jul 20 15:39:55.451714 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:39:55.451620    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fswxq\" (UniqueName: \"kubernetes.io/projected/8fddb36b-00da-4988-b6b1-facf4c44ec22-kube-api-access-fswxq\") pod \"maas-controller-854b67b788-zbmlc\" (UID: \"8fddb36b-00da-4988-b6b1-facf4c44ec22\") " pod="opendatahub/maas-controller-854b67b788-zbmlc"
Jul 20 15:39:55.552638 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:39:55.552616    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8fddb36b-00da-4988-b6b1-facf4c44ec22-webhook-cert\") pod \"maas-controller-854b67b788-zbmlc\" (UID: \"8fddb36b-00da-4988-b6b1-facf4c44ec22\") " pod="opendatahub/maas-controller-854b67b788-zbmlc"
Jul 20 15:39:55.552739 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:39:55.552647    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fswxq\" (UniqueName: \"kubernetes.io/projected/8fddb36b-00da-4988-b6b1-facf4c44ec22-kube-api-access-fswxq\") pod \"maas-controller-854b67b788-zbmlc\" (UID: \"8fddb36b-00da-4988-b6b1-facf4c44ec22\") " pod="opendatahub/maas-controller-854b67b788-zbmlc"
Jul 20 15:39:55.555141 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:39:55.555123    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8fddb36b-00da-4988-b6b1-facf4c44ec22-webhook-cert\") pod \"maas-controller-854b67b788-zbmlc\" (UID: \"8fddb36b-00da-4988-b6b1-facf4c44ec22\") " pod="opendatahub/maas-controller-854b67b788-zbmlc"
Jul 20 15:39:55.562112 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:39:55.562089    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fswxq\" (UniqueName: \"kubernetes.io/projected/8fddb36b-00da-4988-b6b1-facf4c44ec22-kube-api-access-fswxq\") pod \"maas-controller-854b67b788-zbmlc\" (UID: \"8fddb36b-00da-4988-b6b1-facf4c44ec22\") " pod="opendatahub/maas-controller-854b67b788-zbmlc"
Jul 20 15:39:55.677482 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:39:55.677420    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-854b67b788-zbmlc"
Jul 20 15:39:55.802306 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:39:55.802120    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-854b67b788-zbmlc"]
Jul 20 15:39:55.805012 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:39:55.804990    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fddb36b_00da_4988_b6b1_facf4c44ec22.slice/crio-c6536427bcd2a5c8d3c6ebb1532e7ce184ffd9764eeced763964dd7e7491af1f WatchSource:0}: Error finding container c6536427bcd2a5c8d3c6ebb1532e7ce184ffd9764eeced763964dd7e7491af1f: Status 404 returned error can't find the container with id c6536427bcd2a5c8d3c6ebb1532e7ce184ffd9764eeced763964dd7e7491af1f
Jul 20 15:39:55.806706 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:39:55.806689    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 15:39:56.741671 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:39:56.741590    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-854b67b788-zbmlc" event={"ID":"8fddb36b-00da-4988-b6b1-facf4c44ec22","Type":"ContainerStarted","Data":"0688e70cdbd5119c0d351b5ae5239506191cbeb8796935127dc477aa374a0f1e"}
Jul 20 15:39:56.741671 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:39:56.741646    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-854b67b788-zbmlc"
Jul 20 15:39:56.741671 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:39:56.741661    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-854b67b788-zbmlc" event={"ID":"8fddb36b-00da-4988-b6b1-facf4c44ec22","Type":"ContainerStarted","Data":"c6536427bcd2a5c8d3c6ebb1532e7ce184ffd9764eeced763964dd7e7491af1f"}
Jul 20 15:39:56.763915 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:39:56.763873    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-854b67b788-zbmlc" podStartSLOduration=1.305182355 podStartE2EDuration="1.763858612s" totalImagesPullingTime="458.676257ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:39:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:39:56.761421619 +0000 UTC m=+779.771043299" watchObservedRunningTime="2026-07-20 15:39:56.763858612 +0000 UTC m=+779.773480273"
Jul 20 15:40:07.747799 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:07.747767    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-854b67b788-zbmlc"
Jul 20 15:40:12.320246 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:12.320212    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:40:12.432032 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:12.431996    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-854b67b788-zbmlc"]
Jul 20 15:40:12.432214 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:12.432174    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-854b67b788-zbmlc" podUID="8fddb36b-00da-4988-b6b1-facf4c44ec22" containerName="manager" containerID="cri-o://0688e70cdbd5119c0d351b5ae5239506191cbeb8796935127dc477aa374a0f1e" gracePeriod=30
Jul 20 15:40:12.671976 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:12.671953    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-854b67b788-zbmlc"
Jul 20 15:40:12.797476 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:12.797451    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8fddb36b-00da-4988-b6b1-facf4c44ec22-webhook-cert\") pod \"8fddb36b-00da-4988-b6b1-facf4c44ec22\" (UID: \"8fddb36b-00da-4988-b6b1-facf4c44ec22\") "
Jul 20 15:40:12.797629 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:12.797513    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fswxq\" (UniqueName: \"kubernetes.io/projected/8fddb36b-00da-4988-b6b1-facf4c44ec22-kube-api-access-fswxq\") pod \"8fddb36b-00da-4988-b6b1-facf4c44ec22\" (UID: \"8fddb36b-00da-4988-b6b1-facf4c44ec22\") "
Jul 20 15:40:12.799464 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:12.799438    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fddb36b-00da-4988-b6b1-facf4c44ec22-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "8fddb36b-00da-4988-b6b1-facf4c44ec22" (UID: "8fddb36b-00da-4988-b6b1-facf4c44ec22"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 15:40:12.799588 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:12.799530    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fddb36b-00da-4988-b6b1-facf4c44ec22-kube-api-access-fswxq" (OuterVolumeSpecName: "kube-api-access-fswxq") pod "8fddb36b-00da-4988-b6b1-facf4c44ec22" (UID: "8fddb36b-00da-4988-b6b1-facf4c44ec22"). InnerVolumeSpecName "kube-api-access-fswxq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 15:40:12.810257 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:12.810228    2559 generic.go:356] "Generic (PLEG): container finished" podID="8fddb36b-00da-4988-b6b1-facf4c44ec22" containerID="0688e70cdbd5119c0d351b5ae5239506191cbeb8796935127dc477aa374a0f1e" exitCode=0
Jul 20 15:40:12.810380 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:12.810269    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-854b67b788-zbmlc"
Jul 20 15:40:12.810380 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:12.810322    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-854b67b788-zbmlc" event={"ID":"8fddb36b-00da-4988-b6b1-facf4c44ec22","Type":"ContainerDied","Data":"0688e70cdbd5119c0d351b5ae5239506191cbeb8796935127dc477aa374a0f1e"}
Jul 20 15:40:12.810465 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:12.810383    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-854b67b788-zbmlc" event={"ID":"8fddb36b-00da-4988-b6b1-facf4c44ec22","Type":"ContainerDied","Data":"c6536427bcd2a5c8d3c6ebb1532e7ce184ffd9764eeced763964dd7e7491af1f"}
Jul 20 15:40:12.810465 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:12.810420    2559 scope.go:117] "RemoveContainer" containerID="0688e70cdbd5119c0d351b5ae5239506191cbeb8796935127dc477aa374a0f1e"
Jul 20 15:40:12.818767 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:12.818752    2559 scope.go:117] "RemoveContainer" containerID="0688e70cdbd5119c0d351b5ae5239506191cbeb8796935127dc477aa374a0f1e"
Jul 20 15:40:12.819016 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:40:12.819000    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0688e70cdbd5119c0d351b5ae5239506191cbeb8796935127dc477aa374a0f1e\": container with ID starting with 0688e70cdbd5119c0d351b5ae5239506191cbeb8796935127dc477aa374a0f1e not found: ID does not exist" containerID="0688e70cdbd5119c0d351b5ae5239506191cbeb8796935127dc477aa374a0f1e"
Jul 20 15:40:12.819071 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:12.819022    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0688e70cdbd5119c0d351b5ae5239506191cbeb8796935127dc477aa374a0f1e"} err="failed to get container status \"0688e70cdbd5119c0d351b5ae5239506191cbeb8796935127dc477aa374a0f1e\": rpc error: code = NotFound desc = could not find container \"0688e70cdbd5119c0d351b5ae5239506191cbeb8796935127dc477aa374a0f1e\": container with ID starting with 0688e70cdbd5119c0d351b5ae5239506191cbeb8796935127dc477aa374a0f1e not found: ID does not exist"
Jul 20 15:40:12.832869 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:12.832851    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-854b67b788-zbmlc"]
Jul 20 15:40:12.841399 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:12.841374    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-854b67b788-zbmlc"]
Jul 20 15:40:12.898809 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:12.898755    2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8fddb36b-00da-4988-b6b1-facf4c44ec22-webhook-cert\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:40:12.898809 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:12.898777    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fswxq\" (UniqueName: \"kubernetes.io/projected/8fddb36b-00da-4988-b6b1-facf4c44ec22-kube-api-access-fswxq\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:40:13.514487 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:13.514459    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fddb36b-00da-4988-b6b1-facf4c44ec22" path="/var/lib/kubelet/pods/8fddb36b-00da-4988-b6b1-facf4c44ec22/volumes"
Jul 20 15:40:13.650964 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:13.650938    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:40:13.714891 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:13.714859    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-854b67b788-f8jpt"]
Jul 20 15:40:13.715256 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:13.715240    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fddb36b-00da-4988-b6b1-facf4c44ec22" containerName="manager"
Jul 20 15:40:13.715350 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:13.715258    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fddb36b-00da-4988-b6b1-facf4c44ec22" containerName="manager"
Jul 20 15:40:13.715411 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:13.715387    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fddb36b-00da-4988-b6b1-facf4c44ec22" containerName="manager"
Jul 20 15:40:13.719620 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:13.719599    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-854b67b788-f8jpt"
Jul 20 15:40:13.722752 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:13.722733    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 20 15:40:13.722854 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:13.722782    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 20 15:40:13.723747 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:13.723732    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-nvkzc\""
Jul 20 15:40:13.729455 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:13.729436    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-854b67b788-f8jpt"]
Jul 20 15:40:13.907715 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:13.907649    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bd2m9\" (UniqueName: \"kubernetes.io/projected/760c77d8-0f3d-4d1b-8584-f7d087872a76-kube-api-access-bd2m9\") pod \"maas-controller-854b67b788-f8jpt\" (UID: \"760c77d8-0f3d-4d1b-8584-f7d087872a76\") " pod="opendatahub/maas-controller-854b67b788-f8jpt"
Jul 20 15:40:13.907715 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:13.907688    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/760c77d8-0f3d-4d1b-8584-f7d087872a76-webhook-cert\") pod \"maas-controller-854b67b788-f8jpt\" (UID: \"760c77d8-0f3d-4d1b-8584-f7d087872a76\") " pod="opendatahub/maas-controller-854b67b788-f8jpt"
Jul 20 15:40:14.008133 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:14.008101    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/760c77d8-0f3d-4d1b-8584-f7d087872a76-webhook-cert\") pod \"maas-controller-854b67b788-f8jpt\" (UID: \"760c77d8-0f3d-4d1b-8584-f7d087872a76\") " pod="opendatahub/maas-controller-854b67b788-f8jpt"
Jul 20 15:40:14.008262 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:14.008211    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bd2m9\" (UniqueName: \"kubernetes.io/projected/760c77d8-0f3d-4d1b-8584-f7d087872a76-kube-api-access-bd2m9\") pod \"maas-controller-854b67b788-f8jpt\" (UID: \"760c77d8-0f3d-4d1b-8584-f7d087872a76\") " pod="opendatahub/maas-controller-854b67b788-f8jpt"
Jul 20 15:40:14.010653 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:14.010630    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/760c77d8-0f3d-4d1b-8584-f7d087872a76-webhook-cert\") pod \"maas-controller-854b67b788-f8jpt\" (UID: \"760c77d8-0f3d-4d1b-8584-f7d087872a76\") " pod="opendatahub/maas-controller-854b67b788-f8jpt"
Jul 20 15:40:14.017684 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:14.017652    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bd2m9\" (UniqueName: \"kubernetes.io/projected/760c77d8-0f3d-4d1b-8584-f7d087872a76-kube-api-access-bd2m9\") pod \"maas-controller-854b67b788-f8jpt\" (UID: \"760c77d8-0f3d-4d1b-8584-f7d087872a76\") " pod="opendatahub/maas-controller-854b67b788-f8jpt"
Jul 20 15:40:14.029389 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:14.029370    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-854b67b788-f8jpt"
Jul 20 15:40:14.155581 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:14.155558    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-854b67b788-f8jpt"]
Jul 20 15:40:14.158173 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:40:14.158113    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod760c77d8_0f3d_4d1b_8584_f7d087872a76.slice/crio-dd4472e4b85b5789544615c7d0235560e094d461ed77b52e2820379831f6818a WatchSource:0}: Error finding container dd4472e4b85b5789544615c7d0235560e094d461ed77b52e2820379831f6818a: Status 404 returned error can't find the container with id dd4472e4b85b5789544615c7d0235560e094d461ed77b52e2820379831f6818a
Jul 20 15:40:14.820422 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:14.820391    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-854b67b788-f8jpt" event={"ID":"760c77d8-0f3d-4d1b-8584-f7d087872a76","Type":"ContainerStarted","Data":"f388ac170b6c2afac7e932e4125c5d716804b8568bb636ae01ae530cea3cc2d2"}
Jul 20 15:40:14.820836 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:14.820439    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-854b67b788-f8jpt" event={"ID":"760c77d8-0f3d-4d1b-8584-f7d087872a76","Type":"ContainerStarted","Data":"dd4472e4b85b5789544615c7d0235560e094d461ed77b52e2820379831f6818a"}
Jul 20 15:40:14.820836 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:14.820475    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-854b67b788-f8jpt"
Jul 20 15:40:14.840244 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:14.840202    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-854b67b788-f8jpt" podStartSLOduration=1.322034606 podStartE2EDuration="1.840187909s" totalImagesPullingTime="518.153303ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:40:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:40:14.838534004 +0000 UTC m=+797.848155665" watchObservedRunningTime="2026-07-20 15:40:14.840187909 +0000 UTC m=+797.849809571"
Jul 20 15:40:25.827710 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:25.827677    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-854b67b788-f8jpt"
Jul 20 15:40:32.876701 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:32.876667    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:40:56.643487 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:56.643453    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:40:56.737062 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:56.737024    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:40:56.879905 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:40:56.879872    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:41:02.615622 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:41:02.615591    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:41:03.958814 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:41:03.958781    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:41:08.657855 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:41:08.657821    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:41:08.724845 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:41:08.724817    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:41:32.168909 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:41:32.168875    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:43:30.073424 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:43:30.073379    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:43:41.921692 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:43:41.921657    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:46:15.426507 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:46:15.426473    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:47:22.542592 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:47:22.542557    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:48:54.102746 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:48:54.102706    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl"]
Jul 20 15:48:54.106314 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:48:54.106275    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl"
Jul 20 15:48:54.108832 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:48:54.108811    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-759851f5-kserve-self-signed-certs\""
Jul 20 15:48:54.126187 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:48:54.126162    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl"]
Jul 20 15:48:54.163529 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:48:54.163495    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:48:54.207427 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:48:54.207393    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f2a9862-722a-4493-955f-9c456ecfea9c-home\") pod \"e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl\" (UID: \"3f2a9862-722a-4493-955f-9c456ecfea9c\") " pod="llm/e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl"
Jul 20 15:48:54.207427 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:48:54.207429    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xhnk\" (UniqueName: \"kubernetes.io/projected/3f2a9862-722a-4493-955f-9c456ecfea9c-kube-api-access-9xhnk\") pod \"e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl\" (UID: \"3f2a9862-722a-4493-955f-9c456ecfea9c\") " pod="llm/e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl"
Jul 20 15:48:54.207616 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:48:54.207469    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f2a9862-722a-4493-955f-9c456ecfea9c-model-cache\") pod \"e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl\" (UID: \"3f2a9862-722a-4493-955f-9c456ecfea9c\") " pod="llm/e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl"
Jul 20 15:48:54.207616 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:48:54.207504    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f2a9862-722a-4493-955f-9c456ecfea9c-dshm\") pod \"e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl\" (UID: \"3f2a9862-722a-4493-955f-9c456ecfea9c\") " pod="llm/e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl"
Jul 20 15:48:54.207616 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:48:54.207543    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f2a9862-722a-4493-955f-9c456ecfea9c-tls-certs\") pod \"e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl\" (UID: \"3f2a9862-722a-4493-955f-9c456ecfea9c\") " pod="llm/e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl"
Jul 20 15:48:54.207616 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:48:54.207566    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f2a9862-722a-4493-955f-9c456ecfea9c-tmp-dir\") pod \"e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl\" (UID: \"3f2a9862-722a-4493-955f-9c456ecfea9c\") " pod="llm/e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl"
Jul 20 15:48:54.308902 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:48:54.308861    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f2a9862-722a-4493-955f-9c456ecfea9c-tmp-dir\") pod \"e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl\" (UID: \"3f2a9862-722a-4493-955f-9c456ecfea9c\") " pod="llm/e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl"
Jul 20 15:48:54.309077 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:48:54.308954    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f2a9862-722a-4493-955f-9c456ecfea9c-home\") pod \"e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl\" (UID: \"3f2a9862-722a-4493-955f-9c456ecfea9c\") " pod="llm/e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl"
Jul 20 15:48:54.309077 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:48:54.308982    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xhnk\" (UniqueName: \"kubernetes.io/projected/3f2a9862-722a-4493-955f-9c456ecfea9c-kube-api-access-9xhnk\") pod \"e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl\" (UID: \"3f2a9862-722a-4493-955f-9c456ecfea9c\") " pod="llm/e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl"
Jul 20 15:48:54.309077 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:48:54.309026    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f2a9862-722a-4493-955f-9c456ecfea9c-model-cache\") pod \"e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl\" (UID: \"3f2a9862-722a-4493-955f-9c456ecfea9c\") " pod="llm/e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl"
Jul 20 15:48:54.309077 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:48:54.309048    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f2a9862-722a-4493-955f-9c456ecfea9c-dshm\") pod \"e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl\" (UID: \"3f2a9862-722a-4493-955f-9c456ecfea9c\") " pod="llm/e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl"
Jul 20 15:48:54.309276 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:48:54.309086    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f2a9862-722a-4493-955f-9c456ecfea9c-tls-certs\") pod \"e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl\" (UID: \"3f2a9862-722a-4493-955f-9c456ecfea9c\") " pod="llm/e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl"
Jul 20 15:48:54.309419 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:48:54.309395    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f2a9862-722a-4493-955f-9c456ecfea9c-tmp-dir\") pod \"e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl\" (UID: \"3f2a9862-722a-4493-955f-9c456ecfea9c\") " pod="llm/e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl"
Jul 20 15:48:54.309479 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:48:54.309420    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f2a9862-722a-4493-955f-9c456ecfea9c-home\") pod \"e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl\" (UID: \"3f2a9862-722a-4493-955f-9c456ecfea9c\") " pod="llm/e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl"
Jul 20 15:48:54.309479 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:48:54.309449    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f2a9862-722a-4493-955f-9c456ecfea9c-model-cache\") pod \"e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl\" (UID: \"3f2a9862-722a-4493-955f-9c456ecfea9c\") " pod="llm/e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl"
Jul 20 15:48:54.311353 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:48:54.311334    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f2a9862-722a-4493-955f-9c456ecfea9c-dshm\") pod \"e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl\" (UID: \"3f2a9862-722a-4493-955f-9c456ecfea9c\") " pod="llm/e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl"
Jul 20 15:48:54.311607 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:48:54.311592    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f2a9862-722a-4493-955f-9c456ecfea9c-tls-certs\") pod \"e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl\" (UID: \"3f2a9862-722a-4493-955f-9c456ecfea9c\") " pod="llm/e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl"
Jul 20 15:48:54.316709 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:48:54.316685    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xhnk\" (UniqueName: \"kubernetes.io/projected/3f2a9862-722a-4493-955f-9c456ecfea9c-kube-api-access-9xhnk\") pod \"e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl\" (UID: \"3f2a9862-722a-4493-955f-9c456ecfea9c\") " pod="llm/e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl"
Jul 20 15:48:54.417141 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:48:54.417057    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl"
Jul 20 15:48:54.544760 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:48:54.544739    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl"]
Jul 20 15:48:54.547236 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:48:54.547139    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3f2a9862_722a_4493_955f_9c456ecfea9c.slice/crio-f1a4668c99a170199deccbbb7f34dafe9ad5d5933c1d93622ea21b550a0eef1d WatchSource:0}: Error finding container f1a4668c99a170199deccbbb7f34dafe9ad5d5933c1d93622ea21b550a0eef1d: Status 404 returned error can't find the container with id f1a4668c99a170199deccbbb7f34dafe9ad5d5933c1d93622ea21b550a0eef1d
Jul 20 15:48:55.076671 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:48:55.076640    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl" event={"ID":"3f2a9862-722a-4493-955f-9c456ecfea9c","Type":"ContainerStarted","Data":"bae499e704c55644d3e25fe04fd0479885484a2439e23282de4a38e04bb4711a"}
Jul 20 15:48:55.076828 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:48:55.076680    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl" event={"ID":"3f2a9862-722a-4493-955f-9c456ecfea9c","Type":"ContainerStarted","Data":"f1a4668c99a170199deccbbb7f34dafe9ad5d5933c1d93622ea21b550a0eef1d"}
Jul 20 15:48:55.093596 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:48:55.093541    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl" podStartSLOduration=1.093527035 podStartE2EDuration="1.093527035s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:48:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:48:55.092319854 +0000 UTC m=+1318.101941514" watchObservedRunningTime="2026-07-20 15:48:55.093527035 +0000 UTC m=+1318.103148972"
Jul 20 15:49:01.185907 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:01.185877    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4"]
Jul 20 15:49:01.189447 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:01.189427    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4"
Jul 20 15:49:01.191882 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:01.191864    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-759851f5-kserve-self-signed-certs\""
Jul 20 15:49:01.203550 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:01.203530    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4"]
Jul 20 15:49:01.267970 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:01.267940    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18e7006e-721d-4b4c-8de0-1d5de279c6c0-tmp-dir\") pod \"e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4\" (UID: \"18e7006e-721d-4b4c-8de0-1d5de279c6c0\") " pod="llm/e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4"
Jul 20 15:49:01.268136 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:01.267992    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18e7006e-721d-4b4c-8de0-1d5de279c6c0-home\") pod \"e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4\" (UID: \"18e7006e-721d-4b4c-8de0-1d5de279c6c0\") " pod="llm/e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4"
Jul 20 15:49:01.268136 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:01.268060    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18e7006e-721d-4b4c-8de0-1d5de279c6c0-dshm\") pod \"e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4\" (UID: \"18e7006e-721d-4b4c-8de0-1d5de279c6c0\") " pod="llm/e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4"
Jul 20 15:49:01.268221 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:01.268149    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mp2l4\" (UniqueName: \"kubernetes.io/projected/18e7006e-721d-4b4c-8de0-1d5de279c6c0-kube-api-access-mp2l4\") pod \"e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4\" (UID: \"18e7006e-721d-4b4c-8de0-1d5de279c6c0\") " pod="llm/e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4"
Jul 20 15:49:01.268221 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:01.268194    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18e7006e-721d-4b4c-8de0-1d5de279c6c0-tls-certs\") pod \"e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4\" (UID: \"18e7006e-721d-4b4c-8de0-1d5de279c6c0\") " pod="llm/e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4"
Jul 20 15:49:01.268342 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:01.268260    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18e7006e-721d-4b4c-8de0-1d5de279c6c0-model-cache\") pod \"e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4\" (UID: \"18e7006e-721d-4b4c-8de0-1d5de279c6c0\") " pod="llm/e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4"
Jul 20 15:49:01.278849 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:01.278823    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:49:01.369134 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:01.369100    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18e7006e-721d-4b4c-8de0-1d5de279c6c0-home\") pod \"e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4\" (UID: \"18e7006e-721d-4b4c-8de0-1d5de279c6c0\") " pod="llm/e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4"
Jul 20 15:49:01.369134 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:01.369128    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18e7006e-721d-4b4c-8de0-1d5de279c6c0-dshm\") pod \"e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4\" (UID: \"18e7006e-721d-4b4c-8de0-1d5de279c6c0\") " pod="llm/e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4"
Jul 20 15:49:01.369353 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:01.369166    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mp2l4\" (UniqueName: \"kubernetes.io/projected/18e7006e-721d-4b4c-8de0-1d5de279c6c0-kube-api-access-mp2l4\") pod \"e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4\" (UID: \"18e7006e-721d-4b4c-8de0-1d5de279c6c0\") " pod="llm/e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4"
Jul 20 15:49:01.369353 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:01.369207    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18e7006e-721d-4b4c-8de0-1d5de279c6c0-tls-certs\") pod \"e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4\" (UID: \"18e7006e-721d-4b4c-8de0-1d5de279c6c0\") " pod="llm/e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4"
Jul 20 15:49:01.369353 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:01.369251    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18e7006e-721d-4b4c-8de0-1d5de279c6c0-model-cache\") pod \"e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4\" (UID: \"18e7006e-721d-4b4c-8de0-1d5de279c6c0\") " pod="llm/e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4"
Jul 20 15:49:01.369353 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:01.369280    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18e7006e-721d-4b4c-8de0-1d5de279c6c0-tmp-dir\") pod \"e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4\" (UID: \"18e7006e-721d-4b4c-8de0-1d5de279c6c0\") " pod="llm/e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4"
Jul 20 15:49:01.369560 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:01.369540    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18e7006e-721d-4b4c-8de0-1d5de279c6c0-home\") pod \"e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4\" (UID: \"18e7006e-721d-4b4c-8de0-1d5de279c6c0\") " pod="llm/e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4"
Jul 20 15:49:01.369674 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:01.369652    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18e7006e-721d-4b4c-8de0-1d5de279c6c0-model-cache\") pod \"e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4\" (UID: \"18e7006e-721d-4b4c-8de0-1d5de279c6c0\") " pod="llm/e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4"
Jul 20 15:49:01.369787 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:01.369772    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18e7006e-721d-4b4c-8de0-1d5de279c6c0-tmp-dir\") pod \"e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4\" (UID: \"18e7006e-721d-4b4c-8de0-1d5de279c6c0\") " pod="llm/e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4"
Jul 20 15:49:01.371443 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:01.371422    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18e7006e-721d-4b4c-8de0-1d5de279c6c0-dshm\") pod \"e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4\" (UID: \"18e7006e-721d-4b4c-8de0-1d5de279c6c0\") " pod="llm/e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4"
Jul 20 15:49:01.371668 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:01.371648    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18e7006e-721d-4b4c-8de0-1d5de279c6c0-tls-certs\") pod \"e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4\" (UID: \"18e7006e-721d-4b4c-8de0-1d5de279c6c0\") " pod="llm/e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4"
Jul 20 15:49:01.377685 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:01.377667    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mp2l4\" (UniqueName: \"kubernetes.io/projected/18e7006e-721d-4b4c-8de0-1d5de279c6c0-kube-api-access-mp2l4\") pod \"e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4\" (UID: \"18e7006e-721d-4b4c-8de0-1d5de279c6c0\") " pod="llm/e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4"
Jul 20 15:49:01.498440 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:01.498382    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4"
Jul 20 15:49:01.625879 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:01.625851    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4"]
Jul 20 15:49:01.629239 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:49:01.629193    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod18e7006e_721d_4b4c_8de0_1d5de279c6c0.slice/crio-e3e9ad8dd0e28129751b0e0c0ca6e15aabb01aee47252e9cf6b41ccaa7ed9aac WatchSource:0}: Error finding container e3e9ad8dd0e28129751b0e0c0ca6e15aabb01aee47252e9cf6b41ccaa7ed9aac: Status 404 returned error can't find the container with id e3e9ad8dd0e28129751b0e0c0ca6e15aabb01aee47252e9cf6b41ccaa7ed9aac
Jul 20 15:49:02.108538 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:02.108504    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4" event={"ID":"18e7006e-721d-4b4c-8de0-1d5de279c6c0","Type":"ContainerStarted","Data":"2accdcffacee1fbeb72179c80f6dcc3ec3b3d4be7338479c4ad418c97a53ee22"}
Jul 20 15:49:02.108704 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:02.108548    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4" event={"ID":"18e7006e-721d-4b4c-8de0-1d5de279c6c0","Type":"ContainerStarted","Data":"e3e9ad8dd0e28129751b0e0c0ca6e15aabb01aee47252e9cf6b41ccaa7ed9aac"}
Jul 20 15:49:02.126617 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:02.126572    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4" podStartSLOduration=1.126557413 podStartE2EDuration="1.126557413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:49:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:49:02.124880335 +0000 UTC m=+1325.134501997" watchObservedRunningTime="2026-07-20 15:49:02.126557413 +0000 UTC m=+1325.136179074"
Jul 20 15:49:04.417892 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:04.417812    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl"
Jul 20 15:49:04.417892 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:04.417859    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl"
Jul 20 15:49:04.430241 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:04.430218    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl"
Jul 20 15:49:05.131954 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:05.131923    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl"
Jul 20 15:49:11.498945 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:11.498910    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4"
Jul 20 15:49:11.499363 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:11.498959    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4"
Jul 20 15:49:11.518716 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:11.518692    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4"
Jul 20 15:49:12.164336 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:12.164280    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4"
Jul 20 15:49:18.823149 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:18.823112    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl"]
Jul 20 15:49:18.906688 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:18.906617    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:49:19.787855 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:19.787819    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4"]
Jul 20 15:49:19.807758 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:19.807721    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:49:33.844345 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:33.844266    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl" podUID="3f2a9862-722a-4493-955f-9c456ecfea9c" containerName="main" containerID="cri-o://bae499e704c55644d3e25fe04fd0479885484a2439e23282de4a38e04bb4711a" gracePeriod=45
Jul 20 15:49:34.083137 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:34.083111    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl"
Jul 20 15:49:34.153201 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:34.153115    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f2a9862-722a-4493-955f-9c456ecfea9c-home\") pod \"3f2a9862-722a-4493-955f-9c456ecfea9c\" (UID: \"3f2a9862-722a-4493-955f-9c456ecfea9c\") "
Jul 20 15:49:34.153201 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:34.153157    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f2a9862-722a-4493-955f-9c456ecfea9c-model-cache\") pod \"3f2a9862-722a-4493-955f-9c456ecfea9c\" (UID: \"3f2a9862-722a-4493-955f-9c456ecfea9c\") "
Jul 20 15:49:34.153452 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:34.153202    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f2a9862-722a-4493-955f-9c456ecfea9c-tmp-dir\") pod \"3f2a9862-722a-4493-955f-9c456ecfea9c\" (UID: \"3f2a9862-722a-4493-955f-9c456ecfea9c\") "
Jul 20 15:49:34.153452 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:34.153321    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xhnk\" (UniqueName: \"kubernetes.io/projected/3f2a9862-722a-4493-955f-9c456ecfea9c-kube-api-access-9xhnk\") pod \"3f2a9862-722a-4493-955f-9c456ecfea9c\" (UID: \"3f2a9862-722a-4493-955f-9c456ecfea9c\") "
Jul 20 15:49:34.153452 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:34.153344    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f2a9862-722a-4493-955f-9c456ecfea9c-dshm\") pod \"3f2a9862-722a-4493-955f-9c456ecfea9c\" (UID: \"3f2a9862-722a-4493-955f-9c456ecfea9c\") "
Jul 20 15:49:34.153612 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:34.153435    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f2a9862-722a-4493-955f-9c456ecfea9c-home" (OuterVolumeSpecName: "home") pod "3f2a9862-722a-4493-955f-9c456ecfea9c" (UID: "3f2a9862-722a-4493-955f-9c456ecfea9c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 15:49:34.153612 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:34.153505    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f2a9862-722a-4493-955f-9c456ecfea9c-tls-certs\") pod \"3f2a9862-722a-4493-955f-9c456ecfea9c\" (UID: \"3f2a9862-722a-4493-955f-9c456ecfea9c\") "
Jul 20 15:49:34.153612 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:34.153510    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f2a9862-722a-4493-955f-9c456ecfea9c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3f2a9862-722a-4493-955f-9c456ecfea9c" (UID: "3f2a9862-722a-4493-955f-9c456ecfea9c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 15:49:34.153612 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:34.153523    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f2a9862-722a-4493-955f-9c456ecfea9c-model-cache" (OuterVolumeSpecName: "model-cache") pod "3f2a9862-722a-4493-955f-9c456ecfea9c" (UID: "3f2a9862-722a-4493-955f-9c456ecfea9c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 15:49:34.153848 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:34.153822    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f2a9862-722a-4493-955f-9c456ecfea9c-home\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:49:34.153848 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:34.153841    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f2a9862-722a-4493-955f-9c456ecfea9c-model-cache\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:49:34.154072 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:34.153855    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f2a9862-722a-4493-955f-9c456ecfea9c-tmp-dir\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:49:34.155495 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:34.155469    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f2a9862-722a-4493-955f-9c456ecfea9c-dshm" (OuterVolumeSpecName: "dshm") pod "3f2a9862-722a-4493-955f-9c456ecfea9c" (UID: "3f2a9862-722a-4493-955f-9c456ecfea9c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 15:49:34.155579 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:34.155568    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f2a9862-722a-4493-955f-9c456ecfea9c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3f2a9862-722a-4493-955f-9c456ecfea9c" (UID: "3f2a9862-722a-4493-955f-9c456ecfea9c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 15:49:34.155619 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:34.155591    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f2a9862-722a-4493-955f-9c456ecfea9c-kube-api-access-9xhnk" (OuterVolumeSpecName: "kube-api-access-9xhnk") pod "3f2a9862-722a-4493-955f-9c456ecfea9c" (UID: "3f2a9862-722a-4493-955f-9c456ecfea9c"). InnerVolumeSpecName "kube-api-access-9xhnk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 15:49:34.249774 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:34.249739    2559 generic.go:356] "Generic (PLEG): container finished" podID="3f2a9862-722a-4493-955f-9c456ecfea9c" containerID="bae499e704c55644d3e25fe04fd0479885484a2439e23282de4a38e04bb4711a" exitCode=0
Jul 20 15:49:34.249956 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:34.249788    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl"
Jul 20 15:49:34.249956 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:34.249822    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl" event={"ID":"3f2a9862-722a-4493-955f-9c456ecfea9c","Type":"ContainerDied","Data":"bae499e704c55644d3e25fe04fd0479885484a2439e23282de4a38e04bb4711a"}
Jul 20 15:49:34.249956 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:34.249873    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl" event={"ID":"3f2a9862-722a-4493-955f-9c456ecfea9c","Type":"ContainerDied","Data":"f1a4668c99a170199deccbbb7f34dafe9ad5d5933c1d93622ea21b550a0eef1d"}
Jul 20 15:49:34.249956 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:34.249896    2559 scope.go:117] "RemoveContainer" containerID="bae499e704c55644d3e25fe04fd0479885484a2439e23282de4a38e04bb4711a"
Jul 20 15:49:34.254484 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:34.254463    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xhnk\" (UniqueName: \"kubernetes.io/projected/3f2a9862-722a-4493-955f-9c456ecfea9c-kube-api-access-9xhnk\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:49:34.254580 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:34.254487    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f2a9862-722a-4493-955f-9c456ecfea9c-dshm\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:49:34.254580 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:34.254501    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f2a9862-722a-4493-955f-9c456ecfea9c-tls-certs\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:49:34.258195 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:34.258180    2559 scope.go:117] "RemoveContainer" containerID="bae499e704c55644d3e25fe04fd0479885484a2439e23282de4a38e04bb4711a"
Jul 20 15:49:34.258467 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:49:34.258454    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bae499e704c55644d3e25fe04fd0479885484a2439e23282de4a38e04bb4711a\": container with ID starting with bae499e704c55644d3e25fe04fd0479885484a2439e23282de4a38e04bb4711a not found: ID does not exist" containerID="bae499e704c55644d3e25fe04fd0479885484a2439e23282de4a38e04bb4711a"
Jul 20 15:49:34.258523 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:34.258475    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bae499e704c55644d3e25fe04fd0479885484a2439e23282de4a38e04bb4711a"} err="failed to get container status \"bae499e704c55644d3e25fe04fd0479885484a2439e23282de4a38e04bb4711a\": rpc error: code = NotFound desc = could not find container \"bae499e704c55644d3e25fe04fd0479885484a2439e23282de4a38e04bb4711a\": container with ID starting with bae499e704c55644d3e25fe04fd0479885484a2439e23282de4a38e04bb4711a not found: ID does not exist"
Jul 20 15:49:34.269638 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:34.269618    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl"]
Jul 20 15:49:34.276015 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:34.275994    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-759851f5-kserve-54c996c8c9-xmtzl"]
Jul 20 15:49:34.807544 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:34.807505    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4" podUID="18e7006e-721d-4b4c-8de0-1d5de279c6c0" containerName="main" containerID="cri-o://2accdcffacee1fbeb72179c80f6dcc3ec3b3d4be7338479c4ad418c97a53ee22" gracePeriod=45
Jul 20 15:49:35.055590 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:35.055569    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4"
Jul 20 15:49:35.161590 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:35.161516    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18e7006e-721d-4b4c-8de0-1d5de279c6c0-tls-certs\") pod \"18e7006e-721d-4b4c-8de0-1d5de279c6c0\" (UID: \"18e7006e-721d-4b4c-8de0-1d5de279c6c0\") "
Jul 20 15:49:35.161590 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:35.161549    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18e7006e-721d-4b4c-8de0-1d5de279c6c0-model-cache\") pod \"18e7006e-721d-4b4c-8de0-1d5de279c6c0\" (UID: \"18e7006e-721d-4b4c-8de0-1d5de279c6c0\") "
Jul 20 15:49:35.161590 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:35.161584    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mp2l4\" (UniqueName: \"kubernetes.io/projected/18e7006e-721d-4b4c-8de0-1d5de279c6c0-kube-api-access-mp2l4\") pod \"18e7006e-721d-4b4c-8de0-1d5de279c6c0\" (UID: \"18e7006e-721d-4b4c-8de0-1d5de279c6c0\") "
Jul 20 15:49:35.161857 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:35.161641    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18e7006e-721d-4b4c-8de0-1d5de279c6c0-home\") pod \"18e7006e-721d-4b4c-8de0-1d5de279c6c0\" (UID: \"18e7006e-721d-4b4c-8de0-1d5de279c6c0\") "
Jul 20 15:49:35.161857 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:35.161683    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18e7006e-721d-4b4c-8de0-1d5de279c6c0-dshm\") pod \"18e7006e-721d-4b4c-8de0-1d5de279c6c0\" (UID: \"18e7006e-721d-4b4c-8de0-1d5de279c6c0\") "
Jul 20 15:49:35.161857 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:35.161713    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18e7006e-721d-4b4c-8de0-1d5de279c6c0-tmp-dir\") pod \"18e7006e-721d-4b4c-8de0-1d5de279c6c0\" (UID: \"18e7006e-721d-4b4c-8de0-1d5de279c6c0\") "
Jul 20 15:49:35.161857 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:35.161826    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18e7006e-721d-4b4c-8de0-1d5de279c6c0-model-cache" (OuterVolumeSpecName: "model-cache") pod "18e7006e-721d-4b4c-8de0-1d5de279c6c0" (UID: "18e7006e-721d-4b4c-8de0-1d5de279c6c0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 15:49:35.162048 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:35.161930    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18e7006e-721d-4b4c-8de0-1d5de279c6c0-home" (OuterVolumeSpecName: "home") pod "18e7006e-721d-4b4c-8de0-1d5de279c6c0" (UID: "18e7006e-721d-4b4c-8de0-1d5de279c6c0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 15:49:35.162048 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:35.161950    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18e7006e-721d-4b4c-8de0-1d5de279c6c0-model-cache\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:49:35.162048 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:35.162014    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18e7006e-721d-4b4c-8de0-1d5de279c6c0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "18e7006e-721d-4b4c-8de0-1d5de279c6c0" (UID: "18e7006e-721d-4b4c-8de0-1d5de279c6c0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 15:49:35.163660 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:35.163636    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18e7006e-721d-4b4c-8de0-1d5de279c6c0-dshm" (OuterVolumeSpecName: "dshm") pod "18e7006e-721d-4b4c-8de0-1d5de279c6c0" (UID: "18e7006e-721d-4b4c-8de0-1d5de279c6c0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 15:49:35.163760 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:35.163685    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18e7006e-721d-4b4c-8de0-1d5de279c6c0-kube-api-access-mp2l4" (OuterVolumeSpecName: "kube-api-access-mp2l4") pod "18e7006e-721d-4b4c-8de0-1d5de279c6c0" (UID: "18e7006e-721d-4b4c-8de0-1d5de279c6c0"). InnerVolumeSpecName "kube-api-access-mp2l4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 15:49:35.163760 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:35.163720    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18e7006e-721d-4b4c-8de0-1d5de279c6c0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "18e7006e-721d-4b4c-8de0-1d5de279c6c0" (UID: "18e7006e-721d-4b4c-8de0-1d5de279c6c0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 15:49:35.255748 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:35.255717    2559 generic.go:356] "Generic (PLEG): container finished" podID="18e7006e-721d-4b4c-8de0-1d5de279c6c0" containerID="2accdcffacee1fbeb72179c80f6dcc3ec3b3d4be7338479c4ad418c97a53ee22" exitCode=0
Jul 20 15:49:35.255873 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:35.255760    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4"
Jul 20 15:49:35.255873 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:35.255767    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4" event={"ID":"18e7006e-721d-4b4c-8de0-1d5de279c6c0","Type":"ContainerDied","Data":"2accdcffacee1fbeb72179c80f6dcc3ec3b3d4be7338479c4ad418c97a53ee22"}
Jul 20 15:49:35.255873 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:35.255802    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4" event={"ID":"18e7006e-721d-4b4c-8de0-1d5de279c6c0","Type":"ContainerDied","Data":"e3e9ad8dd0e28129751b0e0c0ca6e15aabb01aee47252e9cf6b41ccaa7ed9aac"}
Jul 20 15:49:35.255873 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:35.255821    2559 scope.go:117] "RemoveContainer" containerID="2accdcffacee1fbeb72179c80f6dcc3ec3b3d4be7338479c4ad418c97a53ee22"
Jul 20 15:49:35.263195 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:35.263129    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18e7006e-721d-4b4c-8de0-1d5de279c6c0-tls-certs\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:49:35.263195 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:35.263152    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mp2l4\" (UniqueName: \"kubernetes.io/projected/18e7006e-721d-4b4c-8de0-1d5de279c6c0-kube-api-access-mp2l4\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:49:35.263195 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:35.263169    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18e7006e-721d-4b4c-8de0-1d5de279c6c0-home\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:49:35.263195 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:35.263182    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18e7006e-721d-4b4c-8de0-1d5de279c6c0-dshm\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:49:35.263195 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:35.263195    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18e7006e-721d-4b4c-8de0-1d5de279c6c0-tmp-dir\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:49:35.264008 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:35.263994    2559 scope.go:117] "RemoveContainer" containerID="2accdcffacee1fbeb72179c80f6dcc3ec3b3d4be7338479c4ad418c97a53ee22"
Jul 20 15:49:35.264225 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:49:35.264210    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2accdcffacee1fbeb72179c80f6dcc3ec3b3d4be7338479c4ad418c97a53ee22\": container with ID starting with 2accdcffacee1fbeb72179c80f6dcc3ec3b3d4be7338479c4ad418c97a53ee22 not found: ID does not exist" containerID="2accdcffacee1fbeb72179c80f6dcc3ec3b3d4be7338479c4ad418c97a53ee22"
Jul 20 15:49:35.264260 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:35.264233    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2accdcffacee1fbeb72179c80f6dcc3ec3b3d4be7338479c4ad418c97a53ee22"} err="failed to get container status \"2accdcffacee1fbeb72179c80f6dcc3ec3b3d4be7338479c4ad418c97a53ee22\": rpc error: code = NotFound desc = could not find container \"2accdcffacee1fbeb72179c80f6dcc3ec3b3d4be7338479c4ad418c97a53ee22\": container with ID starting with 2accdcffacee1fbeb72179c80f6dcc3ec3b3d4be7338479c4ad418c97a53ee22 not found: ID does not exist"
Jul 20 15:49:35.276940 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:35.276922    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4"]
Jul 20 15:49:35.280692 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:35.280670    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-759851f5-kserve-cd9f9d49f-zrqt4"]
Jul 20 15:49:35.513674 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:35.513604    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18e7006e-721d-4b4c-8de0-1d5de279c6c0" path="/var/lib/kubelet/pods/18e7006e-721d-4b4c-8de0-1d5de279c6c0/volumes"
Jul 20 15:49:35.514001 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:49:35.513987    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f2a9862-722a-4493-955f-9c456ecfea9c" path="/var/lib/kubelet/pods/3f2a9862-722a-4493-955f-9c456ecfea9c/volumes"
Jul 20 15:50:26.868221 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:50:26.868184    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:50:26.967941 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:50:26.967908    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:50:51.350078 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:50:51.349998    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:01.421243 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:51:01.421200    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:51:06.955614 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:51:06.955577    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:27.302050 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:27.302011    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:29.321738 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:29.321700    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:33.749944 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:33.749916    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:33.827979 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:33.827953    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:33.994169 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:33.994132    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:34.294814 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:34.294780    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:34.517341 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:34.517309    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:36.484099 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:36.484060    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:36.676031 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:36.675997    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a1d4549a-7f56d65654-kx4wr"]
Jul 20 15:52:36.676393 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:36.676379    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f2a9862-722a-4493-955f-9c456ecfea9c" containerName="main"
Jul 20 15:52:36.676393 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:36.676394    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f2a9862-722a-4493-955f-9c456ecfea9c" containerName="main"
Jul 20 15:52:36.676476 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:36.676403    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18e7006e-721d-4b4c-8de0-1d5de279c6c0" containerName="main"
Jul 20 15:52:36.676476 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:36.676408    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="18e7006e-721d-4b4c-8de0-1d5de279c6c0" containerName="main"
Jul 20 15:52:36.676535 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:36.676483    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="18e7006e-721d-4b4c-8de0-1d5de279c6c0" containerName="main"
Jul 20 15:52:36.676535 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:36.676493    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f2a9862-722a-4493-955f-9c456ecfea9c" containerName="main"
Jul 20 15:52:36.679640 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:36.679622    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d4549a-7f56d65654-kx4wr"
Jul 20 15:52:36.682550 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:36.682527    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:36.682745 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:36.682729    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-a1d4549a-serving-cert\""
Jul 20 15:52:36.682962 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:36.682949    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-96kqc\""
Jul 20 15:52:36.699534 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:36.699512    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a1d4549a-7f56d65654-kx4wr"]
Jul 20 15:52:36.764051 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:36.764031    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87b04583-6540-4cdf-990b-30b6ed8ce5d3-maas-api-tls\") pod \"maas-api-e2e-ait-a1d4549a-7f56d65654-kx4wr\" (UID: \"87b04583-6540-4cdf-990b-30b6ed8ce5d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d4549a-7f56d65654-kx4wr"
Jul 20 15:52:36.764215 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:36.764065    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpnhv\" (UniqueName: \"kubernetes.io/projected/87b04583-6540-4cdf-990b-30b6ed8ce5d3-kube-api-access-qpnhv\") pod \"maas-api-e2e-ait-a1d4549a-7f56d65654-kx4wr\" (UID: \"87b04583-6540-4cdf-990b-30b6ed8ce5d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d4549a-7f56d65654-kx4wr"
Jul 20 15:52:36.764215 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:36.764088    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87b04583-6540-4cdf-990b-30b6ed8ce5d3-openshift-service-ca\") pod \"maas-api-e2e-ait-a1d4549a-7f56d65654-kx4wr\" (UID: \"87b04583-6540-4cdf-990b-30b6ed8ce5d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d4549a-7f56d65654-kx4wr"
Jul 20 15:52:36.833957 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:36.833925    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:36.865029 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:36.865004    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpnhv\" (UniqueName: \"kubernetes.io/projected/87b04583-6540-4cdf-990b-30b6ed8ce5d3-kube-api-access-qpnhv\") pod \"maas-api-e2e-ait-a1d4549a-7f56d65654-kx4wr\" (UID: \"87b04583-6540-4cdf-990b-30b6ed8ce5d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d4549a-7f56d65654-kx4wr"
Jul 20 15:52:36.865115 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:36.865039    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87b04583-6540-4cdf-990b-30b6ed8ce5d3-openshift-service-ca\") pod \"maas-api-e2e-ait-a1d4549a-7f56d65654-kx4wr\" (UID: \"87b04583-6540-4cdf-990b-30b6ed8ce5d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d4549a-7f56d65654-kx4wr"
Jul 20 15:52:36.865166 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:36.865149    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87b04583-6540-4cdf-990b-30b6ed8ce5d3-maas-api-tls\") pod \"maas-api-e2e-ait-a1d4549a-7f56d65654-kx4wr\" (UID: \"87b04583-6540-4cdf-990b-30b6ed8ce5d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d4549a-7f56d65654-kx4wr"
Jul 20 15:52:36.865675 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:36.865657    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87b04583-6540-4cdf-990b-30b6ed8ce5d3-openshift-service-ca\") pod \"maas-api-e2e-ait-a1d4549a-7f56d65654-kx4wr\" (UID: \"87b04583-6540-4cdf-990b-30b6ed8ce5d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d4549a-7f56d65654-kx4wr"
Jul 20 15:52:36.867544 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:36.867518    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87b04583-6540-4cdf-990b-30b6ed8ce5d3-maas-api-tls\") pod \"maas-api-e2e-ait-a1d4549a-7f56d65654-kx4wr\" (UID: \"87b04583-6540-4cdf-990b-30b6ed8ce5d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d4549a-7f56d65654-kx4wr"
Jul 20 15:52:36.875630 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:36.875604    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpnhv\" (UniqueName: \"kubernetes.io/projected/87b04583-6540-4cdf-990b-30b6ed8ce5d3-kube-api-access-qpnhv\") pod \"maas-api-e2e-ait-a1d4549a-7f56d65654-kx4wr\" (UID: \"87b04583-6540-4cdf-990b-30b6ed8ce5d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d4549a-7f56d65654-kx4wr"
Jul 20 15:52:36.959269 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:36.959245    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:36.989644 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:36.989621    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d4549a-7f56d65654-kx4wr"
Jul 20 15:52:37.122272 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:37.122250    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a1d4549a-7f56d65654-kx4wr"]
Jul 20 15:52:37.123818 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:52:37.123788    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87b04583_6540_4cdf_990b_30b6ed8ce5d3.slice/crio-33ad42f645ae260916f8aaf9e33d7794f0b6fc26a8e287d20da84d56246aa393 WatchSource:0}: Error finding container 33ad42f645ae260916f8aaf9e33d7794f0b6fc26a8e287d20da84d56246aa393: Status 404 returned error can't find the container with id 33ad42f645ae260916f8aaf9e33d7794f0b6fc26a8e287d20da84d56246aa393
Jul 20 15:52:37.125147 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:37.125131    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 15:52:37.255754 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:37.255721    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:37.364518 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:37.364447    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:52:38.056986 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:38.056950    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d4549a-7f56d65654-kx4wr" event={"ID":"87b04583-6540-4cdf-990b-30b6ed8ce5d3","Type":"ContainerStarted","Data":"24422c6890855ae369dd6907f826f5209e34a33d01c8008b9922256caab78a9e"}
Jul 20 15:52:38.057376 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:38.057018    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d4549a-7f56d65654-kx4wr" event={"ID":"87b04583-6540-4cdf-990b-30b6ed8ce5d3","Type":"ContainerStarted","Data":"33ad42f645ae260916f8aaf9e33d7794f0b6fc26a8e287d20da84d56246aa393"}
Jul 20 15:52:38.057376 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:38.057129    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d4549a-7f56d65654-kx4wr"
Jul 20 15:52:38.092584 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:38.092539    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d4549a-7f56d65654-kx4wr" podStartSLOduration=1.33376148 podStartE2EDuration="2.092525599s" totalImagesPullingTime="758.764119ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:52:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:52:38.090982646 +0000 UTC m=+1541.100604307" watchObservedRunningTime="2026-07-20 15:52:38.092525599 +0000 UTC m=+1541.102147261"
Jul 20 15:52:44.067661 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:52:44.067634    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d4549a-7f56d65654-kx4wr"
Jul 20 15:53:12.604854 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:12.604810    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a1d4549a-7f56d65654-kx4wr"]
Jul 20 15:53:12.605471 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:12.605067    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d4549a-7f56d65654-kx4wr" podUID="87b04583-6540-4cdf-990b-30b6ed8ce5d3" containerName="maas-api" containerID="cri-o://24422c6890855ae369dd6907f826f5209e34a33d01c8008b9922256caab78a9e" gracePeriod=30
Jul 20 15:53:12.638889 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:12.638865    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:53:12.847871 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:12.847846    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d4549a-7f56d65654-kx4wr"
Jul 20 15:53:12.956176 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:12.956105    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87b04583-6540-4cdf-990b-30b6ed8ce5d3-maas-api-tls\") pod \"87b04583-6540-4cdf-990b-30b6ed8ce5d3\" (UID: \"87b04583-6540-4cdf-990b-30b6ed8ce5d3\") "
Jul 20 15:53:12.956176 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:12.956154    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qpnhv\" (UniqueName: \"kubernetes.io/projected/87b04583-6540-4cdf-990b-30b6ed8ce5d3-kube-api-access-qpnhv\") pod \"87b04583-6540-4cdf-990b-30b6ed8ce5d3\" (UID: \"87b04583-6540-4cdf-990b-30b6ed8ce5d3\") "
Jul 20 15:53:12.956422 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:12.956212    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87b04583-6540-4cdf-990b-30b6ed8ce5d3-openshift-service-ca\") pod \"87b04583-6540-4cdf-990b-30b6ed8ce5d3\" (UID: \"87b04583-6540-4cdf-990b-30b6ed8ce5d3\") "
Jul 20 15:53:12.956597 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:12.956573    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87b04583-6540-4cdf-990b-30b6ed8ce5d3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "87b04583-6540-4cdf-990b-30b6ed8ce5d3" (UID: "87b04583-6540-4cdf-990b-30b6ed8ce5d3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 15:53:12.958356 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:12.958335    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87b04583-6540-4cdf-990b-30b6ed8ce5d3-kube-api-access-qpnhv" (OuterVolumeSpecName: "kube-api-access-qpnhv") pod "87b04583-6540-4cdf-990b-30b6ed8ce5d3" (UID: "87b04583-6540-4cdf-990b-30b6ed8ce5d3"). InnerVolumeSpecName "kube-api-access-qpnhv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 15:53:12.958485 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:12.958466    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87b04583-6540-4cdf-990b-30b6ed8ce5d3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "87b04583-6540-4cdf-990b-30b6ed8ce5d3" (UID: "87b04583-6540-4cdf-990b-30b6ed8ce5d3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 15:53:13.057036 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:13.057006    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpnhv\" (UniqueName: \"kubernetes.io/projected/87b04583-6540-4cdf-990b-30b6ed8ce5d3-kube-api-access-qpnhv\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:53:13.057036 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:13.057029    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87b04583-6540-4cdf-990b-30b6ed8ce5d3-openshift-service-ca\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:53:13.057036 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:13.057039    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87b04583-6540-4cdf-990b-30b6ed8ce5d3-maas-api-tls\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:53:13.212017 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:13.211959    2559 generic.go:356] "Generic (PLEG): container finished" podID="87b04583-6540-4cdf-990b-30b6ed8ce5d3" containerID="24422c6890855ae369dd6907f826f5209e34a33d01c8008b9922256caab78a9e" exitCode=0
Jul 20 15:53:13.212017 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:13.212006    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d4549a-7f56d65654-kx4wr"
Jul 20 15:53:13.212157 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:13.212043    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d4549a-7f56d65654-kx4wr" event={"ID":"87b04583-6540-4cdf-990b-30b6ed8ce5d3","Type":"ContainerDied","Data":"24422c6890855ae369dd6907f826f5209e34a33d01c8008b9922256caab78a9e"}
Jul 20 15:53:13.212157 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:13.212098    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d4549a-7f56d65654-kx4wr" event={"ID":"87b04583-6540-4cdf-990b-30b6ed8ce5d3","Type":"ContainerDied","Data":"33ad42f645ae260916f8aaf9e33d7794f0b6fc26a8e287d20da84d56246aa393"}
Jul 20 15:53:13.212157 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:13.212126    2559 scope.go:117] "RemoveContainer" containerID="24422c6890855ae369dd6907f826f5209e34a33d01c8008b9922256caab78a9e"
Jul 20 15:53:13.220178 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:13.220163    2559 scope.go:117] "RemoveContainer" containerID="24422c6890855ae369dd6907f826f5209e34a33d01c8008b9922256caab78a9e"
Jul 20 15:53:13.220411 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:53:13.220390    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24422c6890855ae369dd6907f826f5209e34a33d01c8008b9922256caab78a9e\": container with ID starting with 24422c6890855ae369dd6907f826f5209e34a33d01c8008b9922256caab78a9e not found: ID does not exist" containerID="24422c6890855ae369dd6907f826f5209e34a33d01c8008b9922256caab78a9e"
Jul 20 15:53:13.220484 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:13.220422    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24422c6890855ae369dd6907f826f5209e34a33d01c8008b9922256caab78a9e"} err="failed to get container status \"24422c6890855ae369dd6907f826f5209e34a33d01c8008b9922256caab78a9e\": rpc error: code = NotFound desc = could not find container \"24422c6890855ae369dd6907f826f5209e34a33d01c8008b9922256caab78a9e\": container with ID starting with 24422c6890855ae369dd6907f826f5209e34a33d01c8008b9922256caab78a9e not found: ID does not exist"
Jul 20 15:53:13.236912 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:13.236860    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a1d4549a-7f56d65654-kx4wr"]
Jul 20 15:53:13.236912 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:13.236911    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a1d4549a-7f56d65654-kx4wr"]
Jul 20 15:53:13.513970 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:13.513945    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87b04583-6540-4cdf-990b-30b6ed8ce5d3" path="/var/lib/kubelet/pods/87b04583-6540-4cdf-990b-30b6ed8ce5d3/volumes"
Jul 20 15:53:14.692367 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:14.692333    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:53:14.749242 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:14.749218    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:53:17.240391 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:17.240361    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:53:17.414100 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:17.414059    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:53:17.650094 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:17.650062    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:53:17.909387 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:17.909312    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:53:18.017860 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.017826    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:53:18.170603 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.170520    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:53:18.526994 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.526952    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"]
Jul 20 15:53:18.527378 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.527347    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87b04583-6540-4cdf-990b-30b6ed8ce5d3" containerName="maas-api"
Jul 20 15:53:18.527378 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.527365    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="87b04583-6540-4cdf-990b-30b6ed8ce5d3" containerName="maas-api"
Jul 20 15:53:18.527480 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.527468    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="87b04583-6540-4cdf-990b-30b6ed8ce5d3" containerName="maas-api"
Jul 20 15:53:18.531912 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.531896    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"
Jul 20 15:53:18.535491 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.535471    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-e9c2c9fb-gw-openshift-default-dockercfg-xbvp2\""
Jul 20 15:53:18.541052 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.541036    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 20 15:53:18.549679 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.549661    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"]
Jul 20 15:53:18.558130 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.558109    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:53:18.705023 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.704988    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-istio-podinfo\") pod \"e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m\" (UID: \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\") " pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"
Jul 20 15:53:18.705159 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.705032    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-istio-token\") pod \"e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m\" (UID: \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\") " pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"
Jul 20 15:53:18.705159 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.705058    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-istiod-ca-cert\") pod \"e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m\" (UID: \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\") " pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"
Jul 20 15:53:18.705159 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.705105    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f66xt\" (UniqueName: \"kubernetes.io/projected/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-kube-api-access-f66xt\") pod \"e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m\" (UID: \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\") " pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"
Jul 20 15:53:18.705159 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.705147    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-istio-envoy\") pod \"e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m\" (UID: \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\") " pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"
Jul 20 15:53:18.705359 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.705168    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-workload-certs\") pod \"e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m\" (UID: \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\") " pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"
Jul 20 15:53:18.705359 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.705189    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-credential-socket\") pod \"e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m\" (UID: \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\") " pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"
Jul 20 15:53:18.705359 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.705273    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-workload-socket\") pod \"e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m\" (UID: \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\") " pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"
Jul 20 15:53:18.705359 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.705332    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-istio-data\") pod \"e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m\" (UID: \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\") " pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"
Jul 20 15:53:18.805712 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.805652    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-istio-podinfo\") pod \"e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m\" (UID: \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\") " pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"
Jul 20 15:53:18.805712 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.805687    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-istio-token\") pod \"e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m\" (UID: \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\") " pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"
Jul 20 15:53:18.805712 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.805703    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-istiod-ca-cert\") pod \"e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m\" (UID: \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\") " pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"
Jul 20 15:53:18.805922 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.805722    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f66xt\" (UniqueName: \"kubernetes.io/projected/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-kube-api-access-f66xt\") pod \"e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m\" (UID: \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\") " pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"
Jul 20 15:53:18.805922 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.805754    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-istio-envoy\") pod \"e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m\" (UID: \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\") " pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"
Jul 20 15:53:18.805922 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.805785    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-workload-certs\") pod \"e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m\" (UID: \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\") " pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"
Jul 20 15:53:18.805922 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.805812    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-credential-socket\") pod \"e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m\" (UID: \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\") " pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"
Jul 20 15:53:18.805922 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.805849    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-workload-socket\") pod \"e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m\" (UID: \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\") " pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"
Jul 20 15:53:18.805922 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.805882    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-istio-data\") pod \"e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m\" (UID: \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\") " pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"
Jul 20 15:53:18.806215 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.806171    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-credential-socket\") pod \"e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m\" (UID: \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\") " pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"
Jul 20 15:53:18.806215 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.806211    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-workload-certs\") pod \"e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m\" (UID: \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\") " pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"
Jul 20 15:53:18.806356 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.806282    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-istio-data\") pod \"e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m\" (UID: \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\") " pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"
Jul 20 15:53:18.806415 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.806369    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-workload-socket\") pod \"e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m\" (UID: \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\") " pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"
Jul 20 15:53:18.806474 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.806453    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-istiod-ca-cert\") pod \"e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m\" (UID: \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\") " pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"
Jul 20 15:53:18.808114 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.808095    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-istio-envoy\") pod \"e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m\" (UID: \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\") " pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"
Jul 20 15:53:18.808207 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.808191    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-istio-podinfo\") pod \"e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m\" (UID: \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\") " pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"
Jul 20 15:53:18.815592 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.815564    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f66xt\" (UniqueName: \"kubernetes.io/projected/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-kube-api-access-f66xt\") pod \"e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m\" (UID: \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\") " pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"
Jul 20 15:53:18.816865 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.816843    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-istio-token\") pod \"e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m\" (UID: \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\") " pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"
Jul 20 15:53:18.818654 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.818634    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:53:18.841252 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.841236    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"
Jul 20 15:53:18.977834 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:18.977813    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"]
Jul 20 15:53:18.980059 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:53:18.980020    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7461d38_e9d1_4c56_874c_8c4fc5e5be66.slice/crio-b3e6f457134f861122e66359d166e500ad48c40f61164b21b4833b4444d21caf WatchSource:0}: Error finding container b3e6f457134f861122e66359d166e500ad48c40f61164b21b4833b4444d21caf: Status 404 returned error can't find the container with id b3e6f457134f861122e66359d166e500ad48c40f61164b21b4833b4444d21caf
Jul 20 15:53:19.239337 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:19.239261    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m" event={"ID":"a7461d38-e9d1-4c56-874c-8c4fc5e5be66","Type":"ContainerStarted","Data":"b3e6f457134f861122e66359d166e500ad48c40f61164b21b4833b4444d21caf"}
Jul 20 15:53:22.824416 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:22.824382    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 15:53:22.824653 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:22.824450    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 15:53:22.824653 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:22.824477    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 15:53:23.263779 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:23.263693    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m" event={"ID":"a7461d38-e9d1-4c56-874c-8c4fc5e5be66","Type":"ContainerStarted","Data":"ab272bf6e6007fc4302c33d624980d0cf47f5c080429ca4a8b03c3143624a9b7"}
Jul 20 15:53:23.287835 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:23.287791    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m" podStartSLOduration=1.445663156 podStartE2EDuration="5.287777226s" totalImagesPullingTime="3.84211407s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:53:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:53:23.285234718 +0000 UTC m=+1586.294856379" watchObservedRunningTime="2026-07-20 15:53:23.287777226 +0000 UTC m=+1586.297398888"
Jul 20 15:53:23.841985 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:23.841950    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"
Jul 20 15:53:23.847172 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:23.847148    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"
Jul 20 15:53:24.268751 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:24.268722    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"
Jul 20 15:53:24.269707 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:24.269689    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"
Jul 20 15:53:24.770256 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:24.770217    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:53:24.859503 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:24.859466    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e9c2c9fb-9984585b9-vbzhb"]
Jul 20 15:53:24.863593 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:24.863573    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e9c2c9fb-9984585b9-vbzhb"
Jul 20 15:53:24.865950 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:24.865931    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-e9c2c9fb-serving-cert\""
Jul 20 15:53:24.866218 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:24.865962    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-96kqc\""
Jul 20 15:53:24.872437 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:24.872406    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e9c2c9fb-9984585b9-vbzhb"]
Jul 20 15:53:24.892483 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:24.892449    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:53:24.958442 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:24.958411    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b76506e-320b-4f7e-a2e3-969cf99547a6-maas-api-tls\") pod \"maas-api-e2e-migrate-e9c2c9fb-9984585b9-vbzhb\" (UID: \"2b76506e-320b-4f7e-a2e3-969cf99547a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e9c2c9fb-9984585b9-vbzhb"
Jul 20 15:53:24.958601 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:24.958464    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7f8c7\" (UniqueName: \"kubernetes.io/projected/2b76506e-320b-4f7e-a2e3-969cf99547a6-kube-api-access-7f8c7\") pod \"maas-api-e2e-migrate-e9c2c9fb-9984585b9-vbzhb\" (UID: \"2b76506e-320b-4f7e-a2e3-969cf99547a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e9c2c9fb-9984585b9-vbzhb"
Jul 20 15:53:24.958601 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:24.958553    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b76506e-320b-4f7e-a2e3-969cf99547a6-openshift-service-ca\") pod \"maas-api-e2e-migrate-e9c2c9fb-9984585b9-vbzhb\" (UID: \"2b76506e-320b-4f7e-a2e3-969cf99547a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e9c2c9fb-9984585b9-vbzhb"
Jul 20 15:53:25.059037 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:25.058964    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b76506e-320b-4f7e-a2e3-969cf99547a6-openshift-service-ca\") pod \"maas-api-e2e-migrate-e9c2c9fb-9984585b9-vbzhb\" (UID: \"2b76506e-320b-4f7e-a2e3-969cf99547a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e9c2c9fb-9984585b9-vbzhb"
Jul 20 15:53:25.059037 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:25.059014    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b76506e-320b-4f7e-a2e3-969cf99547a6-maas-api-tls\") pod \"maas-api-e2e-migrate-e9c2c9fb-9984585b9-vbzhb\" (UID: \"2b76506e-320b-4f7e-a2e3-969cf99547a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e9c2c9fb-9984585b9-vbzhb"
Jul 20 15:53:25.059231 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:25.059051    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7f8c7\" (UniqueName: \"kubernetes.io/projected/2b76506e-320b-4f7e-a2e3-969cf99547a6-kube-api-access-7f8c7\") pod \"maas-api-e2e-migrate-e9c2c9fb-9984585b9-vbzhb\" (UID: \"2b76506e-320b-4f7e-a2e3-969cf99547a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e9c2c9fb-9984585b9-vbzhb"
Jul 20 15:53:25.059231 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:53:25.059171    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-e9c2c9fb-serving-cert: secret "maas-api-e2e-migrate-e9c2c9fb-serving-cert" not found
Jul 20 15:53:25.059354 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:53:25.059245    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b76506e-320b-4f7e-a2e3-969cf99547a6-maas-api-tls podName:2b76506e-320b-4f7e-a2e3-969cf99547a6 nodeName:}" failed. No retries permitted until 2026-07-20 15:53:25.559226422 +0000 UTC m=+1588.568848285 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2b76506e-320b-4f7e-a2e3-969cf99547a6-maas-api-tls") pod "maas-api-e2e-migrate-e9c2c9fb-9984585b9-vbzhb" (UID: "2b76506e-320b-4f7e-a2e3-969cf99547a6") : secret "maas-api-e2e-migrate-e9c2c9fb-serving-cert" not found
Jul 20 15:53:25.059646 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:25.059629    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b76506e-320b-4f7e-a2e3-969cf99547a6-openshift-service-ca\") pod \"maas-api-e2e-migrate-e9c2c9fb-9984585b9-vbzhb\" (UID: \"2b76506e-320b-4f7e-a2e3-969cf99547a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e9c2c9fb-9984585b9-vbzhb"
Jul 20 15:53:25.071002 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:25.070978    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7f8c7\" (UniqueName: \"kubernetes.io/projected/2b76506e-320b-4f7e-a2e3-969cf99547a6-kube-api-access-7f8c7\") pod \"maas-api-e2e-migrate-e9c2c9fb-9984585b9-vbzhb\" (UID: \"2b76506e-320b-4f7e-a2e3-969cf99547a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e9c2c9fb-9984585b9-vbzhb"
Jul 20 15:53:25.318326 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:25.318233    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:53:25.418444 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:25.418410    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:53:25.518721 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:25.518693    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:53:25.563889 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:25.563857    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b76506e-320b-4f7e-a2e3-969cf99547a6-maas-api-tls\") pod \"maas-api-e2e-migrate-e9c2c9fb-9984585b9-vbzhb\" (UID: \"2b76506e-320b-4f7e-a2e3-969cf99547a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e9c2c9fb-9984585b9-vbzhb"
Jul 20 15:53:25.566215 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:25.566192    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b76506e-320b-4f7e-a2e3-969cf99547a6-maas-api-tls\") pod \"maas-api-e2e-migrate-e9c2c9fb-9984585b9-vbzhb\" (UID: \"2b76506e-320b-4f7e-a2e3-969cf99547a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e9c2c9fb-9984585b9-vbzhb"
Jul 20 15:53:25.775335 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:25.775305    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e9c2c9fb-9984585b9-vbzhb"
Jul 20 15:53:26.115034 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:26.115005    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e9c2c9fb-9984585b9-vbzhb"]
Jul 20 15:53:26.116966 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:53:26.116940    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b76506e_320b_4f7e_a2e3_969cf99547a6.slice/crio-35f1da0c427f2474ace7e6b8a59ea2ede36d3f3d2bf5fb36122516d9720da5cc WatchSource:0}: Error finding container 35f1da0c427f2474ace7e6b8a59ea2ede36d3f3d2bf5fb36122516d9720da5cc: Status 404 returned error can't find the container with id 35f1da0c427f2474ace7e6b8a59ea2ede36d3f3d2bf5fb36122516d9720da5cc
Jul 20 15:53:26.278400 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:26.278366    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e9c2c9fb-9984585b9-vbzhb" event={"ID":"2b76506e-320b-4f7e-a2e3-969cf99547a6","Type":"ContainerStarted","Data":"35f1da0c427f2474ace7e6b8a59ea2ede36d3f3d2bf5fb36122516d9720da5cc"}
Jul 20 15:53:27.283773 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:27.283733    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e9c2c9fb-9984585b9-vbzhb" event={"ID":"2b76506e-320b-4f7e-a2e3-969cf99547a6","Type":"ContainerStarted","Data":"d6b2b551ee43b3715c59b20baa514d173abd73755914e86cb11f26c8321d817a"}
Jul 20 15:53:27.284191 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:27.283802    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e9c2c9fb-9984585b9-vbzhb"
Jul 20 15:53:27.303801 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:27.303758    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e9c2c9fb-9984585b9-vbzhb" podStartSLOduration=2.908645965 podStartE2EDuration="3.303743512s" totalImagesPullingTime="395.097547ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:53:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:53:27.301860217 +0000 UTC m=+1590.311481880" watchObservedRunningTime="2026-07-20 15:53:27.303743512 +0000 UTC m=+1590.313365173"
Jul 20 15:53:33.294205 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:33.294171    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e9c2c9fb-9984585b9-vbzhb"
Jul 20 15:53:36.328263 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:36.328232    2559 generic.go:356] "Generic (PLEG): container finished" podID="a7461d38-e9d1-4c56-874c-8c4fc5e5be66" containerID="ab272bf6e6007fc4302c33d624980d0cf47f5c080429ca4a8b03c3143624a9b7" exitCode=0
Jul 20 15:53:36.328740 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:36.328320    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m" event={"ID":"a7461d38-e9d1-4c56-874c-8c4fc5e5be66","Type":"ContainerDied","Data":"ab272bf6e6007fc4302c33d624980d0cf47f5c080429ca4a8b03c3143624a9b7"}
Jul 20 15:53:36.328740 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:36.328675    2559 scope.go:117] "RemoveContainer" containerID="ab272bf6e6007fc4302c33d624980d0cf47f5c080429ca4a8b03c3143624a9b7"
Jul 20 15:53:36.329638 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:36.329611    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 15:53:36.329712 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:36.329698    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 15:53:36.329756 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:36.329732    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 15:53:37.338528 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:37.338490    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m" event={"ID":"a7461d38-e9d1-4c56-874c-8c4fc5e5be66","Type":"ContainerStarted","Data":"f5e1ecb7d7d04215481610f7cd9a6dafb00f91f0e9a918c842f8a0499b0730de"}
Jul 20 15:53:37.842762 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:37.842730    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"
Jul 20 15:53:37.843125 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:37.843101    2559 patch_prober.go:28] interesting pod/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Jul 20 15:53:37.843302 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:37.843256    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m" podUID="a7461d38-e9d1-4c56-874c-8c4fc5e5be66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Jul 20 15:53:38.842560 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:38.842521    2559 patch_prober.go:28] interesting pod/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Jul 20 15:53:38.843119 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:38.843088    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m" podUID="a7461d38-e9d1-4c56-874c-8c4fc5e5be66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Jul 20 15:53:39.268980 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:39.268947    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"
Jul 20 15:53:39.842386 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:39.842345    2559 patch_prober.go:28] interesting pod/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Jul 20 15:53:39.842673 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:39.842646    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m" podUID="a7461d38-e9d1-4c56-874c-8c4fc5e5be66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Jul 20 15:53:40.842360 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:40.842315    2559 patch_prober.go:28] interesting pod/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Jul 20 15:53:40.842665 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:40.842620    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m" podUID="a7461d38-e9d1-4c56-874c-8c4fc5e5be66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Jul 20 15:53:41.845809 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:41.845780    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"
Jul 20 15:53:41.846799 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:53:41.846773    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"
Jul 20 15:54:00.285782 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:00.285743    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e9c2c9fb-9984585b9-vbzhb"]
Jul 20 15:54:00.286203 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:00.286033    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e9c2c9fb-9984585b9-vbzhb" podUID="2b76506e-320b-4f7e-a2e3-969cf99547a6" containerName="maas-api" containerID="cri-o://d6b2b551ee43b3715c59b20baa514d173abd73755914e86cb11f26c8321d817a" gracePeriod=30
Jul 20 15:54:00.320561 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:00.320534    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:00.454715 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:00.454686    2559 generic.go:356] "Generic (PLEG): container finished" podID="2b76506e-320b-4f7e-a2e3-969cf99547a6" containerID="d6b2b551ee43b3715c59b20baa514d173abd73755914e86cb11f26c8321d817a" exitCode=0
Jul 20 15:54:00.454866 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:00.454742    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e9c2c9fb-9984585b9-vbzhb" event={"ID":"2b76506e-320b-4f7e-a2e3-969cf99547a6","Type":"ContainerDied","Data":"d6b2b551ee43b3715c59b20baa514d173abd73755914e86cb11f26c8321d817a"}
Jul 20 15:54:00.536031 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:00.535977    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e9c2c9fb-9984585b9-vbzhb"
Jul 20 15:54:00.680831 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:00.680805    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7f8c7\" (UniqueName: \"kubernetes.io/projected/2b76506e-320b-4f7e-a2e3-969cf99547a6-kube-api-access-7f8c7\") pod \"2b76506e-320b-4f7e-a2e3-969cf99547a6\" (UID: \"2b76506e-320b-4f7e-a2e3-969cf99547a6\") "
Jul 20 15:54:00.680972 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:00.680868    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b76506e-320b-4f7e-a2e3-969cf99547a6-maas-api-tls\") pod \"2b76506e-320b-4f7e-a2e3-969cf99547a6\" (UID: \"2b76506e-320b-4f7e-a2e3-969cf99547a6\") "
Jul 20 15:54:00.681013 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:00.680972    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b76506e-320b-4f7e-a2e3-969cf99547a6-openshift-service-ca\") pod \"2b76506e-320b-4f7e-a2e3-969cf99547a6\" (UID: \"2b76506e-320b-4f7e-a2e3-969cf99547a6\") "
Jul 20 15:54:00.681323 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:00.681274    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b76506e-320b-4f7e-a2e3-969cf99547a6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2b76506e-320b-4f7e-a2e3-969cf99547a6" (UID: "2b76506e-320b-4f7e-a2e3-969cf99547a6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 15:54:00.682829 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:00.682806    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b76506e-320b-4f7e-a2e3-969cf99547a6-kube-api-access-7f8c7" (OuterVolumeSpecName: "kube-api-access-7f8c7") pod "2b76506e-320b-4f7e-a2e3-969cf99547a6" (UID: "2b76506e-320b-4f7e-a2e3-969cf99547a6"). InnerVolumeSpecName "kube-api-access-7f8c7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 15:54:00.682894 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:00.682832    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b76506e-320b-4f7e-a2e3-969cf99547a6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2b76506e-320b-4f7e-a2e3-969cf99547a6" (UID: "2b76506e-320b-4f7e-a2e3-969cf99547a6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 15:54:00.782501 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:00.782474    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7f8c7\" (UniqueName: \"kubernetes.io/projected/2b76506e-320b-4f7e-a2e3-969cf99547a6-kube-api-access-7f8c7\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:54:00.782501 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:00.782501    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b76506e-320b-4f7e-a2e3-969cf99547a6-maas-api-tls\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:54:00.782672 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:00.782510    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b76506e-320b-4f7e-a2e3-969cf99547a6-openshift-service-ca\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:54:01.460425 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:01.460392    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e9c2c9fb-9984585b9-vbzhb"
Jul 20 15:54:01.460824 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:01.460390    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e9c2c9fb-9984585b9-vbzhb" event={"ID":"2b76506e-320b-4f7e-a2e3-969cf99547a6","Type":"ContainerDied","Data":"35f1da0c427f2474ace7e6b8a59ea2ede36d3f3d2bf5fb36122516d9720da5cc"}
Jul 20 15:54:01.461379 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:01.460523    2559 scope.go:117] "RemoveContainer" containerID="d6b2b551ee43b3715c59b20baa514d173abd73755914e86cb11f26c8321d817a"
Jul 20 15:54:01.481869 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:01.481846    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e9c2c9fb-9984585b9-vbzhb"]
Jul 20 15:54:01.487326 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:01.487282    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e9c2c9fb-9984585b9-vbzhb"]
Jul 20 15:54:01.514453 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:01.514433    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b76506e-320b-4f7e-a2e3-969cf99547a6" path="/var/lib/kubelet/pods/2b76506e-320b-4f7e-a2e3-969cf99547a6/volumes"
Jul 20 15:54:04.890533 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:04.890497    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:05.057711 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:05.057678    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"]
Jul 20 15:54:05.057907 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:05.057884    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m" podUID="a7461d38-e9d1-4c56-874c-8c4fc5e5be66" containerName="istio-proxy" containerID="cri-o://f5e1ecb7d7d04215481610f7cd9a6dafb00f91f0e9a918c842f8a0499b0730de" gracePeriod=30
Jul 20 15:54:05.083676 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:05.083647    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:05.305624 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:05.305594    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:05.550345 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:05.550315    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:05.866467 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:05.866435    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:06.242504 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:06.242430    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:08.055133 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:08.055091    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:08.156130 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:08.156099    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dde17afb-68f4d7478f-699qj"]
Jul 20 15:54:08.156659 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:08.156634    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b76506e-320b-4f7e-a2e3-969cf99547a6" containerName="maas-api"
Jul 20 15:54:08.156659 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:08.156659    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b76506e-320b-4f7e-a2e3-969cf99547a6" containerName="maas-api"
Jul 20 15:54:08.156854 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:08.156771    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b76506e-320b-4f7e-a2e3-969cf99547a6" containerName="maas-api"
Jul 20 15:54:08.161057 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:08.161035    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dde17afb-68f4d7478f-699qj"
Jul 20 15:54:08.163897 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:08.163880    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-dde17afb-serving-cert\""
Jul 20 15:54:08.164002 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:08.163940    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-96kqc\""
Jul 20 15:54:08.169385 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:08.169363    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:08.173203 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:08.173177    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dde17afb-68f4d7478f-699qj"]
Jul 20 15:54:08.237811 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:08.237781    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa263126-cbbe-422c-8979-1c0a7eadfe90-openshift-service-ca\") pod \"maas-api-e2e-ait-dde17afb-68f4d7478f-699qj\" (UID: \"aa263126-cbbe-422c-8979-1c0a7eadfe90\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dde17afb-68f4d7478f-699qj"
Jul 20 15:54:08.237995 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:08.237845    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa263126-cbbe-422c-8979-1c0a7eadfe90-maas-api-tls\") pod \"maas-api-e2e-ait-dde17afb-68f4d7478f-699qj\" (UID: \"aa263126-cbbe-422c-8979-1c0a7eadfe90\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dde17afb-68f4d7478f-699qj"
Jul 20 15:54:08.237995 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:08.237932    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmvxb\" (UniqueName: \"kubernetes.io/projected/aa263126-cbbe-422c-8979-1c0a7eadfe90-kube-api-access-fmvxb\") pod \"maas-api-e2e-ait-dde17afb-68f4d7478f-699qj\" (UID: \"aa263126-cbbe-422c-8979-1c0a7eadfe90\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dde17afb-68f4d7478f-699qj"
Jul 20 15:54:08.239054 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:08.239033    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:08.338884 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:08.338799    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa263126-cbbe-422c-8979-1c0a7eadfe90-maas-api-tls\") pod \"maas-api-e2e-ait-dde17afb-68f4d7478f-699qj\" (UID: \"aa263126-cbbe-422c-8979-1c0a7eadfe90\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dde17afb-68f4d7478f-699qj"
Jul 20 15:54:08.338884 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:08.338854    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmvxb\" (UniqueName: \"kubernetes.io/projected/aa263126-cbbe-422c-8979-1c0a7eadfe90-kube-api-access-fmvxb\") pod \"maas-api-e2e-ait-dde17afb-68f4d7478f-699qj\" (UID: \"aa263126-cbbe-422c-8979-1c0a7eadfe90\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dde17afb-68f4d7478f-699qj"
Jul 20 15:54:08.339064 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:08.338889    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa263126-cbbe-422c-8979-1c0a7eadfe90-openshift-service-ca\") pod \"maas-api-e2e-ait-dde17afb-68f4d7478f-699qj\" (UID: \"aa263126-cbbe-422c-8979-1c0a7eadfe90\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dde17afb-68f4d7478f-699qj"
Jul 20 15:54:08.339518 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:08.339501    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa263126-cbbe-422c-8979-1c0a7eadfe90-openshift-service-ca\") pod \"maas-api-e2e-ait-dde17afb-68f4d7478f-699qj\" (UID: \"aa263126-cbbe-422c-8979-1c0a7eadfe90\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dde17afb-68f4d7478f-699qj"
Jul 20 15:54:08.341351 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:08.341333    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa263126-cbbe-422c-8979-1c0a7eadfe90-maas-api-tls\") pod \"maas-api-e2e-ait-dde17afb-68f4d7478f-699qj\" (UID: \"aa263126-cbbe-422c-8979-1c0a7eadfe90\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dde17afb-68f4d7478f-699qj"
Jul 20 15:54:08.366181 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:08.366157    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmvxb\" (UniqueName: \"kubernetes.io/projected/aa263126-cbbe-422c-8979-1c0a7eadfe90-kube-api-access-fmvxb\") pod \"maas-api-e2e-ait-dde17afb-68f4d7478f-699qj\" (UID: \"aa263126-cbbe-422c-8979-1c0a7eadfe90\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dde17afb-68f4d7478f-699qj"
Jul 20 15:54:08.415714 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:08.415684    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:08.471137 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:08.471111    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dde17afb-68f4d7478f-699qj"
Jul 20 15:54:08.605037 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:08.603177    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dde17afb-68f4d7478f-699qj"]
Jul 20 15:54:08.606205 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:54:08.606178    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa263126_cbbe_422c_8979_1c0a7eadfe90.slice/crio-a0326f483e8e3c13886e325a82827b8f331c9d98ee9b144fbc488935f2b72797 WatchSource:0}: Error finding container a0326f483e8e3c13886e325a82827b8f331c9d98ee9b144fbc488935f2b72797: Status 404 returned error can't find the container with id a0326f483e8e3c13886e325a82827b8f331c9d98ee9b144fbc488935f2b72797
Jul 20 15:54:08.880012 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:08.879931    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:08.979054 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:08.979023    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:09.503886 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:09.503856    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dde17afb-68f4d7478f-699qj" event={"ID":"aa263126-cbbe-422c-8979-1c0a7eadfe90","Type":"ContainerStarted","Data":"95e54c2d2cdade79bb84a35e75f8f4c0d56df8a99f709d55bb6f3caf39896678"}
Jul 20 15:54:09.504249 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:09.503907    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dde17afb-68f4d7478f-699qj" event={"ID":"aa263126-cbbe-422c-8979-1c0a7eadfe90","Type":"ContainerStarted","Data":"a0326f483e8e3c13886e325a82827b8f331c9d98ee9b144fbc488935f2b72797"}
Jul 20 15:54:09.504249 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:09.503953    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dde17afb-68f4d7478f-699qj"
Jul 20 15:54:09.524911 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:09.524861    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dde17afb-68f4d7478f-699qj" podStartSLOduration=1.120509449 podStartE2EDuration="1.524846027s" totalImagesPullingTime="404.336578ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:54:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:54:09.523173583 +0000 UTC m=+1632.532795247" watchObservedRunningTime="2026-07-20 15:54:09.524846027 +0000 UTC m=+1632.534467688"
Jul 20 15:54:10.296747 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:10.296724    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"
Jul 20 15:54:10.357950 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:10.357885    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-workload-certs\") pod \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\" (UID: \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\") "
Jul 20 15:54:10.357950 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:10.357920    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-istiod-ca-cert\") pod \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\" (UID: \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\") "
Jul 20 15:54:10.357950 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:10.357937    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-istio-envoy\") pod \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\" (UID: \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\") "
Jul 20 15:54:10.358188 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:10.357955    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f66xt\" (UniqueName: \"kubernetes.io/projected/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-kube-api-access-f66xt\") pod \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\" (UID: \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\") "
Jul 20 15:54:10.358188 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:10.358001    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-istio-data\") pod \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\" (UID: \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\") "
Jul 20 15:54:10.358188 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:10.358035    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-workload-socket\") pod \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\" (UID: \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\") "
Jul 20 15:54:10.358188 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:10.358070    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-istio-podinfo\") pod \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\" (UID: \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\") "
Jul 20 15:54:10.358188 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:10.358151    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-istio-token\") pod \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\" (UID: \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\") "
Jul 20 15:54:10.358188 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:10.358179    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a7461d38-e9d1-4c56-874c-8c4fc5e5be66" (UID: "a7461d38-e9d1-4c56-874c-8c4fc5e5be66"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 15:54:10.358535 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:10.358199    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-credential-socket\") pod \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\" (UID: \"a7461d38-e9d1-4c56-874c-8c4fc5e5be66\") "
Jul 20 15:54:10.358535 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:10.358390    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a7461d38-e9d1-4c56-874c-8c4fc5e5be66" (UID: "a7461d38-e9d1-4c56-874c-8c4fc5e5be66"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 15:54:10.358535 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:10.358512    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-workload-certs\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:54:10.358684 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:10.358600    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-istiod-ca-cert\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:54:10.358722 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:10.358694    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a7461d38-e9d1-4c56-874c-8c4fc5e5be66" (UID: "a7461d38-e9d1-4c56-874c-8c4fc5e5be66"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 15:54:10.358768 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:10.358750    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a7461d38-e9d1-4c56-874c-8c4fc5e5be66" (UID: "a7461d38-e9d1-4c56-874c-8c4fc5e5be66"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 15:54:10.358945 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:10.358922    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-istio-data" (OuterVolumeSpecName: "istio-data") pod "a7461d38-e9d1-4c56-874c-8c4fc5e5be66" (UID: "a7461d38-e9d1-4c56-874c-8c4fc5e5be66"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 15:54:10.360349 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:10.360326    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-kube-api-access-f66xt" (OuterVolumeSpecName: "kube-api-access-f66xt") pod "a7461d38-e9d1-4c56-874c-8c4fc5e5be66" (UID: "a7461d38-e9d1-4c56-874c-8c4fc5e5be66"). InnerVolumeSpecName "kube-api-access-f66xt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 15:54:10.360453 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:10.360351    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-istio-token" (OuterVolumeSpecName: "istio-token") pod "a7461d38-e9d1-4c56-874c-8c4fc5e5be66" (UID: "a7461d38-e9d1-4c56-874c-8c4fc5e5be66"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 15:54:10.360500 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:10.360479    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a7461d38-e9d1-4c56-874c-8c4fc5e5be66" (UID: "a7461d38-e9d1-4c56-874c-8c4fc5e5be66"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 20 15:54:10.360539 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:10.360479    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a7461d38-e9d1-4c56-874c-8c4fc5e5be66" (UID: "a7461d38-e9d1-4c56-874c-8c4fc5e5be66"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 15:54:10.459646 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:10.459619    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-istio-envoy\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:54:10.459646 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:10.459640    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f66xt\" (UniqueName: \"kubernetes.io/projected/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-kube-api-access-f66xt\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:54:10.459646 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:10.459649    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-istio-data\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:54:10.459811 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:10.459657    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-workload-socket\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:54:10.459811 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:10.459665    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-istio-podinfo\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:54:10.459811 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:10.459674    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-istio-token\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:54:10.459811 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:10.459681    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7461d38-e9d1-4c56-874c-8c4fc5e5be66-credential-socket\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:54:10.509880 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:10.509853    2559 generic.go:356] "Generic (PLEG): container finished" podID="a7461d38-e9d1-4c56-874c-8c4fc5e5be66" containerID="f5e1ecb7d7d04215481610f7cd9a6dafb00f91f0e9a918c842f8a0499b0730de" exitCode=0
Jul 20 15:54:10.510267 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:10.509910    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"
Jul 20 15:54:10.510267 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:10.509934    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m" event={"ID":"a7461d38-e9d1-4c56-874c-8c4fc5e5be66","Type":"ContainerDied","Data":"f5e1ecb7d7d04215481610f7cd9a6dafb00f91f0e9a918c842f8a0499b0730de"}
Jul 20 15:54:10.510267 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:10.509977    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m" event={"ID":"a7461d38-e9d1-4c56-874c-8c4fc5e5be66","Type":"ContainerDied","Data":"b3e6f457134f861122e66359d166e500ad48c40f61164b21b4833b4444d21caf"}
Jul 20 15:54:10.510267 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:10.509990    2559 scope.go:117] "RemoveContainer" containerID="f5e1ecb7d7d04215481610f7cd9a6dafb00f91f0e9a918c842f8a0499b0730de"
Jul 20 15:54:10.519583 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:10.519568    2559 scope.go:117] "RemoveContainer" containerID="ab272bf6e6007fc4302c33d624980d0cf47f5c080429ca4a8b03c3143624a9b7"
Jul 20 15:54:10.527280 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:10.527265    2559 scope.go:117] "RemoveContainer" containerID="f5e1ecb7d7d04215481610f7cd9a6dafb00f91f0e9a918c842f8a0499b0730de"
Jul 20 15:54:10.527532 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:54:10.527514    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5e1ecb7d7d04215481610f7cd9a6dafb00f91f0e9a918c842f8a0499b0730de\": container with ID starting with f5e1ecb7d7d04215481610f7cd9a6dafb00f91f0e9a918c842f8a0499b0730de not found: ID does not exist" containerID="f5e1ecb7d7d04215481610f7cd9a6dafb00f91f0e9a918c842f8a0499b0730de"
Jul 20 15:54:10.527602 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:10.527545    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5e1ecb7d7d04215481610f7cd9a6dafb00f91f0e9a918c842f8a0499b0730de"} err="failed to get container status \"f5e1ecb7d7d04215481610f7cd9a6dafb00f91f0e9a918c842f8a0499b0730de\": rpc error: code = NotFound desc = could not find container \"f5e1ecb7d7d04215481610f7cd9a6dafb00f91f0e9a918c842f8a0499b0730de\": container with ID starting with f5e1ecb7d7d04215481610f7cd9a6dafb00f91f0e9a918c842f8a0499b0730de not found: ID does not exist"
Jul 20 15:54:10.527602 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:10.527569    2559 scope.go:117] "RemoveContainer" containerID="ab272bf6e6007fc4302c33d624980d0cf47f5c080429ca4a8b03c3143624a9b7"
Jul 20 15:54:10.527779 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:54:10.527758    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab272bf6e6007fc4302c33d624980d0cf47f5c080429ca4a8b03c3143624a9b7\": container with ID starting with ab272bf6e6007fc4302c33d624980d0cf47f5c080429ca4a8b03c3143624a9b7 not found: ID does not exist" containerID="ab272bf6e6007fc4302c33d624980d0cf47f5c080429ca4a8b03c3143624a9b7"
Jul 20 15:54:10.527824 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:10.527784    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab272bf6e6007fc4302c33d624980d0cf47f5c080429ca4a8b03c3143624a9b7"} err="failed to get container status \"ab272bf6e6007fc4302c33d624980d0cf47f5c080429ca4a8b03c3143624a9b7\": rpc error: code = NotFound desc = could not find container \"ab272bf6e6007fc4302c33d624980d0cf47f5c080429ca4a8b03c3143624a9b7\": container with ID starting with ab272bf6e6007fc4302c33d624980d0cf47f5c080429ca4a8b03c3143624a9b7 not found: ID does not exist"
Jul 20 15:54:10.531690 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:10.531672    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"]
Jul 20 15:54:10.537242 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:10.537220    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-e9c2c9fb-gw-openshift-default-98fd74599-5ft2m"]
Jul 20 15:54:11.515421 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:11.515395    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7461d38-e9d1-4c56-874c-8c4fc5e5be66" path="/var/lib/kubelet/pods/a7461d38-e9d1-4c56-874c-8c4fc5e5be66/volumes"
Jul 20 15:54:15.515805 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:15.515777    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dde17afb-68f4d7478f-699qj"
Jul 20 15:54:44.646025 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:44.645992    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dde17afb-68f4d7478f-699qj"]
Jul 20 15:54:44.646513 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:44.646221    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dde17afb-68f4d7478f-699qj" podUID="aa263126-cbbe-422c-8979-1c0a7eadfe90" containerName="maas-api" containerID="cri-o://95e54c2d2cdade79bb84a35e75f8f4c0d56df8a99f709d55bb6f3caf39896678" gracePeriod=30
Jul 20 15:54:44.722230 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:44.722194    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:44.890904 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:44.890881    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dde17afb-68f4d7478f-699qj"
Jul 20 15:54:45.036033 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:45.036007    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa263126-cbbe-422c-8979-1c0a7eadfe90-openshift-service-ca\") pod \"aa263126-cbbe-422c-8979-1c0a7eadfe90\" (UID: \"aa263126-cbbe-422c-8979-1c0a7eadfe90\") "
Jul 20 15:54:45.036160 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:45.036081    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fmvxb\" (UniqueName: \"kubernetes.io/projected/aa263126-cbbe-422c-8979-1c0a7eadfe90-kube-api-access-fmvxb\") pod \"aa263126-cbbe-422c-8979-1c0a7eadfe90\" (UID: \"aa263126-cbbe-422c-8979-1c0a7eadfe90\") "
Jul 20 15:54:45.036160 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:45.036121    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa263126-cbbe-422c-8979-1c0a7eadfe90-maas-api-tls\") pod \"aa263126-cbbe-422c-8979-1c0a7eadfe90\" (UID: \"aa263126-cbbe-422c-8979-1c0a7eadfe90\") "
Jul 20 15:54:45.036404 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:45.036378    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa263126-cbbe-422c-8979-1c0a7eadfe90-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "aa263126-cbbe-422c-8979-1c0a7eadfe90" (UID: "aa263126-cbbe-422c-8979-1c0a7eadfe90"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 15:54:45.037867 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:45.037842    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa263126-cbbe-422c-8979-1c0a7eadfe90-kube-api-access-fmvxb" (OuterVolumeSpecName: "kube-api-access-fmvxb") pod "aa263126-cbbe-422c-8979-1c0a7eadfe90" (UID: "aa263126-cbbe-422c-8979-1c0a7eadfe90"). InnerVolumeSpecName "kube-api-access-fmvxb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 15:54:45.037970 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:45.037902    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa263126-cbbe-422c-8979-1c0a7eadfe90-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "aa263126-cbbe-422c-8979-1c0a7eadfe90" (UID: "aa263126-cbbe-422c-8979-1c0a7eadfe90"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 15:54:45.137121 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:45.137094    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa263126-cbbe-422c-8979-1c0a7eadfe90-openshift-service-ca\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:54:45.137121 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:45.137116    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmvxb\" (UniqueName: \"kubernetes.io/projected/aa263126-cbbe-422c-8979-1c0a7eadfe90-kube-api-access-fmvxb\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:54:45.137121 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:45.137125    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa263126-cbbe-422c-8979-1c0a7eadfe90-maas-api-tls\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:54:45.669620 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:45.669590    2559 generic.go:356] "Generic (PLEG): container finished" podID="aa263126-cbbe-422c-8979-1c0a7eadfe90" containerID="95e54c2d2cdade79bb84a35e75f8f4c0d56df8a99f709d55bb6f3caf39896678" exitCode=0
Jul 20 15:54:45.670067 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:45.669641    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dde17afb-68f4d7478f-699qj"
Jul 20 15:54:45.670067 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:45.669657    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dde17afb-68f4d7478f-699qj" event={"ID":"aa263126-cbbe-422c-8979-1c0a7eadfe90","Type":"ContainerDied","Data":"95e54c2d2cdade79bb84a35e75f8f4c0d56df8a99f709d55bb6f3caf39896678"}
Jul 20 15:54:45.670067 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:45.669703    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dde17afb-68f4d7478f-699qj" event={"ID":"aa263126-cbbe-422c-8979-1c0a7eadfe90","Type":"ContainerDied","Data":"a0326f483e8e3c13886e325a82827b8f331c9d98ee9b144fbc488935f2b72797"}
Jul 20 15:54:45.670067 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:45.669724    2559 scope.go:117] "RemoveContainer" containerID="95e54c2d2cdade79bb84a35e75f8f4c0d56df8a99f709d55bb6f3caf39896678"
Jul 20 15:54:45.677702 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:45.677685    2559 scope.go:117] "RemoveContainer" containerID="95e54c2d2cdade79bb84a35e75f8f4c0d56df8a99f709d55bb6f3caf39896678"
Jul 20 15:54:45.677988 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:54:45.677951    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95e54c2d2cdade79bb84a35e75f8f4c0d56df8a99f709d55bb6f3caf39896678\": container with ID starting with 95e54c2d2cdade79bb84a35e75f8f4c0d56df8a99f709d55bb6f3caf39896678 not found: ID does not exist" containerID="95e54c2d2cdade79bb84a35e75f8f4c0d56df8a99f709d55bb6f3caf39896678"
Jul 20 15:54:45.678105 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:45.677980    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95e54c2d2cdade79bb84a35e75f8f4c0d56df8a99f709d55bb6f3caf39896678"} err="failed to get container status \"95e54c2d2cdade79bb84a35e75f8f4c0d56df8a99f709d55bb6f3caf39896678\": rpc error: code = NotFound desc = could not find container \"95e54c2d2cdade79bb84a35e75f8f4c0d56df8a99f709d55bb6f3caf39896678\": container with ID starting with 95e54c2d2cdade79bb84a35e75f8f4c0d56df8a99f709d55bb6f3caf39896678 not found: ID does not exist"
Jul 20 15:54:45.703277 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:45.703247    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dde17afb-68f4d7478f-699qj"]
Jul 20 15:54:45.710204 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:45.710185    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dde17afb-68f4d7478f-699qj"]
Jul 20 15:54:47.519379 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:47.519347    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa263126-cbbe-422c-8979-1c0a7eadfe90" path="/var/lib/kubelet/pods/aa263126-cbbe-422c-8979-1c0a7eadfe90/volumes"
Jul 20 15:54:49.145249 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:49.145218    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:50.594570 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:50.594536    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:50.837502 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:50.837466    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:51.084276 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.084235    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:51.429889 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.429817    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"]
Jul 20 15:54:51.430181 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.430169    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa263126-cbbe-422c-8979-1c0a7eadfe90" containerName="maas-api"
Jul 20 15:54:51.430224 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.430183    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa263126-cbbe-422c-8979-1c0a7eadfe90" containerName="maas-api"
Jul 20 15:54:51.430224 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.430198    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7461d38-e9d1-4c56-874c-8c4fc5e5be66" containerName="istio-proxy"
Jul 20 15:54:51.430224 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.430204    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7461d38-e9d1-4c56-874c-8c4fc5e5be66" containerName="istio-proxy"
Jul 20 15:54:51.430355 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.430296    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7461d38-e9d1-4c56-874c-8c4fc5e5be66" containerName="istio-proxy"
Jul 20 15:54:51.430355 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.430306    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7461d38-e9d1-4c56-874c-8c4fc5e5be66" containerName="istio-proxy"
Jul 20 15:54:51.430355 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.430314    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa263126-cbbe-422c-8979-1c0a7eadfe90" containerName="maas-api"
Jul 20 15:54:51.434667 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.434648    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"
Jul 20 15:54:51.437350 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.437332    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 20 15:54:51.437456 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.437394    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-28fc283e-openshift-default-dockercfg-fwfn7\""
Jul 20 15:54:51.467249 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.467224    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"]
Jul 20 15:54:51.490449 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.490420    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:51.590470 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.590440    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8b2155f-4487-4196-9c47-10eb031ef08c-istio-podinfo\") pod \"e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb\" (UID: \"e8b2155f-4487-4196-9c47-10eb031ef08c\") " pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"
Jul 20 15:54:51.590595 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.590475    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8b2155f-4487-4196-9c47-10eb031ef08c-workload-certs\") pod \"e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb\" (UID: \"e8b2155f-4487-4196-9c47-10eb031ef08c\") " pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"
Jul 20 15:54:51.590595 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.590494    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8b2155f-4487-4196-9c47-10eb031ef08c-istio-data\") pod \"e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb\" (UID: \"e8b2155f-4487-4196-9c47-10eb031ef08c\") " pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"
Jul 20 15:54:51.590595 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.590576    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8b2155f-4487-4196-9c47-10eb031ef08c-credential-socket\") pod \"e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb\" (UID: \"e8b2155f-4487-4196-9c47-10eb031ef08c\") " pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"
Jul 20 15:54:51.590703 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.590621    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8b2155f-4487-4196-9c47-10eb031ef08c-istio-envoy\") pod \"e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb\" (UID: \"e8b2155f-4487-4196-9c47-10eb031ef08c\") " pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"
Jul 20 15:54:51.590703 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.590651    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8b2155f-4487-4196-9c47-10eb031ef08c-istio-token\") pod \"e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb\" (UID: \"e8b2155f-4487-4196-9c47-10eb031ef08c\") " pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"
Jul 20 15:54:51.590766 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.590732    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8b2155f-4487-4196-9c47-10eb031ef08c-istiod-ca-cert\") pod \"e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb\" (UID: \"e8b2155f-4487-4196-9c47-10eb031ef08c\") " pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"
Jul 20 15:54:51.590766 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.590758    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmcd5\" (UniqueName: \"kubernetes.io/projected/e8b2155f-4487-4196-9c47-10eb031ef08c-kube-api-access-pmcd5\") pod \"e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb\" (UID: \"e8b2155f-4487-4196-9c47-10eb031ef08c\") " pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"
Jul 20 15:54:51.590831 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.590814    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8b2155f-4487-4196-9c47-10eb031ef08c-workload-socket\") pod \"e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb\" (UID: \"e8b2155f-4487-4196-9c47-10eb031ef08c\") " pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"
Jul 20 15:54:51.619850 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.619826    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:51.691505 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.691447    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8b2155f-4487-4196-9c47-10eb031ef08c-istiod-ca-cert\") pod \"e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb\" (UID: \"e8b2155f-4487-4196-9c47-10eb031ef08c\") " pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"
Jul 20 15:54:51.691505 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.691480    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmcd5\" (UniqueName: \"kubernetes.io/projected/e8b2155f-4487-4196-9c47-10eb031ef08c-kube-api-access-pmcd5\") pod \"e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb\" (UID: \"e8b2155f-4487-4196-9c47-10eb031ef08c\") " pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"
Jul 20 15:54:51.691648 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.691508    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8b2155f-4487-4196-9c47-10eb031ef08c-workload-socket\") pod \"e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb\" (UID: \"e8b2155f-4487-4196-9c47-10eb031ef08c\") " pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"
Jul 20 15:54:51.691648 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.691530    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8b2155f-4487-4196-9c47-10eb031ef08c-istio-podinfo\") pod \"e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb\" (UID: \"e8b2155f-4487-4196-9c47-10eb031ef08c\") " pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"
Jul 20 15:54:51.691648 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.691559    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8b2155f-4487-4196-9c47-10eb031ef08c-workload-certs\") pod \"e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb\" (UID: \"e8b2155f-4487-4196-9c47-10eb031ef08c\") " pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"
Jul 20 15:54:51.691648 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.691581    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8b2155f-4487-4196-9c47-10eb031ef08c-istio-data\") pod \"e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb\" (UID: \"e8b2155f-4487-4196-9c47-10eb031ef08c\") " pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"
Jul 20 15:54:51.691648 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.691628    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8b2155f-4487-4196-9c47-10eb031ef08c-credential-socket\") pod \"e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb\" (UID: \"e8b2155f-4487-4196-9c47-10eb031ef08c\") " pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"
Jul 20 15:54:51.691898 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.691661    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8b2155f-4487-4196-9c47-10eb031ef08c-istio-envoy\") pod \"e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb\" (UID: \"e8b2155f-4487-4196-9c47-10eb031ef08c\") " pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"
Jul 20 15:54:51.691898 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.691698    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8b2155f-4487-4196-9c47-10eb031ef08c-istio-token\") pod \"e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb\" (UID: \"e8b2155f-4487-4196-9c47-10eb031ef08c\") " pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"
Jul 20 15:54:51.692002 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.691969    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8b2155f-4487-4196-9c47-10eb031ef08c-workload-socket\") pod \"e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb\" (UID: \"e8b2155f-4487-4196-9c47-10eb031ef08c\") " pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"
Jul 20 15:54:51.692056 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.692000    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8b2155f-4487-4196-9c47-10eb031ef08c-istio-data\") pod \"e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb\" (UID: \"e8b2155f-4487-4196-9c47-10eb031ef08c\") " pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"
Jul 20 15:54:51.692230 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.692167    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8b2155f-4487-4196-9c47-10eb031ef08c-workload-certs\") pod \"e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb\" (UID: \"e8b2155f-4487-4196-9c47-10eb031ef08c\") " pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"
Jul 20 15:54:51.692230 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.692182    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8b2155f-4487-4196-9c47-10eb031ef08c-credential-socket\") pod \"e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb\" (UID: \"e8b2155f-4487-4196-9c47-10eb031ef08c\") " pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"
Jul 20 15:54:51.692383 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.692241    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8b2155f-4487-4196-9c47-10eb031ef08c-istiod-ca-cert\") pod \"e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb\" (UID: \"e8b2155f-4487-4196-9c47-10eb031ef08c\") " pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"
Jul 20 15:54:51.694400 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.694378    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8b2155f-4487-4196-9c47-10eb031ef08c-istio-envoy\") pod \"e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb\" (UID: \"e8b2155f-4487-4196-9c47-10eb031ef08c\") " pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"
Jul 20 15:54:51.694716 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.694695    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8b2155f-4487-4196-9c47-10eb031ef08c-istio-podinfo\") pod \"e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb\" (UID: \"e8b2155f-4487-4196-9c47-10eb031ef08c\") " pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"
Jul 20 15:54:51.702510 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.702491    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8b2155f-4487-4196-9c47-10eb031ef08c-istio-token\") pod \"e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb\" (UID: \"e8b2155f-4487-4196-9c47-10eb031ef08c\") " pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"
Jul 20 15:54:51.702883 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.702862    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmcd5\" (UniqueName: \"kubernetes.io/projected/e8b2155f-4487-4196-9c47-10eb031ef08c-kube-api-access-pmcd5\") pod \"e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb\" (UID: \"e8b2155f-4487-4196-9c47-10eb031ef08c\") " pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"
Jul 20 15:54:51.745377 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.745347    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"
Jul 20 15:54:51.887756 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.887725    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"]
Jul 20 15:54:51.890698 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:54:51.890675    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8b2155f_4487_4196_9c47_10eb031ef08c.slice/crio-b792f5d51785060e333b8c5a5e1b962660ea4a6e7f0c49f825a3313c575795b2 WatchSource:0}: Error finding container b792f5d51785060e333b8c5a5e1b962660ea4a6e7f0c49f825a3313c575795b2: Status 404 returned error can't find the container with id b792f5d51785060e333b8c5a5e1b962660ea4a6e7f0c49f825a3313c575795b2
Jul 20 15:54:51.892712 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.892685    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 15:54:51.892786 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.892746    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 15:54:51.892786 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:51.892775    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 15:54:52.702887 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:52.702858    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb" event={"ID":"e8b2155f-4487-4196-9c47-10eb031ef08c","Type":"ContainerStarted","Data":"8897172e7e747345643cdb0bea320044d7353d617e751446e695af5acdf03d93"}
Jul 20 15:54:52.703248 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:52.702908    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb" event={"ID":"e8b2155f-4487-4196-9c47-10eb031ef08c","Type":"ContainerStarted","Data":"b792f5d51785060e333b8c5a5e1b962660ea4a6e7f0c49f825a3313c575795b2"}
Jul 20 15:54:52.729711 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:52.729667    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb" podStartSLOduration=1.72965076 podStartE2EDuration="1.72965076s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:54:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:54:52.728048206 +0000 UTC m=+1675.737669867" watchObservedRunningTime="2026-07-20 15:54:52.72965076 +0000 UTC m=+1675.739272421"
Jul 20 15:54:52.745707 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:52.745683    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"
Jul 20 15:54:52.750575 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:52.750558    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"
Jul 20 15:54:53.708347 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:53.708311    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"
Jul 20 15:54:53.709195 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:53.709175    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"
Jul 20 15:54:54.154985 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:54.154947    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:54.287349 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:54.287319    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-28fc283e-7788cf9c9-8p5f5"]
Jul 20 15:54:54.287683 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:54.287671    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7461d38-e9d1-4c56-874c-8c4fc5e5be66" containerName="istio-proxy"
Jul 20 15:54:54.287723 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:54.287684    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7461d38-e9d1-4c56-874c-8c4fc5e5be66" containerName="istio-proxy"
Jul 20 15:54:54.290741 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:54.290727    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-28fc283e-7788cf9c9-8p5f5"
Jul 20 15:54:54.294328 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:54.294309    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-96kqc\""
Jul 20 15:54:54.294433 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:54.294308    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-28fc283e-serving-cert\""
Jul 20 15:54:54.308535 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:54.308519    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-28fc283e-7788cf9c9-8p5f5"]
Jul 20 15:54:54.409717 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:54.409640    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:54.413529 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:54.413508    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87-openshift-service-ca\") pod \"maas-api-e2e-derive-28fc283e-7788cf9c9-8p5f5\" (UID: \"b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-28fc283e-7788cf9c9-8p5f5"
Jul 20 15:54:54.413648 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:54.413542    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bg26\" (UniqueName: \"kubernetes.io/projected/b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87-kube-api-access-4bg26\") pod \"maas-api-e2e-derive-28fc283e-7788cf9c9-8p5f5\" (UID: \"b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-28fc283e-7788cf9c9-8p5f5"
Jul 20 15:54:54.413648 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:54.413617    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87-maas-api-tls\") pod \"maas-api-e2e-derive-28fc283e-7788cf9c9-8p5f5\" (UID: \"b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-28fc283e-7788cf9c9-8p5f5"
Jul 20 15:54:54.514270 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:54.514239    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87-maas-api-tls\") pod \"maas-api-e2e-derive-28fc283e-7788cf9c9-8p5f5\" (UID: \"b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-28fc283e-7788cf9c9-8p5f5"
Jul 20 15:54:54.514411 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:54.514354    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87-openshift-service-ca\") pod \"maas-api-e2e-derive-28fc283e-7788cf9c9-8p5f5\" (UID: \"b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-28fc283e-7788cf9c9-8p5f5"
Jul 20 15:54:54.514411 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:54.514395    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bg26\" (UniqueName: \"kubernetes.io/projected/b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87-kube-api-access-4bg26\") pod \"maas-api-e2e-derive-28fc283e-7788cf9c9-8p5f5\" (UID: \"b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-28fc283e-7788cf9c9-8p5f5"
Jul 20 15:54:54.514942 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:54.514922    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87-openshift-service-ca\") pod \"maas-api-e2e-derive-28fc283e-7788cf9c9-8p5f5\" (UID: \"b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-28fc283e-7788cf9c9-8p5f5"
Jul 20 15:54:54.516490 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:54.516472    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87-maas-api-tls\") pod \"maas-api-e2e-derive-28fc283e-7788cf9c9-8p5f5\" (UID: \"b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-28fc283e-7788cf9c9-8p5f5"
Jul 20 15:54:54.546891 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:54.546864    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bg26\" (UniqueName: \"kubernetes.io/projected/b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87-kube-api-access-4bg26\") pod \"maas-api-e2e-derive-28fc283e-7788cf9c9-8p5f5\" (UID: \"b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-28fc283e-7788cf9c9-8p5f5"
Jul 20 15:54:54.552900 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:54.552871    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:54.600579 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:54.600558    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-28fc283e-7788cf9c9-8p5f5"
Jul 20 15:54:54.744751 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:54.744718    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:54.949034 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:54.948959    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-28fc283e-7788cf9c9-8p5f5"]
Jul 20 15:54:54.954999 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:54:54.954963    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3db13a6_667b_4ef1_a6ea_8ca47cd1ae87.slice/crio-ae2db42edc6e2f01482268a56f3c5751219451384c27eb1b57dd5f1491cb2ddc WatchSource:0}: Error finding container ae2db42edc6e2f01482268a56f3c5751219451384c27eb1b57dd5f1491cb2ddc: Status 404 returned error can't find the container with id ae2db42edc6e2f01482268a56f3c5751219451384c27eb1b57dd5f1491cb2ddc
Jul 20 15:54:55.176627 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:55.176589    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:55.276697 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:55.276646    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:54:55.718349 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:55.718313    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-28fc283e-7788cf9c9-8p5f5" event={"ID":"b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87","Type":"ContainerStarted","Data":"edefd7d20e266463578449e66fe08b9b9ad0bc5cd75377d9af7f3882d8ba9a53"}
Jul 20 15:54:55.718510 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:55.718485    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-28fc283e-7788cf9c9-8p5f5"
Jul 20 15:54:55.718563 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:55.718518    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-28fc283e-7788cf9c9-8p5f5" event={"ID":"b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87","Type":"ContainerStarted","Data":"ae2db42edc6e2f01482268a56f3c5751219451384c27eb1b57dd5f1491cb2ddc"}
Jul 20 15:54:55.741233 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:54:55.741182    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-28fc283e-7788cf9c9-8p5f5" podStartSLOduration=1.218444372 podStartE2EDuration="1.741166527s" totalImagesPullingTime="522.722155ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:54:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:54:55.739491583 +0000 UTC m=+1678.749113246" watchObservedRunningTime="2026-07-20 15:54:55.741166527 +0000 UTC m=+1678.750788189"
Jul 20 15:55:01.735574 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:01.735534    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-28fc283e-7788cf9c9-8p5f5"
Jul 20 15:55:07.781197 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:07.781166    2559 generic.go:356] "Generic (PLEG): container finished" podID="e8b2155f-4487-4196-9c47-10eb031ef08c" containerID="8897172e7e747345643cdb0bea320044d7353d617e751446e695af5acdf03d93" exitCode=0
Jul 20 15:55:07.781613 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:07.781240    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb" event={"ID":"e8b2155f-4487-4196-9c47-10eb031ef08c","Type":"ContainerDied","Data":"8897172e7e747345643cdb0bea320044d7353d617e751446e695af5acdf03d93"}
Jul 20 15:55:07.781613 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:07.781610    2559 scope.go:117] "RemoveContainer" containerID="8897172e7e747345643cdb0bea320044d7353d617e751446e695af5acdf03d93"
Jul 20 15:55:07.782670 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:07.782647    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 15:55:07.782728 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:07.782712    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 15:55:07.782764 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:07.782752    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 15:55:08.708524 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:08.708470    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"
Jul 20 15:55:08.787507 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:08.787472    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb" event={"ID":"e8b2155f-4487-4196-9c47-10eb031ef08c","Type":"ContainerStarted","Data":"034ffd0ed9e2936353c067470993cbb5437e79f6ba866c80652b03e72717353f"}
Jul 20 15:55:09.746132 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:09.746088    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"
Jul 20 15:55:09.747733 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:09.747704    2559 patch_prober.go:28] interesting pod/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 20 15:55:09.747956 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:09.747931    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb" podUID="e8b2155f-4487-4196-9c47-10eb031ef08c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 20 15:55:10.746346 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:10.746306    2559 patch_prober.go:28] interesting pod/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 20 15:55:10.746932 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:10.746900    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb" podUID="e8b2155f-4487-4196-9c47-10eb031ef08c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 20 15:55:11.745911 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:11.745874    2559 patch_prober.go:28] interesting pod/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 20 15:55:11.746209 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:11.746177    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb" podUID="e8b2155f-4487-4196-9c47-10eb031ef08c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 20 15:55:12.749829 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:12.749799    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"
Jul 20 15:55:12.750198 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:12.750071    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"
Jul 20 15:55:12.750760 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:12.750745    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"
Jul 20 15:55:29.964217 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:29.964185    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-28fc283e-7788cf9c9-8p5f5"]
Jul 20 15:55:29.964885 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:29.964849    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-28fc283e-7788cf9c9-8p5f5" podUID="b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87" containerName="maas-api" containerID="cri-o://edefd7d20e266463578449e66fe08b9b9ad0bc5cd75377d9af7f3882d8ba9a53" gracePeriod=30
Jul 20 15:55:30.021699 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:30.021667    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:30.212045 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:30.212020    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-28fc283e-7788cf9c9-8p5f5"
Jul 20 15:55:30.323472 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:30.323440    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87-maas-api-tls\") pod \"b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87\" (UID: \"b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87\") "
Jul 20 15:55:30.323618 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:30.323499    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87-openshift-service-ca\") pod \"b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87\" (UID: \"b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87\") "
Jul 20 15:55:30.323618 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:30.323524    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4bg26\" (UniqueName: \"kubernetes.io/projected/b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87-kube-api-access-4bg26\") pod \"b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87\" (UID: \"b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87\") "
Jul 20 15:55:30.323869 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:30.323847    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87" (UID: "b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 15:55:30.325621 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:30.325586    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87" (UID: "b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 15:55:30.325621 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:30.325594    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87-kube-api-access-4bg26" (OuterVolumeSpecName: "kube-api-access-4bg26") pod "b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87" (UID: "b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87"). InnerVolumeSpecName "kube-api-access-4bg26". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 15:55:30.424655 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:30.424630    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87-openshift-service-ca\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:55:30.424655 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:30.424653    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bg26\" (UniqueName: \"kubernetes.io/projected/b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87-kube-api-access-4bg26\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:55:30.424805 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:30.424663    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87-maas-api-tls\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:55:30.883029 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:30.882998    2559 generic.go:356] "Generic (PLEG): container finished" podID="b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87" containerID="edefd7d20e266463578449e66fe08b9b9ad0bc5cd75377d9af7f3882d8ba9a53" exitCode=0
Jul 20 15:55:30.883185 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:30.883043    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-28fc283e-7788cf9c9-8p5f5"
Jul 20 15:55:30.883185 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:30.883060    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-28fc283e-7788cf9c9-8p5f5" event={"ID":"b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87","Type":"ContainerDied","Data":"edefd7d20e266463578449e66fe08b9b9ad0bc5cd75377d9af7f3882d8ba9a53"}
Jul 20 15:55:30.883185 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:30.883106    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-28fc283e-7788cf9c9-8p5f5" event={"ID":"b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87","Type":"ContainerDied","Data":"ae2db42edc6e2f01482268a56f3c5751219451384c27eb1b57dd5f1491cb2ddc"}
Jul 20 15:55:30.883185 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:30.883133    2559 scope.go:117] "RemoveContainer" containerID="edefd7d20e266463578449e66fe08b9b9ad0bc5cd75377d9af7f3882d8ba9a53"
Jul 20 15:55:30.892013 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:30.891996    2559 scope.go:117] "RemoveContainer" containerID="edefd7d20e266463578449e66fe08b9b9ad0bc5cd75377d9af7f3882d8ba9a53"
Jul 20 15:55:30.892233 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:55:30.892215    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edefd7d20e266463578449e66fe08b9b9ad0bc5cd75377d9af7f3882d8ba9a53\": container with ID starting with edefd7d20e266463578449e66fe08b9b9ad0bc5cd75377d9af7f3882d8ba9a53 not found: ID does not exist" containerID="edefd7d20e266463578449e66fe08b9b9ad0bc5cd75377d9af7f3882d8ba9a53"
Jul 20 15:55:30.892312 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:30.892241    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edefd7d20e266463578449e66fe08b9b9ad0bc5cd75377d9af7f3882d8ba9a53"} err="failed to get container status \"edefd7d20e266463578449e66fe08b9b9ad0bc5cd75377d9af7f3882d8ba9a53\": rpc error: code = NotFound desc = could not find container \"edefd7d20e266463578449e66fe08b9b9ad0bc5cd75377d9af7f3882d8ba9a53\": container with ID starting with edefd7d20e266463578449e66fe08b9b9ad0bc5cd75377d9af7f3882d8ba9a53 not found: ID does not exist"
Jul 20 15:55:30.910359 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:30.910338    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-28fc283e-7788cf9c9-8p5f5"]
Jul 20 15:55:30.923561 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:30.923544    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-28fc283e-7788cf9c9-8p5f5"]
Jul 20 15:55:31.514282 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:31.514253    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87" path="/var/lib/kubelet/pods/b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87/volumes"
Jul 20 15:55:34.156183 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:34.156151    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:34.368900 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:34.368869    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"]
Jul 20 15:55:34.369134 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:34.369109    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb" podUID="e8b2155f-4487-4196-9c47-10eb031ef08c" containerName="istio-proxy" containerID="cri-o://034ffd0ed9e2936353c067470993cbb5437e79f6ba866c80652b03e72717353f" gracePeriod=30
Jul 20 15:55:34.378880 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:34.378854    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:34.694457 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:34.694428    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:35.189727 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:35.189697    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:35.830121 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:35.830082    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:36.429713 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:36.429661    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:36.575198 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:36.575150    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:36.877534 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:36.877489    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:39.625335 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:39.625315    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"
Jul 20 15:55:39.701986 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:39.701912    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pmcd5\" (UniqueName: \"kubernetes.io/projected/e8b2155f-4487-4196-9c47-10eb031ef08c-kube-api-access-pmcd5\") pod \"e8b2155f-4487-4196-9c47-10eb031ef08c\" (UID: \"e8b2155f-4487-4196-9c47-10eb031ef08c\") "
Jul 20 15:55:39.701986 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:39.701975    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8b2155f-4487-4196-9c47-10eb031ef08c-istio-podinfo\") pod \"e8b2155f-4487-4196-9c47-10eb031ef08c\" (UID: \"e8b2155f-4487-4196-9c47-10eb031ef08c\") "
Jul 20 15:55:39.702174 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:39.701994    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8b2155f-4487-4196-9c47-10eb031ef08c-istio-envoy\") pod \"e8b2155f-4487-4196-9c47-10eb031ef08c\" (UID: \"e8b2155f-4487-4196-9c47-10eb031ef08c\") "
Jul 20 15:55:39.702174 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:39.702020    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8b2155f-4487-4196-9c47-10eb031ef08c-workload-socket\") pod \"e8b2155f-4487-4196-9c47-10eb031ef08c\" (UID: \"e8b2155f-4487-4196-9c47-10eb031ef08c\") "
Jul 20 15:55:39.702174 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:39.702083    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8b2155f-4487-4196-9c47-10eb031ef08c-istio-data\") pod \"e8b2155f-4487-4196-9c47-10eb031ef08c\" (UID: \"e8b2155f-4487-4196-9c47-10eb031ef08c\") "
Jul 20 15:55:39.702174 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:39.702124    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8b2155f-4487-4196-9c47-10eb031ef08c-credential-socket\") pod \"e8b2155f-4487-4196-9c47-10eb031ef08c\" (UID: \"e8b2155f-4487-4196-9c47-10eb031ef08c\") "
Jul 20 15:55:39.702174 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:39.702158    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8b2155f-4487-4196-9c47-10eb031ef08c-istiod-ca-cert\") pod \"e8b2155f-4487-4196-9c47-10eb031ef08c\" (UID: \"e8b2155f-4487-4196-9c47-10eb031ef08c\") "
Jul 20 15:55:39.702474 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:39.702186    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8b2155f-4487-4196-9c47-10eb031ef08c-workload-certs\") pod \"e8b2155f-4487-4196-9c47-10eb031ef08c\" (UID: \"e8b2155f-4487-4196-9c47-10eb031ef08c\") "
Jul 20 15:55:39.702474 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:39.702208    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8b2155f-4487-4196-9c47-10eb031ef08c-istio-token\") pod \"e8b2155f-4487-4196-9c47-10eb031ef08c\" (UID: \"e8b2155f-4487-4196-9c47-10eb031ef08c\") "
Jul 20 15:55:39.702474 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:39.702413    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8b2155f-4487-4196-9c47-10eb031ef08c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e8b2155f-4487-4196-9c47-10eb031ef08c" (UID: "e8b2155f-4487-4196-9c47-10eb031ef08c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 15:55:39.702618 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:39.702554    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8b2155f-4487-4196-9c47-10eb031ef08c-credential-socket\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:55:39.702676 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:39.702644    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8b2155f-4487-4196-9c47-10eb031ef08c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e8b2155f-4487-4196-9c47-10eb031ef08c" (UID: "e8b2155f-4487-4196-9c47-10eb031ef08c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 15:55:39.702765 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:39.702734    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8b2155f-4487-4196-9c47-10eb031ef08c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e8b2155f-4487-4196-9c47-10eb031ef08c" (UID: "e8b2155f-4487-4196-9c47-10eb031ef08c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 15:55:39.702841 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:39.702806    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8b2155f-4487-4196-9c47-10eb031ef08c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e8b2155f-4487-4196-9c47-10eb031ef08c" (UID: "e8b2155f-4487-4196-9c47-10eb031ef08c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 15:55:39.702959 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:39.702935    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8b2155f-4487-4196-9c47-10eb031ef08c-istio-data" (OuterVolumeSpecName: "istio-data") pod "e8b2155f-4487-4196-9c47-10eb031ef08c" (UID: "e8b2155f-4487-4196-9c47-10eb031ef08c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 15:55:39.704311 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:39.704265    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8b2155f-4487-4196-9c47-10eb031ef08c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e8b2155f-4487-4196-9c47-10eb031ef08c" (UID: "e8b2155f-4487-4196-9c47-10eb031ef08c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 15:55:39.704311 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:39.704280    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8b2155f-4487-4196-9c47-10eb031ef08c-kube-api-access-pmcd5" (OuterVolumeSpecName: "kube-api-access-pmcd5") pod "e8b2155f-4487-4196-9c47-10eb031ef08c" (UID: "e8b2155f-4487-4196-9c47-10eb031ef08c"). InnerVolumeSpecName "kube-api-access-pmcd5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 15:55:39.704601 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:39.704587    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8b2155f-4487-4196-9c47-10eb031ef08c-istio-token" (OuterVolumeSpecName: "istio-token") pod "e8b2155f-4487-4196-9c47-10eb031ef08c" (UID: "e8b2155f-4487-4196-9c47-10eb031ef08c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 15:55:39.704658 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:39.704596    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e8b2155f-4487-4196-9c47-10eb031ef08c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e8b2155f-4487-4196-9c47-10eb031ef08c" (UID: "e8b2155f-4487-4196-9c47-10eb031ef08c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 20 15:55:39.803000 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:39.802966    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmcd5\" (UniqueName: \"kubernetes.io/projected/e8b2155f-4487-4196-9c47-10eb031ef08c-kube-api-access-pmcd5\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:55:39.803000 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:39.802996    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8b2155f-4487-4196-9c47-10eb031ef08c-istio-podinfo\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:55:39.803000 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:39.803008    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8b2155f-4487-4196-9c47-10eb031ef08c-istio-envoy\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:55:39.803232 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:39.803019    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8b2155f-4487-4196-9c47-10eb031ef08c-workload-socket\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:55:39.803232 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:39.803029    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8b2155f-4487-4196-9c47-10eb031ef08c-istio-data\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:55:39.803232 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:39.803042    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8b2155f-4487-4196-9c47-10eb031ef08c-istiod-ca-cert\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:55:39.803232 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:39.803052    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8b2155f-4487-4196-9c47-10eb031ef08c-workload-certs\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:55:39.803232 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:39.803061    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8b2155f-4487-4196-9c47-10eb031ef08c-istio-token\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:55:39.924483 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:39.924450    2559 generic.go:356] "Generic (PLEG): container finished" podID="e8b2155f-4487-4196-9c47-10eb031ef08c" containerID="034ffd0ed9e2936353c067470993cbb5437e79f6ba866c80652b03e72717353f" exitCode=0
Jul 20 15:55:39.924619 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:39.924498    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"
Jul 20 15:55:39.924619 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:39.924510    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb" event={"ID":"e8b2155f-4487-4196-9c47-10eb031ef08c","Type":"ContainerDied","Data":"034ffd0ed9e2936353c067470993cbb5437e79f6ba866c80652b03e72717353f"}
Jul 20 15:55:39.924619 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:39.924561    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb" event={"ID":"e8b2155f-4487-4196-9c47-10eb031ef08c","Type":"ContainerDied","Data":"b792f5d51785060e333b8c5a5e1b962660ea4a6e7f0c49f825a3313c575795b2"}
Jul 20 15:55:39.924619 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:39.924585    2559 scope.go:117] "RemoveContainer" containerID="034ffd0ed9e2936353c067470993cbb5437e79f6ba866c80652b03e72717353f"
Jul 20 15:55:39.933413 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:39.933319    2559 scope.go:117] "RemoveContainer" containerID="8897172e7e747345643cdb0bea320044d7353d617e751446e695af5acdf03d93"
Jul 20 15:55:39.940862 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:39.940847    2559 scope.go:117] "RemoveContainer" containerID="034ffd0ed9e2936353c067470993cbb5437e79f6ba866c80652b03e72717353f"
Jul 20 15:55:39.941093 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:55:39.941074    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"034ffd0ed9e2936353c067470993cbb5437e79f6ba866c80652b03e72717353f\": container with ID starting with 034ffd0ed9e2936353c067470993cbb5437e79f6ba866c80652b03e72717353f not found: ID does not exist" containerID="034ffd0ed9e2936353c067470993cbb5437e79f6ba866c80652b03e72717353f"
Jul 20 15:55:39.941155 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:39.941101    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"034ffd0ed9e2936353c067470993cbb5437e79f6ba866c80652b03e72717353f"} err="failed to get container status \"034ffd0ed9e2936353c067470993cbb5437e79f6ba866c80652b03e72717353f\": rpc error: code = NotFound desc = could not find container \"034ffd0ed9e2936353c067470993cbb5437e79f6ba866c80652b03e72717353f\": container with ID starting with 034ffd0ed9e2936353c067470993cbb5437e79f6ba866c80652b03e72717353f not found: ID does not exist"
Jul 20 15:55:39.941155 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:39.941119    2559 scope.go:117] "RemoveContainer" containerID="8897172e7e747345643cdb0bea320044d7353d617e751446e695af5acdf03d93"
Jul 20 15:55:39.941347 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:55:39.941329    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8897172e7e747345643cdb0bea320044d7353d617e751446e695af5acdf03d93\": container with ID starting with 8897172e7e747345643cdb0bea320044d7353d617e751446e695af5acdf03d93 not found: ID does not exist" containerID="8897172e7e747345643cdb0bea320044d7353d617e751446e695af5acdf03d93"
Jul 20 15:55:39.941402 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:39.941351    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8897172e7e747345643cdb0bea320044d7353d617e751446e695af5acdf03d93"} err="failed to get container status \"8897172e7e747345643cdb0bea320044d7353d617e751446e695af5acdf03d93\": rpc error: code = NotFound desc = could not find container \"8897172e7e747345643cdb0bea320044d7353d617e751446e695af5acdf03d93\": container with ID starting with 8897172e7e747345643cdb0bea320044d7353d617e751446e695af5acdf03d93 not found: ID does not exist"
Jul 20 15:55:39.953507 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:39.953461    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"]
Jul 20 15:55:39.960837 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:39.960817    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-28fc283e-openshift-default-849d6496cf-t8mbb"]
Jul 20 15:55:41.514386 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:41.514360    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8b2155f-4487-4196-9c47-10eb031ef08c" path="/var/lib/kubelet/pods/e8b2155f-4487-4196-9c47-10eb031ef08c/volumes"
Jul 20 15:55:43.911354 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:43.911313    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:44.016738 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:44.016697    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:44.188096 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:44.188017    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:44.359398 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:44.359370    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:55:44.671969 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:55:44.671934    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:56:07.994717 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:56:07.994686    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:56:10.668824 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:56:10.668792    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:56:17.470321 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:56:17.470270    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:56:17.579884 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:56:17.579849    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:56:17.679836 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:56:17.679804    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:56:33.588120 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:56:33.588084    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:56:39.533455 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:56:39.533423    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:56:39.674185 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:56:39.674151    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:56:39.773995 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:56:39.773953    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:56:50.272018 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:56:50.271944    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:56:51.036535 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:56:51.036499    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:56:57.274006 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:56:57.273977    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:56:57.379491 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:56:57.379460    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:56:57.478702 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:56:57.478660    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:56:57.575330 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:56:57.575233    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:56:58.414737 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:56:58.414705    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:56:58.601122 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:56:58.601089    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:00.302459 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:00.302426    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:07.563085 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:07.563022    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:07.664033 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:07.663988    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:08.263438 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:08.263399    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:15.437957 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:15.437897    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:15.540697 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:15.540618    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:15.691820 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:15.691742    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:16.018221 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.018188    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"]
Jul 20 15:57:16.018624 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.018607    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8b2155f-4487-4196-9c47-10eb031ef08c" containerName="istio-proxy"
Jul 20 15:57:16.018712 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.018626    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8b2155f-4487-4196-9c47-10eb031ef08c" containerName="istio-proxy"
Jul 20 15:57:16.018712 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.018665    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87" containerName="maas-api"
Jul 20 15:57:16.018712 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.018674    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87" containerName="maas-api"
Jul 20 15:57:16.018859 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.018773    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3db13a6-667b-4ef1-a6ea-8ca47cd1ae87" containerName="maas-api"
Jul 20 15:57:16.018859 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.018787    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8b2155f-4487-4196-9c47-10eb031ef08c" containerName="istio-proxy"
Jul 20 15:57:16.018859 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.018800    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8b2155f-4487-4196-9c47-10eb031ef08c" containerName="istio-proxy"
Jul 20 15:57:16.021854 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.021831    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"
Jul 20 15:57:16.024149 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.024129    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-edf0382f-openshift-default-dockercfg-cdv5m\""
Jul 20 15:57:16.024637 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.024620    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 20 15:57:16.053877 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.053842    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"]
Jul 20 15:57:16.067417 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.067393    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:16.077476 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.077449    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-credential-socket\") pod \"e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc\" (UID: \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\") " pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"
Jul 20 15:57:16.077623 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.077483    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpjdn\" (UniqueName: \"kubernetes.io/projected/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-kube-api-access-cpjdn\") pod \"e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc\" (UID: \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\") " pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"
Jul 20 15:57:16.077623 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.077576    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-istio-data\") pod \"e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc\" (UID: \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\") " pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"
Jul 20 15:57:16.077623 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.077611    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-istiod-ca-cert\") pod \"e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc\" (UID: \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\") " pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"
Jul 20 15:57:16.077843 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.077667    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-istio-envoy\") pod \"e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc\" (UID: \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\") " pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"
Jul 20 15:57:16.077843 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.077706    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-workload-certs\") pod \"e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc\" (UID: \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\") " pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"
Jul 20 15:57:16.077843 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.077730    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-istio-podinfo\") pod \"e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc\" (UID: \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\") " pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"
Jul 20 15:57:16.077843 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.077758    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-workload-socket\") pod \"e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc\" (UID: \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\") " pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"
Jul 20 15:57:16.077843 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.077780    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-istio-token\") pod \"e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc\" (UID: \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\") " pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"
Jul 20 15:57:16.178817 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.178783    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-istio-envoy\") pod \"e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc\" (UID: \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\") " pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"
Jul 20 15:57:16.178971 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.178822    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-workload-certs\") pod \"e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc\" (UID: \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\") " pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"
Jul 20 15:57:16.178971 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.178840    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-istio-podinfo\") pod \"e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc\" (UID: \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\") " pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"
Jul 20 15:57:16.178971 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.178859    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-workload-socket\") pod \"e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc\" (UID: \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\") " pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"
Jul 20 15:57:16.178971 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.178880    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-istio-token\") pod \"e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc\" (UID: \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\") " pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"
Jul 20 15:57:16.178971 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.178928    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-credential-socket\") pod \"e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc\" (UID: \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\") " pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"
Jul 20 15:57:16.178971 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.178951    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpjdn\" (UniqueName: \"kubernetes.io/projected/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-kube-api-access-cpjdn\") pod \"e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc\" (UID: \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\") " pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"
Jul 20 15:57:16.179320 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.178999    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-istio-data\") pod \"e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc\" (UID: \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\") " pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"
Jul 20 15:57:16.179320 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.179036    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-istiod-ca-cert\") pod \"e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc\" (UID: \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\") " pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"
Jul 20 15:57:16.179435 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.179365    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-workload-certs\") pod \"e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc\" (UID: \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\") " pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"
Jul 20 15:57:16.179492 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.179431    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-credential-socket\") pod \"e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc\" (UID: \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\") " pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"
Jul 20 15:57:16.179492 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.179476    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-workload-socket\") pod \"e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc\" (UID: \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\") " pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"
Jul 20 15:57:16.179678 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.179655    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-istio-data\") pod \"e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc\" (UID: \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\") " pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"
Jul 20 15:57:16.179841 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.179821    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-istiod-ca-cert\") pod \"e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc\" (UID: \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\") " pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"
Jul 20 15:57:16.181348 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.181328    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-istio-envoy\") pod \"e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc\" (UID: \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\") " pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"
Jul 20 15:57:16.181823 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.181807    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-istio-podinfo\") pod \"e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc\" (UID: \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\") " pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"
Jul 20 15:57:16.189477 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.189447    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpjdn\" (UniqueName: \"kubernetes.io/projected/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-kube-api-access-cpjdn\") pod \"e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc\" (UID: \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\") " pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"
Jul 20 15:57:16.191668 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.191651    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-istio-token\") pod \"e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc\" (UID: \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\") " pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"
Jul 20 15:57:16.334394 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.334311    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"
Jul 20 15:57:16.353352 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.353318    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:16.496722 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.496696    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"]
Jul 20 15:57:16.498190 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:57:16.498169    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6edc30c_a27d_4a32_8a92_4e65c17fa1c6.slice/crio-5e14e5ef8d7713449fe7dd9d55a86b315375e2ee15444877b23b5b21a33532a2 WatchSource:0}: Error finding container 5e14e5ef8d7713449fe7dd9d55a86b315375e2ee15444877b23b5b21a33532a2: Status 404 returned error can't find the container with id 5e14e5ef8d7713449fe7dd9d55a86b315375e2ee15444877b23b5b21a33532a2
Jul 20 15:57:16.500214 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.500183    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 15:57:16.500308 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.500246    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 15:57:16.500308 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:16.500299    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 15:57:17.360019 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:17.359987    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc" event={"ID":"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6","Type":"ContainerStarted","Data":"8e9e89e7f8101d983ac471ce04e322c660a8072801929d23b39616763710dad0"}
Jul 20 15:57:17.360176 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:17.360048    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc" event={"ID":"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6","Type":"ContainerStarted","Data":"5e14e5ef8d7713449fe7dd9d55a86b315375e2ee15444877b23b5b21a33532a2"}
Jul 20 15:57:17.434831 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:17.434774    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc" podStartSLOduration=2.434758494 podStartE2EDuration="2.434758494s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:57:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:57:17.431726027 +0000 UTC m=+1820.441347691" watchObservedRunningTime="2026-07-20 15:57:17.434758494 +0000 UTC m=+1820.444380155"
Jul 20 15:57:18.334704 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:18.334668    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"
Jul 20 15:57:18.341206 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:18.341181    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"
Jul 20 15:57:18.364421 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:18.364398    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"
Jul 20 15:57:18.365145 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:18.365131    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"
Jul 20 15:57:19.739171 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:19.739130    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:19.811201 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:19.811164    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-edf0382f-6b5fb6bbf4-57p4v"]
Jul 20 15:57:19.811615 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:19.811597    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8b2155f-4487-4196-9c47-10eb031ef08c" containerName="istio-proxy"
Jul 20 15:57:19.811615 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:19.811617    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8b2155f-4487-4196-9c47-10eb031ef08c" containerName="istio-proxy"
Jul 20 15:57:19.814779 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:19.814758    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-edf0382f-6b5fb6bbf4-57p4v"
Jul 20 15:57:19.818024 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:19.818001    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-edf0382f-serving-cert\""
Jul 20 15:57:19.818640 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:19.818621    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-96kqc\""
Jul 20 15:57:19.828426 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:19.828403    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-edf0382f-6b5fb6bbf4-57p4v"]
Jul 20 15:57:19.844829 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:19.844810    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:19.911886 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:19.911863    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b758f404-4d1e-42f4-9dbb-ce7c176c7573-maas-api-tls\") pod \"maas-api-e2e-mt-edf0382f-6b5fb6bbf4-57p4v\" (UID: \"b758f404-4d1e-42f4-9dbb-ce7c176c7573\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-edf0382f-6b5fb6bbf4-57p4v"
Jul 20 15:57:19.911989 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:19.911934    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b758f404-4d1e-42f4-9dbb-ce7c176c7573-openshift-service-ca\") pod \"maas-api-e2e-mt-edf0382f-6b5fb6bbf4-57p4v\" (UID: \"b758f404-4d1e-42f4-9dbb-ce7c176c7573\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-edf0382f-6b5fb6bbf4-57p4v"
Jul 20 15:57:19.911989 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:19.911975    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zch4n\" (UniqueName: \"kubernetes.io/projected/b758f404-4d1e-42f4-9dbb-ce7c176c7573-kube-api-access-zch4n\") pod \"maas-api-e2e-mt-edf0382f-6b5fb6bbf4-57p4v\" (UID: \"b758f404-4d1e-42f4-9dbb-ce7c176c7573\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-edf0382f-6b5fb6bbf4-57p4v"
Jul 20 15:57:19.950741 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:19.950713    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:20.013460 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:20.013390    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zch4n\" (UniqueName: \"kubernetes.io/projected/b758f404-4d1e-42f4-9dbb-ce7c176c7573-kube-api-access-zch4n\") pod \"maas-api-e2e-mt-edf0382f-6b5fb6bbf4-57p4v\" (UID: \"b758f404-4d1e-42f4-9dbb-ce7c176c7573\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-edf0382f-6b5fb6bbf4-57p4v"
Jul 20 15:57:20.013573 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:20.013469    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b758f404-4d1e-42f4-9dbb-ce7c176c7573-maas-api-tls\") pod \"maas-api-e2e-mt-edf0382f-6b5fb6bbf4-57p4v\" (UID: \"b758f404-4d1e-42f4-9dbb-ce7c176c7573\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-edf0382f-6b5fb6bbf4-57p4v"
Jul 20 15:57:20.013643 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:20.013566    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b758f404-4d1e-42f4-9dbb-ce7c176c7573-openshift-service-ca\") pod \"maas-api-e2e-mt-edf0382f-6b5fb6bbf4-57p4v\" (UID: \"b758f404-4d1e-42f4-9dbb-ce7c176c7573\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-edf0382f-6b5fb6bbf4-57p4v"
Jul 20 15:57:20.013643 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:57:20.013575    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-edf0382f-serving-cert: secret "maas-api-e2e-mt-edf0382f-serving-cert" not found
Jul 20 15:57:20.013643 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:57:20.013636    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b758f404-4d1e-42f4-9dbb-ce7c176c7573-maas-api-tls podName:b758f404-4d1e-42f4-9dbb-ce7c176c7573 nodeName:}" failed. No retries permitted until 2026-07-20 15:57:20.51361487 +0000 UTC m=+1823.523236510 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b758f404-4d1e-42f4-9dbb-ce7c176c7573-maas-api-tls") pod "maas-api-e2e-mt-edf0382f-6b5fb6bbf4-57p4v" (UID: "b758f404-4d1e-42f4-9dbb-ce7c176c7573") : secret "maas-api-e2e-mt-edf0382f-serving-cert" not found
Jul 20 15:57:20.014214 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:20.014188    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b758f404-4d1e-42f4-9dbb-ce7c176c7573-openshift-service-ca\") pod \"maas-api-e2e-mt-edf0382f-6b5fb6bbf4-57p4v\" (UID: \"b758f404-4d1e-42f4-9dbb-ce7c176c7573\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-edf0382f-6b5fb6bbf4-57p4v"
Jul 20 15:57:20.024891 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:20.024871    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zch4n\" (UniqueName: \"kubernetes.io/projected/b758f404-4d1e-42f4-9dbb-ce7c176c7573-kube-api-access-zch4n\") pod \"maas-api-e2e-mt-edf0382f-6b5fb6bbf4-57p4v\" (UID: \"b758f404-4d1e-42f4-9dbb-ce7c176c7573\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-edf0382f-6b5fb6bbf4-57p4v"
Jul 20 15:57:20.370646 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:20.370605    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:20.517703 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:20.517668    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b758f404-4d1e-42f4-9dbb-ce7c176c7573-maas-api-tls\") pod \"maas-api-e2e-mt-edf0382f-6b5fb6bbf4-57p4v\" (UID: \"b758f404-4d1e-42f4-9dbb-ce7c176c7573\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-edf0382f-6b5fb6bbf4-57p4v"
Jul 20 15:57:20.520047 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:20.520028    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b758f404-4d1e-42f4-9dbb-ce7c176c7573-maas-api-tls\") pod \"maas-api-e2e-mt-edf0382f-6b5fb6bbf4-57p4v\" (UID: \"b758f404-4d1e-42f4-9dbb-ce7c176c7573\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-edf0382f-6b5fb6bbf4-57p4v"
Jul 20 15:57:20.610063 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:20.610031    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:20.711810 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:20.711729    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:20.726398 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:20.726376    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-edf0382f-6b5fb6bbf4-57p4v"
Jul 20 15:57:20.862564 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:20.862538    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-edf0382f-6b5fb6bbf4-57p4v"]
Jul 20 15:57:20.865111 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:57:20.865081    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb758f404_4d1e_42f4_9dbb_ce7c176c7573.slice/crio-eb99e7ba483aa102770172bd7233510e657b9b01ad82a16741eb0ceef8c5c23b WatchSource:0}: Error finding container eb99e7ba483aa102770172bd7233510e657b9b01ad82a16741eb0ceef8c5c23b: Status 404 returned error can't find the container with id eb99e7ba483aa102770172bd7233510e657b9b01ad82a16741eb0ceef8c5c23b
Jul 20 15:57:21.380093 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:21.380056    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-edf0382f-6b5fb6bbf4-57p4v" event={"ID":"b758f404-4d1e-42f4-9dbb-ce7c176c7573","Type":"ContainerStarted","Data":"a90692816ab58bd7ca5ce30bcd37da7b336d8dd26df178c23d6d37c7363f5dd6"}
Jul 20 15:57:21.380193 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:21.380115    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-edf0382f-6b5fb6bbf4-57p4v" event={"ID":"b758f404-4d1e-42f4-9dbb-ce7c176c7573","Type":"ContainerStarted","Data":"eb99e7ba483aa102770172bd7233510e657b9b01ad82a16741eb0ceef8c5c23b"}
Jul 20 15:57:21.380193 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:21.380147    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-edf0382f-6b5fb6bbf4-57p4v"
Jul 20 15:57:21.418375 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:21.418328    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-edf0382f-6b5fb6bbf4-57p4v" podStartSLOduration=1.984815764 podStartE2EDuration="2.41831349s" totalImagesPullingTime="433.497726ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:57:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:57:21.417253878 +0000 UTC m=+1824.426875541" watchObservedRunningTime="2026-07-20 15:57:21.41831349 +0000 UTC m=+1824.427935152"
Jul 20 15:57:24.163905 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:24.163874    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:25.041506 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:25.041478    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:25.287402 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:25.287366    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:25.525119 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:25.525084    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:27.389614 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:27.389588    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-edf0382f-6b5fb6bbf4-57p4v"
Jul 20 15:57:32.419958 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:32.419924    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:33.365398 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:33.365336    2559 patch_prober.go:28] interesting pod/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Jul 20 15:57:33.365398 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:33.365390    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc" podUID="d6edc30c-a27d-4a32-8a92-4e65c17fa1c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Jul 20 15:57:33.433249 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:33.433223    2559 generic.go:356] "Generic (PLEG): container finished" podID="d6edc30c-a27d-4a32-8a92-4e65c17fa1c6" containerID="8e9e89e7f8101d983ac471ce04e322c660a8072801929d23b39616763710dad0" exitCode=0
Jul 20 15:57:33.433633 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:33.433318    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc" event={"ID":"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6","Type":"ContainerDied","Data":"8e9e89e7f8101d983ac471ce04e322c660a8072801929d23b39616763710dad0"}
Jul 20 15:57:33.433700 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:33.433685    2559 scope.go:117] "RemoveContainer" containerID="8e9e89e7f8101d983ac471ce04e322c660a8072801929d23b39616763710dad0"
Jul 20 15:57:33.434701 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:33.434674    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 15:57:33.434773 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:33.434755    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 15:57:33.434824 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:33.434812    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 15:57:34.175934 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:34.175902    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:34.442771 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:34.442672    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc" event={"ID":"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6","Type":"ContainerStarted","Data":"7583186946fa20b1afc3ed78b1c5771018f32900eda5c0262598590c029adc00"}
Jul 20 15:57:35.335367 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:35.335330    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"
Jul 20 15:57:35.335796 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:35.335749    2559 patch_prober.go:28] interesting pod/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Jul 20 15:57:35.335923 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:35.335832    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc" podUID="d6edc30c-a27d-4a32-8a92-4e65c17fa1c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Jul 20 15:57:36.335192 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:36.335155    2559 patch_prober.go:28] interesting pod/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Jul 20 15:57:36.335689 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:36.335237    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc" podUID="d6edc30c-a27d-4a32-8a92-4e65c17fa1c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Jul 20 15:57:37.335726 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:37.335692    2559 patch_prober.go:28] interesting pod/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Jul 20 15:57:37.336304 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:37.336255    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc" podUID="d6edc30c-a27d-4a32-8a92-4e65c17fa1c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Jul 20 15:57:38.340666 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:38.340639    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"
Jul 20 15:57:38.341106 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:38.340979    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"
Jul 20 15:57:38.341735 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:38.341718    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"
Jul 20 15:57:43.339638 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:43.339602    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:43.426529 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:43.426497    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:43.622214 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:43.622135    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-edf0382f-6b5fb6bbf4-57p4v"]
Jul 20 15:57:43.622395 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:43.622372    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-edf0382f-6b5fb6bbf4-57p4v" podUID="b758f404-4d1e-42f4-9dbb-ce7c176c7573" containerName="maas-api" containerID="cri-o://a90692816ab58bd7ca5ce30bcd37da7b336d8dd26df178c23d6d37c7363f5dd6" gracePeriod=30
Jul 20 15:57:43.667894 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:43.667869    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:43.866469 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:43.866448    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-edf0382f-6b5fb6bbf4-57p4v"
Jul 20 15:57:43.931485 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:43.931413    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zch4n\" (UniqueName: \"kubernetes.io/projected/b758f404-4d1e-42f4-9dbb-ce7c176c7573-kube-api-access-zch4n\") pod \"b758f404-4d1e-42f4-9dbb-ce7c176c7573\" (UID: \"b758f404-4d1e-42f4-9dbb-ce7c176c7573\") "
Jul 20 15:57:43.931485 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:43.931463    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b758f404-4d1e-42f4-9dbb-ce7c176c7573-maas-api-tls\") pod \"b758f404-4d1e-42f4-9dbb-ce7c176c7573\" (UID: \"b758f404-4d1e-42f4-9dbb-ce7c176c7573\") "
Jul 20 15:57:43.933551 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:43.933522    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b758f404-4d1e-42f4-9dbb-ce7c176c7573-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b758f404-4d1e-42f4-9dbb-ce7c176c7573" (UID: "b758f404-4d1e-42f4-9dbb-ce7c176c7573"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 15:57:43.933657 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:43.933524    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b758f404-4d1e-42f4-9dbb-ce7c176c7573-kube-api-access-zch4n" (OuterVolumeSpecName: "kube-api-access-zch4n") pod "b758f404-4d1e-42f4-9dbb-ce7c176c7573" (UID: "b758f404-4d1e-42f4-9dbb-ce7c176c7573"). InnerVolumeSpecName "kube-api-access-zch4n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 15:57:44.032555 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:44.032529    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b758f404-4d1e-42f4-9dbb-ce7c176c7573-openshift-service-ca\") pod \"b758f404-4d1e-42f4-9dbb-ce7c176c7573\" (UID: \"b758f404-4d1e-42f4-9dbb-ce7c176c7573\") "
Jul 20 15:57:44.032684 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:44.032677    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zch4n\" (UniqueName: \"kubernetes.io/projected/b758f404-4d1e-42f4-9dbb-ce7c176c7573-kube-api-access-zch4n\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:57:44.032725 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:44.032688    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b758f404-4d1e-42f4-9dbb-ce7c176c7573-maas-api-tls\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:57:44.032858 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:44.032836    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b758f404-4d1e-42f4-9dbb-ce7c176c7573-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b758f404-4d1e-42f4-9dbb-ce7c176c7573" (UID: "b758f404-4d1e-42f4-9dbb-ce7c176c7573"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 15:57:44.133331 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:44.133304    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b758f404-4d1e-42f4-9dbb-ce7c176c7573-openshift-service-ca\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:57:44.491469 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:44.491441    2559 generic.go:356] "Generic (PLEG): container finished" podID="b758f404-4d1e-42f4-9dbb-ce7c176c7573" containerID="a90692816ab58bd7ca5ce30bcd37da7b336d8dd26df178c23d6d37c7363f5dd6" exitCode=0
Jul 20 15:57:44.491837 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:44.491484    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-edf0382f-6b5fb6bbf4-57p4v"
Jul 20 15:57:44.491837 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:44.491494    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-edf0382f-6b5fb6bbf4-57p4v" event={"ID":"b758f404-4d1e-42f4-9dbb-ce7c176c7573","Type":"ContainerDied","Data":"a90692816ab58bd7ca5ce30bcd37da7b336d8dd26df178c23d6d37c7363f5dd6"}
Jul 20 15:57:44.491837 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:44.491533    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-edf0382f-6b5fb6bbf4-57p4v" event={"ID":"b758f404-4d1e-42f4-9dbb-ce7c176c7573","Type":"ContainerDied","Data":"eb99e7ba483aa102770172bd7233510e657b9b01ad82a16741eb0ceef8c5c23b"}
Jul 20 15:57:44.491837 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:44.491554    2559 scope.go:117] "RemoveContainer" containerID="a90692816ab58bd7ca5ce30bcd37da7b336d8dd26df178c23d6d37c7363f5dd6"
Jul 20 15:57:44.500404 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:44.500335    2559 scope.go:117] "RemoveContainer" containerID="a90692816ab58bd7ca5ce30bcd37da7b336d8dd26df178c23d6d37c7363f5dd6"
Jul 20 15:57:44.500640 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:57:44.500618    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a90692816ab58bd7ca5ce30bcd37da7b336d8dd26df178c23d6d37c7363f5dd6\": container with ID starting with a90692816ab58bd7ca5ce30bcd37da7b336d8dd26df178c23d6d37c7363f5dd6 not found: ID does not exist" containerID="a90692816ab58bd7ca5ce30bcd37da7b336d8dd26df178c23d6d37c7363f5dd6"
Jul 20 15:57:44.500687 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:44.500650    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a90692816ab58bd7ca5ce30bcd37da7b336d8dd26df178c23d6d37c7363f5dd6"} err="failed to get container status \"a90692816ab58bd7ca5ce30bcd37da7b336d8dd26df178c23d6d37c7363f5dd6\": rpc error: code = NotFound desc = could not find container \"a90692816ab58bd7ca5ce30bcd37da7b336d8dd26df178c23d6d37c7363f5dd6\": container with ID starting with a90692816ab58bd7ca5ce30bcd37da7b336d8dd26df178c23d6d37c7363f5dd6 not found: ID does not exist"
Jul 20 15:57:44.522146 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:44.522124    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-edf0382f-6b5fb6bbf4-57p4v"]
Jul 20 15:57:44.529085 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:44.529060    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-edf0382f-6b5fb6bbf4-57p4v"]
Jul 20 15:57:45.514524 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:45.514491    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b758f404-4d1e-42f4-9dbb-ce7c176c7573" path="/var/lib/kubelet/pods/b758f404-4d1e-42f4-9dbb-ce7c176c7573/volumes"
Jul 20 15:57:48.032175 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:48.032138    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:48.429196 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:48.429113    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:54.962850 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:54.962813    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:55.080770 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:55.080737    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:55.280576 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:55.280544    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"]
Jul 20 15:57:55.280803 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:55.280769    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc" podUID="d6edc30c-a27d-4a32-8a92-4e65c17fa1c6" containerName="istio-proxy" containerID="cri-o://7583186946fa20b1afc3ed78b1c5771018f32900eda5c0262598590c029adc00" gracePeriod=30
Jul 20 15:57:55.326198 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:55.326171    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:55.535754 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:55.535671    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:55.696184 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:55.696153    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:55.856777 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:55.856684    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:57:55.950850 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:57:55.950818    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:00.533057 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:00.533032    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"
Jul 20 15:58:00.563392 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:00.563333    2559 generic.go:356] "Generic (PLEG): container finished" podID="d6edc30c-a27d-4a32-8a92-4e65c17fa1c6" containerID="7583186946fa20b1afc3ed78b1c5771018f32900eda5c0262598590c029adc00" exitCode=0
Jul 20 15:58:00.563392 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:00.563383    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"
Jul 20 15:58:00.563559 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:00.563416    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc" event={"ID":"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6","Type":"ContainerDied","Data":"7583186946fa20b1afc3ed78b1c5771018f32900eda5c0262598590c029adc00"}
Jul 20 15:58:00.563559 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:00.563475    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc" event={"ID":"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6","Type":"ContainerDied","Data":"5e14e5ef8d7713449fe7dd9d55a86b315375e2ee15444877b23b5b21a33532a2"}
Jul 20 15:58:00.563559 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:00.563509    2559 scope.go:117] "RemoveContainer" containerID="7583186946fa20b1afc3ed78b1c5771018f32900eda5c0262598590c029adc00"
Jul 20 15:58:00.565707 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:00.565688    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-istiod-ca-cert\") pod \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\" (UID: \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\") "
Jul 20 15:58:00.565802 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:00.565742    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-credential-socket\") pod \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\" (UID: \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\") "
Jul 20 15:58:00.565802 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:00.565763    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-workload-certs\") pod \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\" (UID: \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\") "
Jul 20 15:58:00.565802 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:00.565788    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-istio-envoy\") pod \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\" (UID: \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\") "
Jul 20 15:58:00.565945 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:00.565810    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-istio-token\") pod \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\" (UID: \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\") "
Jul 20 15:58:00.565945 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:00.565840    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-istio-podinfo\") pod \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\" (UID: \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\") "
Jul 20 15:58:00.565945 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:00.565896    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-workload-socket\") pod \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\" (UID: \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\") "
Jul 20 15:58:00.566067 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:00.565960    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-istio-data\") pod \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\" (UID: \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\") "
Jul 20 15:58:00.566067 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:00.566024    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpjdn\" (UniqueName: \"kubernetes.io/projected/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-kube-api-access-cpjdn\") pod \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\" (UID: \"d6edc30c-a27d-4a32-8a92-4e65c17fa1c6\") "
Jul 20 15:58:00.566067 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:00.566049    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d6edc30c-a27d-4a32-8a92-4e65c17fa1c6" (UID: "d6edc30c-a27d-4a32-8a92-4e65c17fa1c6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 15:58:00.566221 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:00.566088    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d6edc30c-a27d-4a32-8a92-4e65c17fa1c6" (UID: "d6edc30c-a27d-4a32-8a92-4e65c17fa1c6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 15:58:00.567251 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:00.566276    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d6edc30c-a27d-4a32-8a92-4e65c17fa1c6" (UID: "d6edc30c-a27d-4a32-8a92-4e65c17fa1c6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 15:58:00.567251 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:00.566415    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d6edc30c-a27d-4a32-8a92-4e65c17fa1c6" (UID: "d6edc30c-a27d-4a32-8a92-4e65c17fa1c6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 15:58:00.567251 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:00.566984    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-workload-socket\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:58:00.567251 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:00.567003    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-istiod-ca-cert\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:58:00.567251 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:00.567019    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-credential-socket\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:58:00.567251 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:00.567032    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-workload-certs\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:58:00.567251 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:00.567044    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-istio-data" (OuterVolumeSpecName: "istio-data") pod "d6edc30c-a27d-4a32-8a92-4e65c17fa1c6" (UID: "d6edc30c-a27d-4a32-8a92-4e65c17fa1c6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 15:58:00.568598 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:00.568569    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-istio-token" (OuterVolumeSpecName: "istio-token") pod "d6edc30c-a27d-4a32-8a92-4e65c17fa1c6" (UID: "d6edc30c-a27d-4a32-8a92-4e65c17fa1c6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 15:58:00.568598 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:00.568582    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d6edc30c-a27d-4a32-8a92-4e65c17fa1c6" (UID: "d6edc30c-a27d-4a32-8a92-4e65c17fa1c6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 20 15:58:00.568771 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:00.568582    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d6edc30c-a27d-4a32-8a92-4e65c17fa1c6" (UID: "d6edc30c-a27d-4a32-8a92-4e65c17fa1c6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 15:58:00.569366 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:00.569344    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-kube-api-access-cpjdn" (OuterVolumeSpecName: "kube-api-access-cpjdn") pod "d6edc30c-a27d-4a32-8a92-4e65c17fa1c6" (UID: "d6edc30c-a27d-4a32-8a92-4e65c17fa1c6"). InnerVolumeSpecName "kube-api-access-cpjdn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 15:58:00.573005 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:00.572913    2559 scope.go:117] "RemoveContainer" containerID="8e9e89e7f8101d983ac471ce04e322c660a8072801929d23b39616763710dad0"
Jul 20 15:58:00.580976 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:00.580959    2559 scope.go:117] "RemoveContainer" containerID="7583186946fa20b1afc3ed78b1c5771018f32900eda5c0262598590c029adc00"
Jul 20 15:58:00.581255 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:58:00.581237    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7583186946fa20b1afc3ed78b1c5771018f32900eda5c0262598590c029adc00\": container with ID starting with 7583186946fa20b1afc3ed78b1c5771018f32900eda5c0262598590c029adc00 not found: ID does not exist" containerID="7583186946fa20b1afc3ed78b1c5771018f32900eda5c0262598590c029adc00"
Jul 20 15:58:00.581336 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:00.581263    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7583186946fa20b1afc3ed78b1c5771018f32900eda5c0262598590c029adc00"} err="failed to get container status \"7583186946fa20b1afc3ed78b1c5771018f32900eda5c0262598590c029adc00\": rpc error: code = NotFound desc = could not find container \"7583186946fa20b1afc3ed78b1c5771018f32900eda5c0262598590c029adc00\": container with ID starting with 7583186946fa20b1afc3ed78b1c5771018f32900eda5c0262598590c029adc00 not found: ID does not exist"
Jul 20 15:58:00.581336 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:00.581280    2559 scope.go:117] "RemoveContainer" containerID="8e9e89e7f8101d983ac471ce04e322c660a8072801929d23b39616763710dad0"
Jul 20 15:58:00.581522 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:58:00.581505    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e9e89e7f8101d983ac471ce04e322c660a8072801929d23b39616763710dad0\": container with ID starting with 8e9e89e7f8101d983ac471ce04e322c660a8072801929d23b39616763710dad0 not found: ID does not exist" containerID="8e9e89e7f8101d983ac471ce04e322c660a8072801929d23b39616763710dad0"
Jul 20 15:58:00.581566 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:00.581526    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e9e89e7f8101d983ac471ce04e322c660a8072801929d23b39616763710dad0"} err="failed to get container status \"8e9e89e7f8101d983ac471ce04e322c660a8072801929d23b39616763710dad0\": rpc error: code = NotFound desc = could not find container \"8e9e89e7f8101d983ac471ce04e322c660a8072801929d23b39616763710dad0\": container with ID starting with 8e9e89e7f8101d983ac471ce04e322c660a8072801929d23b39616763710dad0 not found: ID does not exist"
Jul 20 15:58:00.667857 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:00.667836    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-istio-data\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:58:00.667857 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:00.667856    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpjdn\" (UniqueName: \"kubernetes.io/projected/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-kube-api-access-cpjdn\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:58:00.667970 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:00.667866    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-istio-envoy\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:58:00.667970 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:00.667875    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-istio-token\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:58:00.667970 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:00.667883    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6-istio-podinfo\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:58:00.888935 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:00.888908    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"]
Jul 20 15:58:00.893930 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:00.893911    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-edf0382f-openshift-default-5dddc88b99-pjzcc"]
Jul 20 15:58:01.514277 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:01.514250    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6edc30c-a27d-4a32-8a92-4e65c17fa1c6" path="/var/lib/kubelet/pods/d6edc30c-a27d-4a32-8a92-4e65c17fa1c6/volumes"
Jul 20 15:58:02.294059 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:02.294030    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:02.419179 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:02.419147    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:02.524749 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:02.524714    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:02.658734 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:02.658643    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:09.620940 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:09.620904    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:09.766913 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:09.766877    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:18.657694 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:18.657652    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:18.909205 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:18.909109    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"]
Jul 20 15:58:18.909525 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:18.909509    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6edc30c-a27d-4a32-8a92-4e65c17fa1c6" containerName="istio-proxy"
Jul 20 15:58:18.909566 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:18.909526    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6edc30c-a27d-4a32-8a92-4e65c17fa1c6" containerName="istio-proxy"
Jul 20 15:58:18.909566 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:18.909534    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b758f404-4d1e-42f4-9dbb-ce7c176c7573" containerName="maas-api"
Jul 20 15:58:18.909566 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:18.909539    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b758f404-4d1e-42f4-9dbb-ce7c176c7573" containerName="maas-api"
Jul 20 15:58:18.909566 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:18.909545    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6edc30c-a27d-4a32-8a92-4e65c17fa1c6" containerName="istio-proxy"
Jul 20 15:58:18.909566 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:18.909550    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6edc30c-a27d-4a32-8a92-4e65c17fa1c6" containerName="istio-proxy"
Jul 20 15:58:18.909730 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:18.909614    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b758f404-4d1e-42f4-9dbb-ce7c176c7573" containerName="maas-api"
Jul 20 15:58:18.909730 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:18.909624    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6edc30c-a27d-4a32-8a92-4e65c17fa1c6" containerName="istio-proxy"
Jul 20 15:58:18.909730 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:18.909631    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6edc30c-a27d-4a32-8a92-4e65c17fa1c6" containerName="istio-proxy"
Jul 20 15:58:18.913696 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:18.913675    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"
Jul 20 15:58:18.916551 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:18.916526    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 20 15:58:18.916851 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:18.916833    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-c7baa3ee-openshift-default-dockercfg-kvwds\""
Jul 20 15:58:18.932474 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:18.932442    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"]
Jul 20 15:58:18.944098 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:18.944076    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:19.021277 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:19.021255    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b564f59a-3215-4735-98b9-ca72b06ab84a-workload-certs\") pod \"e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j\" (UID: \"b564f59a-3215-4735-98b9-ca72b06ab84a\") " pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"
Jul 20 15:58:19.021412 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:19.021282    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b564f59a-3215-4735-98b9-ca72b06ab84a-istio-data\") pod \"e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j\" (UID: \"b564f59a-3215-4735-98b9-ca72b06ab84a\") " pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"
Jul 20 15:58:19.021412 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:19.021322    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b564f59a-3215-4735-98b9-ca72b06ab84a-istio-envoy\") pod \"e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j\" (UID: \"b564f59a-3215-4735-98b9-ca72b06ab84a\") " pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"
Jul 20 15:58:19.021412 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:19.021342    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b564f59a-3215-4735-98b9-ca72b06ab84a-istio-token\") pod \"e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j\" (UID: \"b564f59a-3215-4735-98b9-ca72b06ab84a\") " pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"
Jul 20 15:58:19.021587 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:19.021420    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xm79f\" (UniqueName: \"kubernetes.io/projected/b564f59a-3215-4735-98b9-ca72b06ab84a-kube-api-access-xm79f\") pod \"e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j\" (UID: \"b564f59a-3215-4735-98b9-ca72b06ab84a\") " pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"
Jul 20 15:58:19.021587 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:19.021529    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b564f59a-3215-4735-98b9-ca72b06ab84a-workload-socket\") pod \"e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j\" (UID: \"b564f59a-3215-4735-98b9-ca72b06ab84a\") " pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"
Jul 20 15:58:19.021587 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:19.021556    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b564f59a-3215-4735-98b9-ca72b06ab84a-istio-podinfo\") pod \"e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j\" (UID: \"b564f59a-3215-4735-98b9-ca72b06ab84a\") " pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"
Jul 20 15:58:19.021719 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:19.021589    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b564f59a-3215-4735-98b9-ca72b06ab84a-credential-socket\") pod \"e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j\" (UID: \"b564f59a-3215-4735-98b9-ca72b06ab84a\") " pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"
Jul 20 15:58:19.021719 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:19.021614    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b564f59a-3215-4735-98b9-ca72b06ab84a-istiod-ca-cert\") pod \"e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j\" (UID: \"b564f59a-3215-4735-98b9-ca72b06ab84a\") " pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"
Jul 20 15:58:19.122557 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:19.122532    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b564f59a-3215-4735-98b9-ca72b06ab84a-workload-socket\") pod \"e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j\" (UID: \"b564f59a-3215-4735-98b9-ca72b06ab84a\") " pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"
Jul 20 15:58:19.122651 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:19.122561    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b564f59a-3215-4735-98b9-ca72b06ab84a-istio-podinfo\") pod \"e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j\" (UID: \"b564f59a-3215-4735-98b9-ca72b06ab84a\") " pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"
Jul 20 15:58:19.122651 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:19.122585    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b564f59a-3215-4735-98b9-ca72b06ab84a-credential-socket\") pod \"e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j\" (UID: \"b564f59a-3215-4735-98b9-ca72b06ab84a\") " pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"
Jul 20 15:58:19.122651 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:19.122599    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b564f59a-3215-4735-98b9-ca72b06ab84a-istiod-ca-cert\") pod \"e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j\" (UID: \"b564f59a-3215-4735-98b9-ca72b06ab84a\") " pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"
Jul 20 15:58:19.122651 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:19.122631    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b564f59a-3215-4735-98b9-ca72b06ab84a-workload-certs\") pod \"e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j\" (UID: \"b564f59a-3215-4735-98b9-ca72b06ab84a\") " pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"
Jul 20 15:58:19.122651 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:19.122649    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b564f59a-3215-4735-98b9-ca72b06ab84a-istio-data\") pod \"e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j\" (UID: \"b564f59a-3215-4735-98b9-ca72b06ab84a\") " pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"
Jul 20 15:58:19.122872 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:19.122672    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b564f59a-3215-4735-98b9-ca72b06ab84a-istio-envoy\") pod \"e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j\" (UID: \"b564f59a-3215-4735-98b9-ca72b06ab84a\") " pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"
Jul 20 15:58:19.122872 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:19.122699    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b564f59a-3215-4735-98b9-ca72b06ab84a-istio-token\") pod \"e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j\" (UID: \"b564f59a-3215-4735-98b9-ca72b06ab84a\") " pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"
Jul 20 15:58:19.122872 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:19.122746    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm79f\" (UniqueName: \"kubernetes.io/projected/b564f59a-3215-4735-98b9-ca72b06ab84a-kube-api-access-xm79f\") pod \"e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j\" (UID: \"b564f59a-3215-4735-98b9-ca72b06ab84a\") " pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"
Jul 20 15:58:19.123031 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:19.122929    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b564f59a-3215-4735-98b9-ca72b06ab84a-credential-socket\") pod \"e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j\" (UID: \"b564f59a-3215-4735-98b9-ca72b06ab84a\") " pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"
Jul 20 15:58:19.123031 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:19.122995    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b564f59a-3215-4735-98b9-ca72b06ab84a-istio-data\") pod \"e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j\" (UID: \"b564f59a-3215-4735-98b9-ca72b06ab84a\") " pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"
Jul 20 15:58:19.123031 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:19.123025    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b564f59a-3215-4735-98b9-ca72b06ab84a-workload-socket\") pod \"e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j\" (UID: \"b564f59a-3215-4735-98b9-ca72b06ab84a\") " pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"
Jul 20 15:58:19.123236 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:19.123214    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b564f59a-3215-4735-98b9-ca72b06ab84a-workload-certs\") pod \"e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j\" (UID: \"b564f59a-3215-4735-98b9-ca72b06ab84a\") " pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"
Jul 20 15:58:19.123420 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:19.123403    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b564f59a-3215-4735-98b9-ca72b06ab84a-istiod-ca-cert\") pod \"e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j\" (UID: \"b564f59a-3215-4735-98b9-ca72b06ab84a\") " pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"
Jul 20 15:58:19.124946 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:19.124920    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b564f59a-3215-4735-98b9-ca72b06ab84a-istio-podinfo\") pod \"e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j\" (UID: \"b564f59a-3215-4735-98b9-ca72b06ab84a\") " pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"
Jul 20 15:58:19.125051 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:19.124963    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b564f59a-3215-4735-98b9-ca72b06ab84a-istio-envoy\") pod \"e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j\" (UID: \"b564f59a-3215-4735-98b9-ca72b06ab84a\") " pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"
Jul 20 15:58:19.135554 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:19.135528    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xm79f\" (UniqueName: \"kubernetes.io/projected/b564f59a-3215-4735-98b9-ca72b06ab84a-kube-api-access-xm79f\") pod \"e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j\" (UID: \"b564f59a-3215-4735-98b9-ca72b06ab84a\") " pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"
Jul 20 15:58:19.137064 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:19.137044    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b564f59a-3215-4735-98b9-ca72b06ab84a-istio-token\") pod \"e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j\" (UID: \"b564f59a-3215-4735-98b9-ca72b06ab84a\") " pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"
Jul 20 15:58:19.152579 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:19.152551    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:19.223715 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:19.223657    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"
Jul 20 15:58:19.379023 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:19.379001    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"]
Jul 20 15:58:19.381141 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:58:19.381115    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb564f59a_3215_4735_98b9_ca72b06ab84a.slice/crio-92bf186b0528d007050c41b07896ae023ba36656fa418958329c5f4d0f524571 WatchSource:0}: Error finding container 92bf186b0528d007050c41b07896ae023ba36656fa418958329c5f4d0f524571: Status 404 returned error can't find the container with id 92bf186b0528d007050c41b07896ae023ba36656fa418958329c5f4d0f524571
Jul 20 15:58:19.383188 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:19.383157    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 15:58:19.383315 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:19.383242    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 15:58:19.383379 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:19.383324    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 15:58:19.652128 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:19.652096    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j" event={"ID":"b564f59a-3215-4735-98b9-ca72b06ab84a","Type":"ContainerStarted","Data":"fc4abab2ab272211424a1b88f1a01c18cdac619b8cb4a676668ae329f78e38a8"}
Jul 20 15:58:19.652343 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:19.652152    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j" event={"ID":"b564f59a-3215-4735-98b9-ca72b06ab84a","Type":"ContainerStarted","Data":"92bf186b0528d007050c41b07896ae023ba36656fa418958329c5f4d0f524571"}
Jul 20 15:58:19.673461 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:19.673410    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j" podStartSLOduration=1.6733949190000001 podStartE2EDuration="1.673394919s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:58:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:58:19.671852724 +0000 UTC m=+1882.681474389" watchObservedRunningTime="2026-07-20 15:58:19.673394919 +0000 UTC m=+1882.683016581"
Jul 20 15:58:20.224809 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:20.224780    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"
Jul 20 15:58:20.229796 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:20.229774    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"
Jul 20 15:58:20.657431 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:20.657401    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"
Jul 20 15:58:20.658207 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:20.658190    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"
Jul 20 15:58:21.639095 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:21.639056    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:21.749255 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:21.749227    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c7baa3ee-7858f89687-5zmnx"]
Jul 20 15:58:21.752612 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:21.752591    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7baa3ee-7858f89687-5zmnx"
Jul 20 15:58:21.755583 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:21.755561    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-c7baa3ee-serving-cert\""
Jul 20 15:58:21.756085 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:21.756069    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-96kqc\""
Jul 20 15:58:21.789840 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:21.789810    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:21.791506 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:21.791487    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c7baa3ee-7858f89687-5zmnx"]
Jul 20 15:58:21.848271 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:21.848236    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cbe2efc-88fb-473c-abed-26b9fa4fb62a-maas-api-tls\") pod \"maas-api-e2e-mt-c7baa3ee-7858f89687-5zmnx\" (UID: \"1cbe2efc-88fb-473c-abed-26b9fa4fb62a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7baa3ee-7858f89687-5zmnx"
Jul 20 15:58:21.848271 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:21.848310    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cbe2efc-88fb-473c-abed-26b9fa4fb62a-openshift-service-ca\") pod \"maas-api-e2e-mt-c7baa3ee-7858f89687-5zmnx\" (UID: \"1cbe2efc-88fb-473c-abed-26b9fa4fb62a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7baa3ee-7858f89687-5zmnx"
Jul 20 15:58:21.848531 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:21.848359    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sr5z5\" (UniqueName: \"kubernetes.io/projected/1cbe2efc-88fb-473c-abed-26b9fa4fb62a-kube-api-access-sr5z5\") pod \"maas-api-e2e-mt-c7baa3ee-7858f89687-5zmnx\" (UID: \"1cbe2efc-88fb-473c-abed-26b9fa4fb62a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7baa3ee-7858f89687-5zmnx"
Jul 20 15:58:21.871418 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:21.871384    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:21.949670 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:21.949595    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cbe2efc-88fb-473c-abed-26b9fa4fb62a-openshift-service-ca\") pod \"maas-api-e2e-mt-c7baa3ee-7858f89687-5zmnx\" (UID: \"1cbe2efc-88fb-473c-abed-26b9fa4fb62a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7baa3ee-7858f89687-5zmnx"
Jul 20 15:58:21.949670 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:21.949636    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr5z5\" (UniqueName: \"kubernetes.io/projected/1cbe2efc-88fb-473c-abed-26b9fa4fb62a-kube-api-access-sr5z5\") pod \"maas-api-e2e-mt-c7baa3ee-7858f89687-5zmnx\" (UID: \"1cbe2efc-88fb-473c-abed-26b9fa4fb62a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7baa3ee-7858f89687-5zmnx"
Jul 20 15:58:21.949828 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:21.949700    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cbe2efc-88fb-473c-abed-26b9fa4fb62a-maas-api-tls\") pod \"maas-api-e2e-mt-c7baa3ee-7858f89687-5zmnx\" (UID: \"1cbe2efc-88fb-473c-abed-26b9fa4fb62a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7baa3ee-7858f89687-5zmnx"
Jul 20 15:58:21.949828 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:58:21.949790    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-c7baa3ee-serving-cert: secret "maas-api-e2e-mt-c7baa3ee-serving-cert" not found
Jul 20 15:58:21.949891 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:58:21.949848    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cbe2efc-88fb-473c-abed-26b9fa4fb62a-maas-api-tls podName:1cbe2efc-88fb-473c-abed-26b9fa4fb62a nodeName:}" failed. No retries permitted until 2026-07-20 15:58:22.449832547 +0000 UTC m=+1885.459454187 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1cbe2efc-88fb-473c-abed-26b9fa4fb62a-maas-api-tls") pod "maas-api-e2e-mt-c7baa3ee-7858f89687-5zmnx" (UID: "1cbe2efc-88fb-473c-abed-26b9fa4fb62a") : secret "maas-api-e2e-mt-c7baa3ee-serving-cert" not found
Jul 20 15:58:21.950205 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:21.950188    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cbe2efc-88fb-473c-abed-26b9fa4fb62a-openshift-service-ca\") pod \"maas-api-e2e-mt-c7baa3ee-7858f89687-5zmnx\" (UID: \"1cbe2efc-88fb-473c-abed-26b9fa4fb62a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7baa3ee-7858f89687-5zmnx"
Jul 20 15:58:21.996740 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:21.996713    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sr5z5\" (UniqueName: \"kubernetes.io/projected/1cbe2efc-88fb-473c-abed-26b9fa4fb62a-kube-api-access-sr5z5\") pod \"maas-api-e2e-mt-c7baa3ee-7858f89687-5zmnx\" (UID: \"1cbe2efc-88fb-473c-abed-26b9fa4fb62a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7baa3ee-7858f89687-5zmnx"
Jul 20 15:58:22.204230 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:22.204146    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:22.455099 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:22.455003    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cbe2efc-88fb-473c-abed-26b9fa4fb62a-maas-api-tls\") pod \"maas-api-e2e-mt-c7baa3ee-7858f89687-5zmnx\" (UID: \"1cbe2efc-88fb-473c-abed-26b9fa4fb62a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7baa3ee-7858f89687-5zmnx"
Jul 20 15:58:22.457331 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:22.457306    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cbe2efc-88fb-473c-abed-26b9fa4fb62a-maas-api-tls\") pod \"maas-api-e2e-mt-c7baa3ee-7858f89687-5zmnx\" (UID: \"1cbe2efc-88fb-473c-abed-26b9fa4fb62a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7baa3ee-7858f89687-5zmnx"
Jul 20 15:58:22.482477 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:22.482446    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:22.632560 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:22.632523    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:22.662440 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:22.662415    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7baa3ee-7858f89687-5zmnx"
Jul 20 15:58:22.811004 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:22.810984    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c7baa3ee-7858f89687-5zmnx"]
Jul 20 15:58:22.812846 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:58:22.812822    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cbe2efc_88fb_473c_abed_26b9fa4fb62a.slice/crio-d7a8bf69a2192f7d18257dba6f98db180ec164e97cdcfa973eeb78db80fd504e WatchSource:0}: Error finding container d7a8bf69a2192f7d18257dba6f98db180ec164e97cdcfa973eeb78db80fd504e: Status 404 returned error can't find the container with id d7a8bf69a2192f7d18257dba6f98db180ec164e97cdcfa973eeb78db80fd504e
Jul 20 15:58:22.814299 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:22.814275    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 15:58:23.672645 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:23.672610    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7baa3ee-7858f89687-5zmnx" event={"ID":"1cbe2efc-88fb-473c-abed-26b9fa4fb62a","Type":"ContainerStarted","Data":"6a72e016bd5525cee92a1df5d8819b5fd524d0a75749e5dd64d1c4d5ae63e3fb"}
Jul 20 15:58:23.673029 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:23.672667    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7baa3ee-7858f89687-5zmnx"
Jul 20 15:58:23.673029 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:23.672685    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7baa3ee-7858f89687-5zmnx" event={"ID":"1cbe2efc-88fb-473c-abed-26b9fa4fb62a","Type":"ContainerStarted","Data":"d7a8bf69a2192f7d18257dba6f98db180ec164e97cdcfa973eeb78db80fd504e"}
Jul 20 15:58:23.691641 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:23.691590    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7baa3ee-7858f89687-5zmnx" podStartSLOduration=2.308646649 podStartE2EDuration="2.691575435s" totalImagesPullingTime="382.928786ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:58:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:58:23.690024749 +0000 UTC m=+1886.699646412" watchObservedRunningTime="2026-07-20 15:58:23.691575435 +0000 UTC m=+1886.701197096"
Jul 20 15:58:26.139217 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:26.139182    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:29.682959 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:29.682932    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7baa3ee-7858f89687-5zmnx"
Jul 20 15:58:30.997610 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:30.997577    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-c7baa3ee/e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg"]
Jul 20 15:58:31.006778 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:31.006751    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c7baa3ee/e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg"
Jul 20 15:58:31.010680 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:31.010645    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-c7baa3ee\"/\"kube-root-ca.crt\""
Jul 20 15:58:31.010680 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:31.010664    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-c7baa3ee/e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg"]
Jul 20 15:58:31.010872 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:31.010751    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-c7baa3ee\"/\"openshift-service-ca.crt\""
Jul 20 15:58:31.010872 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:31.010776    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-c7baa3ee\"/\"default-dockercfg-q64cw\""
Jul 20 15:58:31.010872 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:31.010643    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-c7baa3ee\"/\"e2e-lifecycle-model-c7baa3ee-kserve-self-signed-certs\""
Jul 20 15:58:31.135811 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:31.135771    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de3312e1-d9e4-478b-b862-d0b750717e01-tmp-dir\") pod \"e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg\" (UID: \"de3312e1-d9e4-478b-b862-d0b750717e01\") " pod="ai-tenant-e2e-mt-c7baa3ee/e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg"
Jul 20 15:58:31.135999 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:31.135834    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de3312e1-d9e4-478b-b862-d0b750717e01-model-cache\") pod \"e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg\" (UID: \"de3312e1-d9e4-478b-b862-d0b750717e01\") " pod="ai-tenant-e2e-mt-c7baa3ee/e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg"
Jul 20 15:58:31.135999 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:31.135883    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxlxx\" (UniqueName: \"kubernetes.io/projected/de3312e1-d9e4-478b-b862-d0b750717e01-kube-api-access-zxlxx\") pod \"e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg\" (UID: \"de3312e1-d9e4-478b-b862-d0b750717e01\") " pod="ai-tenant-e2e-mt-c7baa3ee/e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg"
Jul 20 15:58:31.135999 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:31.135979    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de3312e1-d9e4-478b-b862-d0b750717e01-dshm\") pod \"e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg\" (UID: \"de3312e1-d9e4-478b-b862-d0b750717e01\") " pod="ai-tenant-e2e-mt-c7baa3ee/e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg"
Jul 20 15:58:31.136187 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:31.136076    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de3312e1-d9e4-478b-b862-d0b750717e01-home\") pod \"e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg\" (UID: \"de3312e1-d9e4-478b-b862-d0b750717e01\") " pod="ai-tenant-e2e-mt-c7baa3ee/e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg"
Jul 20 15:58:31.136187 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:31.136117    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de3312e1-d9e4-478b-b862-d0b750717e01-tls-certs\") pod \"e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg\" (UID: \"de3312e1-d9e4-478b-b862-d0b750717e01\") " pod="ai-tenant-e2e-mt-c7baa3ee/e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg"
Jul 20 15:58:31.197367 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:31.197338    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:58:31.240779 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:31.240743    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de3312e1-d9e4-478b-b862-d0b750717e01-home\") pod \"e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg\" (UID: \"de3312e1-d9e4-478b-b862-d0b750717e01\") " pod="ai-tenant-e2e-mt-c7baa3ee/e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg"
Jul 20 15:58:31.241475 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:31.241456    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de3312e1-d9e4-478b-b862-d0b750717e01-tls-certs\") pod \"e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg\" (UID: \"de3312e1-d9e4-478b-b862-d0b750717e01\") " pod="ai-tenant-e2e-mt-c7baa3ee/e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg"
Jul 20 15:58:31.242047 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:31.242028    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de3312e1-d9e4-478b-b862-d0b750717e01-tmp-dir\") pod \"e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg\" (UID: \"de3312e1-d9e4-478b-b862-d0b750717e01\") " pod="ai-tenant-e2e-mt-c7baa3ee/e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg"
Jul 20 15:58:31.242205 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:31.242188    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de3312e1-d9e4-478b-b862-d0b750717e01-model-cache\") pod \"e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg\" (UID: \"de3312e1-d9e4-478b-b862-d0b750717e01\") " pod="ai-tenant-e2e-mt-c7baa3ee/e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg"
Jul 20 15:58:31.242378 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:31.242359    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxlxx\" (UniqueName: \"kubernetes.io/projected/de3312e1-d9e4-478b-b862-d0b750717e01-kube-api-access-zxlxx\") pod \"e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg\" (UID: \"de3312e1-d9e4-478b-b862-d0b750717e01\") " pod="ai-tenant-e2e-mt-c7baa3ee/e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg"
Jul 20 15:58:31.242535 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:31.242520    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de3312e1-d9e4-478b-b862-d0b750717e01-dshm\") pod \"e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg\" (UID: \"de3312e1-d9e4-478b-b862-d0b750717e01\") " pod="ai-tenant-e2e-mt-c7baa3ee/e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg"
Jul 20 15:58:31.245791 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:31.245770    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de3312e1-d9e4-478b-b862-d0b750717e01-tls-certs\") pod \"e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg\" (UID: \"de3312e1-d9e4-478b-b862-d0b750717e01\") " pod="ai-tenant-e2e-mt-c7baa3ee/e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg"
Jul 20 15:58:31.246170 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:31.246150    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de3312e1-d9e4-478b-b862-d0b750717e01-tmp-dir\") pod \"e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg\" (UID: \"de3312e1-d9e4-478b-b862-d0b750717e01\") " pod="ai-tenant-e2e-mt-c7baa3ee/e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg"
Jul 20 15:58:31.246316 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:31.241392    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de3312e1-d9e4-478b-b862-d0b750717e01-home\") pod \"e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg\" (UID: \"de3312e1-d9e4-478b-b862-d0b750717e01\") " pod="ai-tenant-e2e-mt-c7baa3ee/e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg"
Jul 20 15:58:31.246661 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:31.246642    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de3312e1-d9e4-478b-b862-d0b750717e01-model-cache\") pod \"e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg\" (UID: \"de3312e1-d9e4-478b-b862-d0b750717e01\") " pod="ai-tenant-e2e-mt-c7baa3ee/e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg"
Jul 20 15:58:31.251567 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:31.251514    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de3312e1-d9e4-478b-b862-d0b750717e01-dshm\") pod \"e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg\" (UID: \"de3312e1-d9e4-478b-b862-d0b750717e01\") " pod="ai-tenant-e2e-mt-c7baa3ee/e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg"
Jul 20 15:58:31.256171 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:31.256150    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxlxx\" (UniqueName: \"kubernetes.io/projected/de3312e1-d9e4-478b-b862-d0b750717e01-kube-api-access-zxlxx\") pod \"e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg\" (UID: \"de3312e1-d9e4-478b-b862-d0b750717e01\") " pod="ai-tenant-e2e-mt-c7baa3ee/e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg"
Jul 20 15:58:31.318150 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:31.318120    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c7baa3ee/e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg"
Jul 20 15:58:31.493162 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:31.493128    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-c7baa3ee/e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg"]
Jul 20 15:58:31.494876 ip-10-0-130-19 kubenswrapper[2559]: W0720 15:58:31.494848    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podde3312e1_d9e4_478b_b862_d0b750717e01.slice/crio-1d71600541c5aaa4b0eea1ac70531b4e60d43c65eccb4e1e33846cb1297c8284 WatchSource:0}: Error finding container 1d71600541c5aaa4b0eea1ac70531b4e60d43c65eccb4e1e33846cb1297c8284: Status 404 returned error can't find the container with id 1d71600541c5aaa4b0eea1ac70531b4e60d43c65eccb4e1e33846cb1297c8284
Jul 20 15:58:31.711178 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:31.711134    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c7baa3ee/e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg" event={"ID":"de3312e1-d9e4-478b-b862-d0b750717e01","Type":"ContainerStarted","Data":"60ee9ad0e2f439ab9e696903da3d3f88104a28077f5d622ddb2107c63850b1d4"}
Jul 20 15:58:31.711178 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:31.711181    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c7baa3ee/e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg" event={"ID":"de3312e1-d9e4-478b-b862-d0b750717e01","Type":"ContainerStarted","Data":"1d71600541c5aaa4b0eea1ac70531b4e60d43c65eccb4e1e33846cb1297c8284"}
Jul 20 15:58:31.727808 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:31.727738    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-c7baa3ee/e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg" podStartSLOduration=1.727719059 podStartE2EDuration="1.727719059s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:58:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:58:31.727387017 +0000 UTC m=+1894.737008679" watchObservedRunningTime="2026-07-20 15:58:31.727719059 +0000 UTC m=+1894.737340720"
Jul 20 15:58:36.201595 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:36.200875    2559 patch_prober.go:28] interesting pod/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": read tcp 10.134.0.2:53472->10.134.0.45:15021: read: connection reset by peer" start-of-body=
Jul 20 15:58:36.201595 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:36.200939    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j" podUID="b564f59a-3215-4735-98b9-ca72b06ab84a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": read tcp 10.134.0.2:53472->10.134.0.45:15021: read: connection reset by peer"
Jul 20 15:58:36.736541 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:36.736513    2559 generic.go:356] "Generic (PLEG): container finished" podID="b564f59a-3215-4735-98b9-ca72b06ab84a" containerID="fc4abab2ab272211424a1b88f1a01c18cdac619b8cb4a676668ae329f78e38a8" exitCode=0
Jul 20 15:58:36.736711 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:36.736588    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j" event={"ID":"b564f59a-3215-4735-98b9-ca72b06ab84a","Type":"ContainerDied","Data":"fc4abab2ab272211424a1b88f1a01c18cdac619b8cb4a676668ae329f78e38a8"}
Jul 20 15:58:36.737094 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:36.737076    2559 scope.go:117] "RemoveContainer" containerID="fc4abab2ab272211424a1b88f1a01c18cdac619b8cb4a676668ae329f78e38a8"
Jul 20 15:58:36.738064 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:36.738038    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 15:58:36.738122 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:36.738109    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 15:58:36.738159 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:36.738142    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 15:58:37.746517 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:37.746478    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j" event={"ID":"b564f59a-3215-4735-98b9-ca72b06ab84a","Type":"ContainerStarted","Data":"5cf6d02cb9f71e2e26fd9bb922616eb8a6b0b542909bdf02eb9c8cb7736bf991"}
Jul 20 15:58:38.224585 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:38.224550    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"
Jul 20 15:58:38.224819 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:38.224795    2559 patch_prober.go:28] interesting pod/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 20 15:58:38.224874 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:38.224846    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j" podUID="b564f59a-3215-4735-98b9-ca72b06ab84a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 20 15:58:39.224669 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:39.224637    2559 patch_prober.go:28] interesting pod/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 20 15:58:39.225052 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:39.224691    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j" podUID="b564f59a-3215-4735-98b9-ca72b06ab84a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 20 15:58:40.224179 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:40.224144    2559 patch_prober.go:28] interesting pod/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 20 15:58:40.224384 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:40.224200    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j" podUID="b564f59a-3215-4735-98b9-ca72b06ab84a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 20 15:58:41.224978 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:41.224944    2559 patch_prober.go:28] interesting pod/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 20 15:58:41.225421 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:41.224999    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j" podUID="b564f59a-3215-4735-98b9-ca72b06ab84a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 20 15:58:41.318564 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:41.318536    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-c7baa3ee/e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg"
Jul 20 15:58:41.318714 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:41.318619    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-c7baa3ee/e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg"
Jul 20 15:58:41.355949 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:41.355917    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-c7baa3ee/e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg"
Jul 20 15:58:41.778260 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:41.778235    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-c7baa3ee/e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg"
Jul 20 15:58:42.228161 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:42.228069    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"
Jul 20 15:58:42.228537 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:42.228320    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"
Jul 20 15:58:42.229030 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:58:42.229012    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"
Jul 20 15:59:09.925803 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:09.925766    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c7baa3ee-7858f89687-5zmnx"]
Jul 20 15:59:09.926199 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:09.925974    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7baa3ee-7858f89687-5zmnx" podUID="1cbe2efc-88fb-473c-abed-26b9fa4fb62a" containerName="maas-api" containerID="cri-o://6a72e016bd5525cee92a1df5d8819b5fd524d0a75749e5dd64d1c4d5ae63e3fb" gracePeriod=30
Jul 20 15:59:09.961596 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:09.961570    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:59:10.160878 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:10.160854    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7baa3ee-7858f89687-5zmnx"
Jul 20 15:59:10.294320 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:10.294278    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cbe2efc-88fb-473c-abed-26b9fa4fb62a-openshift-service-ca\") pod \"1cbe2efc-88fb-473c-abed-26b9fa4fb62a\" (UID: \"1cbe2efc-88fb-473c-abed-26b9fa4fb62a\") "
Jul 20 15:59:10.294481 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:10.294376    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cbe2efc-88fb-473c-abed-26b9fa4fb62a-maas-api-tls\") pod \"1cbe2efc-88fb-473c-abed-26b9fa4fb62a\" (UID: \"1cbe2efc-88fb-473c-abed-26b9fa4fb62a\") "
Jul 20 15:59:10.294481 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:10.294411    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sr5z5\" (UniqueName: \"kubernetes.io/projected/1cbe2efc-88fb-473c-abed-26b9fa4fb62a-kube-api-access-sr5z5\") pod \"1cbe2efc-88fb-473c-abed-26b9fa4fb62a\" (UID: \"1cbe2efc-88fb-473c-abed-26b9fa4fb62a\") "
Jul 20 15:59:10.294677 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:10.294654    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cbe2efc-88fb-473c-abed-26b9fa4fb62a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1cbe2efc-88fb-473c-abed-26b9fa4fb62a" (UID: "1cbe2efc-88fb-473c-abed-26b9fa4fb62a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 15:59:10.296363 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:10.296336    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cbe2efc-88fb-473c-abed-26b9fa4fb62a-kube-api-access-sr5z5" (OuterVolumeSpecName: "kube-api-access-sr5z5") pod "1cbe2efc-88fb-473c-abed-26b9fa4fb62a" (UID: "1cbe2efc-88fb-473c-abed-26b9fa4fb62a"). InnerVolumeSpecName "kube-api-access-sr5z5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 15:59:10.296469 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:10.296430    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cbe2efc-88fb-473c-abed-26b9fa4fb62a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1cbe2efc-88fb-473c-abed-26b9fa4fb62a" (UID: "1cbe2efc-88fb-473c-abed-26b9fa4fb62a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 15:59:10.395375 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:10.395351    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cbe2efc-88fb-473c-abed-26b9fa4fb62a-maas-api-tls\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:59:10.395375 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:10.395373    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sr5z5\" (UniqueName: \"kubernetes.io/projected/1cbe2efc-88fb-473c-abed-26b9fa4fb62a-kube-api-access-sr5z5\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:59:10.395515 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:10.395385    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cbe2efc-88fb-473c-abed-26b9fa4fb62a-openshift-service-ca\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:59:10.900723 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:10.900691    2559 generic.go:356] "Generic (PLEG): container finished" podID="1cbe2efc-88fb-473c-abed-26b9fa4fb62a" containerID="6a72e016bd5525cee92a1df5d8819b5fd524d0a75749e5dd64d1c4d5ae63e3fb" exitCode=0
Jul 20 15:59:10.900906 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:10.900734    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7baa3ee-7858f89687-5zmnx"
Jul 20 15:59:10.900906 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:10.900759    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7baa3ee-7858f89687-5zmnx" event={"ID":"1cbe2efc-88fb-473c-abed-26b9fa4fb62a","Type":"ContainerDied","Data":"6a72e016bd5525cee92a1df5d8819b5fd524d0a75749e5dd64d1c4d5ae63e3fb"}
Jul 20 15:59:10.900906 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:10.900801    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7baa3ee-7858f89687-5zmnx" event={"ID":"1cbe2efc-88fb-473c-abed-26b9fa4fb62a","Type":"ContainerDied","Data":"d7a8bf69a2192f7d18257dba6f98db180ec164e97cdcfa973eeb78db80fd504e"}
Jul 20 15:59:10.900906 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:10.900822    2559 scope.go:117] "RemoveContainer" containerID="6a72e016bd5525cee92a1df5d8819b5fd524d0a75749e5dd64d1c4d5ae63e3fb"
Jul 20 15:59:10.909734 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:10.909716    2559 scope.go:117] "RemoveContainer" containerID="6a72e016bd5525cee92a1df5d8819b5fd524d0a75749e5dd64d1c4d5ae63e3fb"
Jul 20 15:59:10.910001 ip-10-0-130-19 kubenswrapper[2559]: E0720 15:59:10.909982    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a72e016bd5525cee92a1df5d8819b5fd524d0a75749e5dd64d1c4d5ae63e3fb\": container with ID starting with 6a72e016bd5525cee92a1df5d8819b5fd524d0a75749e5dd64d1c4d5ae63e3fb not found: ID does not exist" containerID="6a72e016bd5525cee92a1df5d8819b5fd524d0a75749e5dd64d1c4d5ae63e3fb"
Jul 20 15:59:10.910047 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:10.910011    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a72e016bd5525cee92a1df5d8819b5fd524d0a75749e5dd64d1c4d5ae63e3fb"} err="failed to get container status \"6a72e016bd5525cee92a1df5d8819b5fd524d0a75749e5dd64d1c4d5ae63e3fb\": rpc error: code = NotFound desc = could not find container \"6a72e016bd5525cee92a1df5d8819b5fd524d0a75749e5dd64d1c4d5ae63e3fb\": container with ID starting with 6a72e016bd5525cee92a1df5d8819b5fd524d0a75749e5dd64d1c4d5ae63e3fb not found: ID does not exist"
Jul 20 15:59:10.923925 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:10.923900    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c7baa3ee-7858f89687-5zmnx"]
Jul 20 15:59:10.927900 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:10.927878    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c7baa3ee-7858f89687-5zmnx"]
Jul 20 15:59:11.514850 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:11.514817    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cbe2efc-88fb-473c-abed-26b9fa4fb62a" path="/var/lib/kubelet/pods/1cbe2efc-88fb-473c-abed-26b9fa4fb62a/volumes"
Jul 20 15:59:14.530498 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:14.530464    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:59:16.820647 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:16.820615    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:59:21.838807 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:21.838772    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-c7baa3ee/e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg"]
Jul 20 15:59:21.925858 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:21.925815    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:59:36.858403 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:36.858370    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-c7baa3ee/e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg" podUID="de3312e1-d9e4-478b-b862-d0b750717e01" containerName="main" containerID="cri-o://60ee9ad0e2f439ab9e696903da3d3f88104a28077f5d622ddb2107c63850b1d4" gracePeriod=45
Jul 20 15:59:37.025865 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:37.025831    2559 generic.go:356] "Generic (PLEG): container finished" podID="de3312e1-d9e4-478b-b862-d0b750717e01" containerID="60ee9ad0e2f439ab9e696903da3d3f88104a28077f5d622ddb2107c63850b1d4" exitCode=0
Jul 20 15:59:37.026000 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:37.025917    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c7baa3ee/e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg" event={"ID":"de3312e1-d9e4-478b-b862-d0b750717e01","Type":"ContainerDied","Data":"60ee9ad0e2f439ab9e696903da3d3f88104a28077f5d622ddb2107c63850b1d4"}
Jul 20 15:59:37.102644 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:37.102622    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c7baa3ee/e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg"
Jul 20 15:59:37.209603 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:37.209531    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de3312e1-d9e4-478b-b862-d0b750717e01-model-cache\") pod \"de3312e1-d9e4-478b-b862-d0b750717e01\" (UID: \"de3312e1-d9e4-478b-b862-d0b750717e01\") "
Jul 20 15:59:37.209603 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:37.209577    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de3312e1-d9e4-478b-b862-d0b750717e01-tmp-dir\") pod \"de3312e1-d9e4-478b-b862-d0b750717e01\" (UID: \"de3312e1-d9e4-478b-b862-d0b750717e01\") "
Jul 20 15:59:37.209603 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:37.209602    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zxlxx\" (UniqueName: \"kubernetes.io/projected/de3312e1-d9e4-478b-b862-d0b750717e01-kube-api-access-zxlxx\") pod \"de3312e1-d9e4-478b-b862-d0b750717e01\" (UID: \"de3312e1-d9e4-478b-b862-d0b750717e01\") "
Jul 20 15:59:37.209871 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:37.209647    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de3312e1-d9e4-478b-b862-d0b750717e01-dshm\") pod \"de3312e1-d9e4-478b-b862-d0b750717e01\" (UID: \"de3312e1-d9e4-478b-b862-d0b750717e01\") "
Jul 20 15:59:37.209871 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:37.209674    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de3312e1-d9e4-478b-b862-d0b750717e01-home\") pod \"de3312e1-d9e4-478b-b862-d0b750717e01\" (UID: \"de3312e1-d9e4-478b-b862-d0b750717e01\") "
Jul 20 15:59:37.209871 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:37.209698    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de3312e1-d9e4-478b-b862-d0b750717e01-tls-certs\") pod \"de3312e1-d9e4-478b-b862-d0b750717e01\" (UID: \"de3312e1-d9e4-478b-b862-d0b750717e01\") "
Jul 20 15:59:37.209871 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:37.209820    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de3312e1-d9e4-478b-b862-d0b750717e01-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "de3312e1-d9e4-478b-b862-d0b750717e01" (UID: "de3312e1-d9e4-478b-b862-d0b750717e01"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 15:59:37.209871 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:37.209824    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de3312e1-d9e4-478b-b862-d0b750717e01-model-cache" (OuterVolumeSpecName: "model-cache") pod "de3312e1-d9e4-478b-b862-d0b750717e01" (UID: "de3312e1-d9e4-478b-b862-d0b750717e01"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 15:59:37.210087 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:37.209957    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de3312e1-d9e4-478b-b862-d0b750717e01-model-cache\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:59:37.210087 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:37.209976    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de3312e1-d9e4-478b-b862-d0b750717e01-tmp-dir\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:59:37.210087 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:37.209975    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de3312e1-d9e4-478b-b862-d0b750717e01-home" (OuterVolumeSpecName: "home") pod "de3312e1-d9e4-478b-b862-d0b750717e01" (UID: "de3312e1-d9e4-478b-b862-d0b750717e01"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 15:59:37.211873 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:37.211848    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de3312e1-d9e4-478b-b862-d0b750717e01-dshm" (OuterVolumeSpecName: "dshm") pod "de3312e1-d9e4-478b-b862-d0b750717e01" (UID: "de3312e1-d9e4-478b-b862-d0b750717e01"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 15:59:37.211873 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:37.211854    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de3312e1-d9e4-478b-b862-d0b750717e01-kube-api-access-zxlxx" (OuterVolumeSpecName: "kube-api-access-zxlxx") pod "de3312e1-d9e4-478b-b862-d0b750717e01" (UID: "de3312e1-d9e4-478b-b862-d0b750717e01"). InnerVolumeSpecName "kube-api-access-zxlxx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 15:59:37.211873 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:37.211854    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de3312e1-d9e4-478b-b862-d0b750717e01-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "de3312e1-d9e4-478b-b862-d0b750717e01" (UID: "de3312e1-d9e4-478b-b862-d0b750717e01"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 15:59:37.310633 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:37.310596    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de3312e1-d9e4-478b-b862-d0b750717e01-tls-certs\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:59:37.310633 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:37.310630    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zxlxx\" (UniqueName: \"kubernetes.io/projected/de3312e1-d9e4-478b-b862-d0b750717e01-kube-api-access-zxlxx\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:59:37.310633 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:37.310639    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de3312e1-d9e4-478b-b862-d0b750717e01-dshm\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:59:37.310633 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:37.310647    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de3312e1-d9e4-478b-b862-d0b750717e01-home\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 15:59:38.031957 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:38.031933    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c7baa3ee/e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg"
Jul 20 15:59:38.032419 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:38.031969    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c7baa3ee/e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg" event={"ID":"de3312e1-d9e4-478b-b862-d0b750717e01","Type":"ContainerDied","Data":"1d71600541c5aaa4b0eea1ac70531b4e60d43c65eccb4e1e33846cb1297c8284"}
Jul 20 15:59:38.032419 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:38.032019    2559 scope.go:117] "RemoveContainer" containerID="60ee9ad0e2f439ab9e696903da3d3f88104a28077f5d622ddb2107c63850b1d4"
Jul 20 15:59:38.054112 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:38.054091    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-c7baa3ee/e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg"]
Jul 20 15:59:38.060722 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:38.060702    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-c7baa3ee/e2e-lifecycle-model-c7baa3ee-kserve-66f874f6ff-jjhhg"]
Jul 20 15:59:39.513849 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:39.513817    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de3312e1-d9e4-478b-b862-d0b750717e01" path="/var/lib/kubelet/pods/de3312e1-d9e4-478b-b862-d0b750717e01/volumes"
Jul 20 15:59:58.036790 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:58.036751    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 15:59:59.490769 ip-10-0-130-19 kubenswrapper[2559]: I0720 15:59:59.490739    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:00:05.465987 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:05.465954    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:00:11.495590 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:11.495555    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:00:11.571659 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:11.571628    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:00:11.875051 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:11.875021    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"]
Jul 20 16:00:11.875230 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:11.875202    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j" podUID="b564f59a-3215-4735-98b9-ca72b06ab84a" containerName="istio-proxy" containerID="cri-o://5cf6d02cb9f71e2e26fd9bb922616eb8a6b0b542909bdf02eb9c8cb7736bf991" gracePeriod=30
Jul 20 16:00:11.894000 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:11.893974    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:00:12.079705 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:12.079650    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:00:12.230800 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:12.230720    2559 patch_prober.go:28] interesting pod/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 20 16:00:12.230800 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:12.230783    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j" podUID="b564f59a-3215-4735-98b9-ca72b06ab84a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 20 16:00:13.148850 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:13.148816    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:00:13.252121 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:13.252089    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:00:13.362243 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:13.362215    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:00:14.225083 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:14.225049    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:00:14.324027 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:14.323997    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:00:15.700024 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:15.699994    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:00:17.155109 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:17.155083    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"
Jul 20 16:00:17.218030 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:17.217999    2559 generic.go:356] "Generic (PLEG): container finished" podID="b564f59a-3215-4735-98b9-ca72b06ab84a" containerID="5cf6d02cb9f71e2e26fd9bb922616eb8a6b0b542909bdf02eb9c8cb7736bf991" exitCode=0
Jul 20 16:00:17.218191 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:17.218046    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"
Jul 20 16:00:17.218191 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:17.218066    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j" event={"ID":"b564f59a-3215-4735-98b9-ca72b06ab84a","Type":"ContainerDied","Data":"5cf6d02cb9f71e2e26fd9bb922616eb8a6b0b542909bdf02eb9c8cb7736bf991"}
Jul 20 16:00:17.218191 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:17.218141    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j" event={"ID":"b564f59a-3215-4735-98b9-ca72b06ab84a","Type":"ContainerDied","Data":"92bf186b0528d007050c41b07896ae023ba36656fa418958329c5f4d0f524571"}
Jul 20 16:00:17.218191 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:17.218163    2559 scope.go:117] "RemoveContainer" containerID="5cf6d02cb9f71e2e26fd9bb922616eb8a6b0b542909bdf02eb9c8cb7736bf991"
Jul 20 16:00:17.228407 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:17.228379    2559 scope.go:117] "RemoveContainer" containerID="fc4abab2ab272211424a1b88f1a01c18cdac619b8cb4a676668ae329f78e38a8"
Jul 20 16:00:17.236894 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:17.236874    2559 scope.go:117] "RemoveContainer" containerID="5cf6d02cb9f71e2e26fd9bb922616eb8a6b0b542909bdf02eb9c8cb7736bf991"
Jul 20 16:00:17.237159 ip-10-0-130-19 kubenswrapper[2559]: E0720 16:00:17.237139    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cf6d02cb9f71e2e26fd9bb922616eb8a6b0b542909bdf02eb9c8cb7736bf991\": container with ID starting with 5cf6d02cb9f71e2e26fd9bb922616eb8a6b0b542909bdf02eb9c8cb7736bf991 not found: ID does not exist" containerID="5cf6d02cb9f71e2e26fd9bb922616eb8a6b0b542909bdf02eb9c8cb7736bf991"
Jul 20 16:00:17.237269 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:17.237171    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cf6d02cb9f71e2e26fd9bb922616eb8a6b0b542909bdf02eb9c8cb7736bf991"} err="failed to get container status \"5cf6d02cb9f71e2e26fd9bb922616eb8a6b0b542909bdf02eb9c8cb7736bf991\": rpc error: code = NotFound desc = could not find container \"5cf6d02cb9f71e2e26fd9bb922616eb8a6b0b542909bdf02eb9c8cb7736bf991\": container with ID starting with 5cf6d02cb9f71e2e26fd9bb922616eb8a6b0b542909bdf02eb9c8cb7736bf991 not found: ID does not exist"
Jul 20 16:00:17.237269 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:17.237190    2559 scope.go:117] "RemoveContainer" containerID="fc4abab2ab272211424a1b88f1a01c18cdac619b8cb4a676668ae329f78e38a8"
Jul 20 16:00:17.237571 ip-10-0-130-19 kubenswrapper[2559]: E0720 16:00:17.237552    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc4abab2ab272211424a1b88f1a01c18cdac619b8cb4a676668ae329f78e38a8\": container with ID starting with fc4abab2ab272211424a1b88f1a01c18cdac619b8cb4a676668ae329f78e38a8 not found: ID does not exist" containerID="fc4abab2ab272211424a1b88f1a01c18cdac619b8cb4a676668ae329f78e38a8"
Jul 20 16:00:17.237619 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:17.237583    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc4abab2ab272211424a1b88f1a01c18cdac619b8cb4a676668ae329f78e38a8"} err="failed to get container status \"fc4abab2ab272211424a1b88f1a01c18cdac619b8cb4a676668ae329f78e38a8\": rpc error: code = NotFound desc = could not find container \"fc4abab2ab272211424a1b88f1a01c18cdac619b8cb4a676668ae329f78e38a8\": container with ID starting with fc4abab2ab272211424a1b88f1a01c18cdac619b8cb4a676668ae329f78e38a8 not found: ID does not exist"
Jul 20 16:00:17.342982 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:17.342940    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b564f59a-3215-4735-98b9-ca72b06ab84a-istio-envoy\") pod \"b564f59a-3215-4735-98b9-ca72b06ab84a\" (UID: \"b564f59a-3215-4735-98b9-ca72b06ab84a\") "
Jul 20 16:00:17.342982 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:17.342995    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b564f59a-3215-4735-98b9-ca72b06ab84a-istio-podinfo\") pod \"b564f59a-3215-4735-98b9-ca72b06ab84a\" (UID: \"b564f59a-3215-4735-98b9-ca72b06ab84a\") "
Jul 20 16:00:17.343229 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:17.343028    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b564f59a-3215-4735-98b9-ca72b06ab84a-workload-socket\") pod \"b564f59a-3215-4735-98b9-ca72b06ab84a\" (UID: \"b564f59a-3215-4735-98b9-ca72b06ab84a\") "
Jul 20 16:00:17.343229 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:17.343054    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b564f59a-3215-4735-98b9-ca72b06ab84a-istiod-ca-cert\") pod \"b564f59a-3215-4735-98b9-ca72b06ab84a\" (UID: \"b564f59a-3215-4735-98b9-ca72b06ab84a\") "
Jul 20 16:00:17.343229 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:17.343077    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b564f59a-3215-4735-98b9-ca72b06ab84a-credential-socket\") pod \"b564f59a-3215-4735-98b9-ca72b06ab84a\" (UID: \"b564f59a-3215-4735-98b9-ca72b06ab84a\") "
Jul 20 16:00:17.343229 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:17.343115    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b564f59a-3215-4735-98b9-ca72b06ab84a-workload-certs\") pod \"b564f59a-3215-4735-98b9-ca72b06ab84a\" (UID: \"b564f59a-3215-4735-98b9-ca72b06ab84a\") "
Jul 20 16:00:17.343229 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:17.343156    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xm79f\" (UniqueName: \"kubernetes.io/projected/b564f59a-3215-4735-98b9-ca72b06ab84a-kube-api-access-xm79f\") pod \"b564f59a-3215-4735-98b9-ca72b06ab84a\" (UID: \"b564f59a-3215-4735-98b9-ca72b06ab84a\") "
Jul 20 16:00:17.343229 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:17.343226    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b564f59a-3215-4735-98b9-ca72b06ab84a-istio-data\") pod \"b564f59a-3215-4735-98b9-ca72b06ab84a\" (UID: \"b564f59a-3215-4735-98b9-ca72b06ab84a\") "
Jul 20 16:00:17.343544 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:17.343252    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b564f59a-3215-4735-98b9-ca72b06ab84a-istio-token\") pod \"b564f59a-3215-4735-98b9-ca72b06ab84a\" (UID: \"b564f59a-3215-4735-98b9-ca72b06ab84a\") "
Jul 20 16:00:17.343544 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:17.343319    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b564f59a-3215-4735-98b9-ca72b06ab84a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b564f59a-3215-4735-98b9-ca72b06ab84a" (UID: "b564f59a-3215-4735-98b9-ca72b06ab84a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:00:17.343655 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:17.343547    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b564f59a-3215-4735-98b9-ca72b06ab84a-workload-socket\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:00:17.343655 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:17.343546    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b564f59a-3215-4735-98b9-ca72b06ab84a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b564f59a-3215-4735-98b9-ca72b06ab84a" (UID: "b564f59a-3215-4735-98b9-ca72b06ab84a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:00:17.343655 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:17.343552    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b564f59a-3215-4735-98b9-ca72b06ab84a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b564f59a-3215-4735-98b9-ca72b06ab84a" (UID: "b564f59a-3215-4735-98b9-ca72b06ab84a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:00:17.343923 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:17.343881    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b564f59a-3215-4735-98b9-ca72b06ab84a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b564f59a-3215-4735-98b9-ca72b06ab84a" (UID: "b564f59a-3215-4735-98b9-ca72b06ab84a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:00:17.344506 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:17.344478    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b564f59a-3215-4735-98b9-ca72b06ab84a-istio-data" (OuterVolumeSpecName: "istio-data") pod "b564f59a-3215-4735-98b9-ca72b06ab84a" (UID: "b564f59a-3215-4735-98b9-ca72b06ab84a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:00:17.345347 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:17.345323    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b564f59a-3215-4735-98b9-ca72b06ab84a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b564f59a-3215-4735-98b9-ca72b06ab84a" (UID: "b564f59a-3215-4735-98b9-ca72b06ab84a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:00:17.345555 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:17.345520    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b564f59a-3215-4735-98b9-ca72b06ab84a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b564f59a-3215-4735-98b9-ca72b06ab84a" (UID: "b564f59a-3215-4735-98b9-ca72b06ab84a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 20 16:00:17.345651 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:17.345629    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b564f59a-3215-4735-98b9-ca72b06ab84a-kube-api-access-xm79f" (OuterVolumeSpecName: "kube-api-access-xm79f") pod "b564f59a-3215-4735-98b9-ca72b06ab84a" (UID: "b564f59a-3215-4735-98b9-ca72b06ab84a"). InnerVolumeSpecName "kube-api-access-xm79f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:00:17.345812 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:17.345796    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b564f59a-3215-4735-98b9-ca72b06ab84a-istio-token" (OuterVolumeSpecName: "istio-token") pod "b564f59a-3215-4735-98b9-ca72b06ab84a" (UID: "b564f59a-3215-4735-98b9-ca72b06ab84a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:00:17.444574 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:17.444488    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b564f59a-3215-4735-98b9-ca72b06ab84a-workload-certs\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:00:17.444574 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:17.444524    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xm79f\" (UniqueName: \"kubernetes.io/projected/b564f59a-3215-4735-98b9-ca72b06ab84a-kube-api-access-xm79f\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:00:17.444574 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:17.444536    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b564f59a-3215-4735-98b9-ca72b06ab84a-istio-data\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:00:17.444574 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:17.444549    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b564f59a-3215-4735-98b9-ca72b06ab84a-istio-token\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:00:17.444574 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:17.444560    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b564f59a-3215-4735-98b9-ca72b06ab84a-istio-envoy\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:00:17.444574 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:17.444570    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b564f59a-3215-4735-98b9-ca72b06ab84a-istio-podinfo\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:00:17.444878 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:17.444581    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b564f59a-3215-4735-98b9-ca72b06ab84a-istiod-ca-cert\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:00:17.444878 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:17.444591    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b564f59a-3215-4735-98b9-ca72b06ab84a-credential-socket\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:00:17.542730 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:17.542698    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"]
Jul 20 16:00:17.552007 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:17.551964    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-c7baa3ee-openshift-default-6458db55c9-dx47j"]
Jul 20 16:00:19.514221 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:19.514192    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b564f59a-3215-4735-98b9-ca72b06ab84a" path="/var/lib/kubelet/pods/b564f59a-3215-4735-98b9-ca72b06ab84a/volumes"
Jul 20 16:00:21.557442 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:21.557410    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:00:21.653930 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:21.653899    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:00:22.254714 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:22.254687    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:00:28.898327 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:28.898250    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:00:29.011826 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:29.011784    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:00:29.110921 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:29.110886    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:00:37.323839 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:37.323761    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:00:37.424707 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:37.424678    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:00:40.075693 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:40.075663    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:00:40.137909 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:00:40.137877    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:01:00.968737 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:00.968697    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:01:07.848523 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:07.848482    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:01:07.949041 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:07.949010    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:01:08.206940 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:08.206859    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:01:08.587434 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:08.587398    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:01:08.871034 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:08.870931    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:01:12.148746 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:12.148711    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:01:12.281210 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:12.281168    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-48c284-6db878bdd7-2chxw"]
Jul 20 16:01:12.281627 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:12.281609    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b564f59a-3215-4735-98b9-ca72b06ab84a" containerName="istio-proxy"
Jul 20 16:01:12.281725 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:12.281629    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b564f59a-3215-4735-98b9-ca72b06ab84a" containerName="istio-proxy"
Jul 20 16:01:12.281725 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:12.281640    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de3312e1-d9e4-478b-b862-d0b750717e01" containerName="main"
Jul 20 16:01:12.281725 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:12.281648    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="de3312e1-d9e4-478b-b862-d0b750717e01" containerName="main"
Jul 20 16:01:12.281725 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:12.281689    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cbe2efc-88fb-473c-abed-26b9fa4fb62a" containerName="maas-api"
Jul 20 16:01:12.281725 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:12.281698    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cbe2efc-88fb-473c-abed-26b9fa4fb62a" containerName="maas-api"
Jul 20 16:01:12.281981 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:12.281777    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="de3312e1-d9e4-478b-b862-d0b750717e01" containerName="main"
Jul 20 16:01:12.281981 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:12.281789    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b564f59a-3215-4735-98b9-ca72b06ab84a" containerName="istio-proxy"
Jul 20 16:01:12.281981 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:12.281801    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b564f59a-3215-4735-98b9-ca72b06ab84a" containerName="istio-proxy"
Jul 20 16:01:12.281981 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:12.281812    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cbe2efc-88fb-473c-abed-26b9fa4fb62a" containerName="maas-api"
Jul 20 16:01:12.284756 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:12.284720    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-48c284-6db878bdd7-2chxw"
Jul 20 16:01:12.289322 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:12.289271    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-48c284-serving-cert\""
Jul 20 16:01:12.289573 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:12.289558    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-96kqc\""
Jul 20 16:01:12.308348 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:12.308319    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-48c284-6db878bdd7-2chxw"]
Jul 20 16:01:12.400814 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:12.400702    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f5e48d8-f896-47af-adee-9927a8819970-maas-api-tls\") pod \"maas-api-e2e-inf-a-48c284-6db878bdd7-2chxw\" (UID: \"9f5e48d8-f896-47af-adee-9927a8819970\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-48c284-6db878bdd7-2chxw"
Jul 20 16:01:12.400814 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:12.400753    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spbv8\" (UniqueName: \"kubernetes.io/projected/9f5e48d8-f896-47af-adee-9927a8819970-kube-api-access-spbv8\") pod \"maas-api-e2e-inf-a-48c284-6db878bdd7-2chxw\" (UID: \"9f5e48d8-f896-47af-adee-9927a8819970\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-48c284-6db878bdd7-2chxw"
Jul 20 16:01:12.401030 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:12.400824    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f5e48d8-f896-47af-adee-9927a8819970-openshift-service-ca\") pod \"maas-api-e2e-inf-a-48c284-6db878bdd7-2chxw\" (UID: \"9f5e48d8-f896-47af-adee-9927a8819970\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-48c284-6db878bdd7-2chxw"
Jul 20 16:01:12.408986 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:12.408956    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:01:12.502029 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:12.501998    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f5e48d8-f896-47af-adee-9927a8819970-maas-api-tls\") pod \"maas-api-e2e-inf-a-48c284-6db878bdd7-2chxw\" (UID: \"9f5e48d8-f896-47af-adee-9927a8819970\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-48c284-6db878bdd7-2chxw"
Jul 20 16:01:12.502196 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:12.502037    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spbv8\" (UniqueName: \"kubernetes.io/projected/9f5e48d8-f896-47af-adee-9927a8819970-kube-api-access-spbv8\") pod \"maas-api-e2e-inf-a-48c284-6db878bdd7-2chxw\" (UID: \"9f5e48d8-f896-47af-adee-9927a8819970\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-48c284-6db878bdd7-2chxw"
Jul 20 16:01:12.502196 ip-10-0-130-19 kubenswrapper[2559]: E0720 16:01:12.502134    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-48c284-serving-cert: secret "maas-api-e2e-inf-a-48c284-serving-cert" not found
Jul 20 16:01:12.502196 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:12.502165    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f5e48d8-f896-47af-adee-9927a8819970-openshift-service-ca\") pod \"maas-api-e2e-inf-a-48c284-6db878bdd7-2chxw\" (UID: \"9f5e48d8-f896-47af-adee-9927a8819970\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-48c284-6db878bdd7-2chxw"
Jul 20 16:01:12.502355 ip-10-0-130-19 kubenswrapper[2559]: E0720 16:01:12.502204    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f5e48d8-f896-47af-adee-9927a8819970-maas-api-tls podName:9f5e48d8-f896-47af-adee-9927a8819970 nodeName:}" failed. No retries permitted until 2026-07-20 16:01:13.002187894 +0000 UTC m=+2056.011809534 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9f5e48d8-f896-47af-adee-9927a8819970-maas-api-tls") pod "maas-api-e2e-inf-a-48c284-6db878bdd7-2chxw" (UID: "9f5e48d8-f896-47af-adee-9927a8819970") : secret "maas-api-e2e-inf-a-48c284-serving-cert" not found
Jul 20 16:01:12.502828 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:12.502811    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f5e48d8-f896-47af-adee-9927a8819970-openshift-service-ca\") pod \"maas-api-e2e-inf-a-48c284-6db878bdd7-2chxw\" (UID: \"9f5e48d8-f896-47af-adee-9927a8819970\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-48c284-6db878bdd7-2chxw"
Jul 20 16:01:12.508735 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:12.508711    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:01:12.523086 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:12.523054    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-spbv8\" (UniqueName: \"kubernetes.io/projected/9f5e48d8-f896-47af-adee-9927a8819970-kube-api-access-spbv8\") pod \"maas-api-e2e-inf-a-48c284-6db878bdd7-2chxw\" (UID: \"9f5e48d8-f896-47af-adee-9927a8819970\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-48c284-6db878bdd7-2chxw"
Jul 20 16:01:12.818023 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:12.817988    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:01:12.932072 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:12.932032    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:01:13.007669 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:13.007637    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f5e48d8-f896-47af-adee-9927a8819970-maas-api-tls\") pod \"maas-api-e2e-inf-a-48c284-6db878bdd7-2chxw\" (UID: \"9f5e48d8-f896-47af-adee-9927a8819970\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-48c284-6db878bdd7-2chxw"
Jul 20 16:01:13.009929 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:13.009908    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f5e48d8-f896-47af-adee-9927a8819970-maas-api-tls\") pod \"maas-api-e2e-inf-a-48c284-6db878bdd7-2chxw\" (UID: \"9f5e48d8-f896-47af-adee-9927a8819970\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-48c284-6db878bdd7-2chxw"
Jul 20 16:01:13.039604 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:13.039574    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:01:13.195395 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:13.195314    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-48c284-6db878bdd7-2chxw"
Jul 20 16:01:13.328668 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:13.328644    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-48c284-6db878bdd7-2chxw"]
Jul 20 16:01:13.330952 ip-10-0-130-19 kubenswrapper[2559]: W0720 16:01:13.330925    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f5e48d8_f896_47af_adee_9927a8819970.slice/crio-e7fc2e02780dea65f5bb30824a4a669cab72a3011f6c6d02ee253719641dc756 WatchSource:0}: Error finding container e7fc2e02780dea65f5bb30824a4a669cab72a3011f6c6d02ee253719641dc756: Status 404 returned error can't find the container with id e7fc2e02780dea65f5bb30824a4a669cab72a3011f6c6d02ee253719641dc756
Jul 20 16:01:13.474530 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:13.474448    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-48c284-6db878bdd7-2chxw" event={"ID":"9f5e48d8-f896-47af-adee-9927a8819970","Type":"ContainerStarted","Data":"e7fc2e02780dea65f5bb30824a4a669cab72a3011f6c6d02ee253719641dc756"}
Jul 20 16:01:14.481079 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:14.481047    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-48c284-6db878bdd7-2chxw" event={"ID":"9f5e48d8-f896-47af-adee-9927a8819970","Type":"ContainerStarted","Data":"be13992e66541fca3f11c688d62fd4b026969ff2869a3510d92fff9e12ff2aae"}
Jul 20 16:01:14.481549 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:14.481149    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-48c284-6db878bdd7-2chxw"
Jul 20 16:01:16.536895 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:16.536859    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:01:16.938647 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:16.938568    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:01:17.247215 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.247112    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-48c284-6db878bdd7-2chxw" podStartSLOduration=4.718880518 podStartE2EDuration="5.247095463s" totalImagesPullingTime="528.214945ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:01:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:01:14.502483623 +0000 UTC m=+2057.512105287" watchObservedRunningTime="2026-07-20 16:01:17.247095463 +0000 UTC m=+2060.256717124"
Jul 20 16:01:17.248166 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.248148    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"]
Jul 20 16:01:17.248540 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.248527    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b564f59a-3215-4735-98b9-ca72b06ab84a" containerName="istio-proxy"
Jul 20 16:01:17.248583 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.248542    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b564f59a-3215-4735-98b9-ca72b06ab84a" containerName="istio-proxy"
Jul 20 16:01:17.252222 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.252203    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"
Jul 20 16:01:17.257436 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.257337    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-f9d39a-openshift-default-dockercfg-vqd2r\""
Jul 20 16:01:17.257436 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.257338    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 20 16:01:17.280775 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.280753    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"]
Jul 20 16:01:17.326258 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.326223    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:01:17.346858 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.346830    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92ccc397-b2d7-49c1-b915-54b4742c9c8e-istio-envoy\") pod \"e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf\" (UID: \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\") " pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"
Jul 20 16:01:17.347067 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.347032    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92ccc397-b2d7-49c1-b915-54b4742c9c8e-workload-socket\") pod \"e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf\" (UID: \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\") " pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"
Jul 20 16:01:17.347205 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.347069    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92ccc397-b2d7-49c1-b915-54b4742c9c8e-workload-certs\") pod \"e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf\" (UID: \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\") " pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"
Jul 20 16:01:17.347205 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.347108    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92ccc397-b2d7-49c1-b915-54b4742c9c8e-istio-data\") pod \"e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf\" (UID: \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\") " pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"
Jul 20 16:01:17.347542 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.347520    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92ccc397-b2d7-49c1-b915-54b4742c9c8e-istiod-ca-cert\") pod \"e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf\" (UID: \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\") " pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"
Jul 20 16:01:17.347736 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.347717    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92ccc397-b2d7-49c1-b915-54b4742c9c8e-istio-token\") pod \"e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf\" (UID: \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\") " pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"
Jul 20 16:01:17.347870 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.347855    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92ccc397-b2d7-49c1-b915-54b4742c9c8e-istio-podinfo\") pod \"e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf\" (UID: \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\") " pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"
Jul 20 16:01:17.348011 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.347992    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92ccc397-b2d7-49c1-b915-54b4742c9c8e-credential-socket\") pod \"e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf\" (UID: \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\") " pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"
Jul 20 16:01:17.348176 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.348153    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-566hm\" (UniqueName: \"kubernetes.io/projected/92ccc397-b2d7-49c1-b915-54b4742c9c8e-kube-api-access-566hm\") pod \"e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf\" (UID: \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\") " pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"
Jul 20 16:01:17.449126 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.449102    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92ccc397-b2d7-49c1-b915-54b4742c9c8e-istio-token\") pod \"e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf\" (UID: \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\") " pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"
Jul 20 16:01:17.449224 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.449133    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92ccc397-b2d7-49c1-b915-54b4742c9c8e-istio-podinfo\") pod \"e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf\" (UID: \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\") " pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"
Jul 20 16:01:17.449224 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.449153    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92ccc397-b2d7-49c1-b915-54b4742c9c8e-credential-socket\") pod \"e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf\" (UID: \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\") " pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"
Jul 20 16:01:17.449224 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.449179    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-566hm\" (UniqueName: \"kubernetes.io/projected/92ccc397-b2d7-49c1-b915-54b4742c9c8e-kube-api-access-566hm\") pod \"e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf\" (UID: \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\") " pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"
Jul 20 16:01:17.449224 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.449199    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92ccc397-b2d7-49c1-b915-54b4742c9c8e-istio-envoy\") pod \"e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf\" (UID: \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\") " pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"
Jul 20 16:01:17.449444 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.449233    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92ccc397-b2d7-49c1-b915-54b4742c9c8e-workload-socket\") pod \"e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf\" (UID: \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\") " pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"
Jul 20 16:01:17.449444 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.449367    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92ccc397-b2d7-49c1-b915-54b4742c9c8e-workload-certs\") pod \"e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf\" (UID: \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\") " pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"
Jul 20 16:01:17.449444 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.449430    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92ccc397-b2d7-49c1-b915-54b4742c9c8e-istio-data\") pod \"e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf\" (UID: \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\") " pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"
Jul 20 16:01:17.449589 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.449474    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92ccc397-b2d7-49c1-b915-54b4742c9c8e-istiod-ca-cert\") pod \"e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf\" (UID: \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\") " pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"
Jul 20 16:01:17.449648 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.449614    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92ccc397-b2d7-49c1-b915-54b4742c9c8e-credential-socket\") pod \"e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf\" (UID: \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\") " pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"
Jul 20 16:01:17.449705 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.449641    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92ccc397-b2d7-49c1-b915-54b4742c9c8e-workload-socket\") pod \"e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf\" (UID: \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\") " pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"
Jul 20 16:01:17.449765 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.449710    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92ccc397-b2d7-49c1-b915-54b4742c9c8e-workload-certs\") pod \"e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf\" (UID: \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\") " pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"
Jul 20 16:01:17.449765 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.449749    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92ccc397-b2d7-49c1-b915-54b4742c9c8e-istio-data\") pod \"e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf\" (UID: \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\") " pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"
Jul 20 16:01:17.450097 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.450077    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92ccc397-b2d7-49c1-b915-54b4742c9c8e-istiod-ca-cert\") pod \"e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf\" (UID: \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\") " pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"
Jul 20 16:01:17.451582 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.451562    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92ccc397-b2d7-49c1-b915-54b4742c9c8e-istio-podinfo\") pod \"e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf\" (UID: \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\") " pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"
Jul 20 16:01:17.451773 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.451753    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92ccc397-b2d7-49c1-b915-54b4742c9c8e-istio-envoy\") pod \"e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf\" (UID: \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\") " pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"
Jul 20 16:01:17.458262 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.458240    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92ccc397-b2d7-49c1-b915-54b4742c9c8e-istio-token\") pod \"e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf\" (UID: \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\") " pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"
Jul 20 16:01:17.458651 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.458626    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-566hm\" (UniqueName: \"kubernetes.io/projected/92ccc397-b2d7-49c1-b915-54b4742c9c8e-kube-api-access-566hm\") pod \"e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf\" (UID: \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\") " pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"
Jul 20 16:01:17.561756 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.561731    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"
Jul 20 16:01:17.700991 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.700851    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"]
Jul 20 16:01:17.703651 ip-10-0-130-19 kubenswrapper[2559]: W0720 16:01:17.703622    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92ccc397_b2d7_49c1_b915_54b4742c9c8e.slice/crio-87b2104bffa9f3ddd3fe4c813999cf8d939aa71d03091d15dff773ab1a9884de WatchSource:0}: Error finding container 87b2104bffa9f3ddd3fe4c813999cf8d939aa71d03091d15dff773ab1a9884de: Status 404 returned error can't find the container with id 87b2104bffa9f3ddd3fe4c813999cf8d939aa71d03091d15dff773ab1a9884de
Jul 20 16:01:17.706806 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.706768    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 16:01:17.706907 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.706850    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 16:01:17.706907 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.706889    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 16:01:17.826614 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:17.826541    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:01:18.501807 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:18.501771    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf" event={"ID":"92ccc397-b2d7-49c1-b915-54b4742c9c8e","Type":"ContainerStarted","Data":"deaa11bf8bf0ae820d1a780e57846341328e7420c3e2510ef1493583b862be29"}
Jul 20 16:01:18.501998 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:18.501821    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf" event={"ID":"92ccc397-b2d7-49c1-b915-54b4742c9c8e","Type":"ContainerStarted","Data":"87b2104bffa9f3ddd3fe4c813999cf8d939aa71d03091d15dff773ab1a9884de"}
Jul 20 16:01:18.550015 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:18.549970    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf" podStartSLOduration=1.5499556349999999 podStartE2EDuration="1.549955635s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:01:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:01:18.548487535 +0000 UTC m=+2061.558109208" watchObservedRunningTime="2026-07-20 16:01:18.549955635 +0000 UTC m=+2061.559577296"
Jul 20 16:01:18.562743 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:18.562718    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"
Jul 20 16:01:18.567723 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:18.567702    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"
Jul 20 16:01:19.506192 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:19.506115    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"
Jul 20 16:01:19.507051 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:19.507029    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"
Jul 20 16:01:20.492338 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:20.492270    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-48c284-6db878bdd7-2chxw"
Jul 20 16:01:20.633363 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:20.633333    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:01:20.707112 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:20.707075    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d39a-688999747f-nhdx8"]
Jul 20 16:01:20.710556 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:20.710535    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d39a-688999747f-nhdx8"
Jul 20 16:01:20.713017 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:20.712998    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-f9d39a-serving-cert\""
Jul 20 16:01:20.725941 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:20.725916    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d39a-688999747f-nhdx8"]
Jul 20 16:01:20.753762 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:20.753691    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:01:20.780957 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:20.780927    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8z5c8\" (UniqueName: \"kubernetes.io/projected/c9137f39-1b8b-4d84-b436-f208db71d773-kube-api-access-8z5c8\") pod \"maas-api-e2e-inf-b-f9d39a-688999747f-nhdx8\" (UID: \"c9137f39-1b8b-4d84-b436-f208db71d773\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d39a-688999747f-nhdx8"
Jul 20 16:01:20.781108 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:20.780980    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9137f39-1b8b-4d84-b436-f208db71d773-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f9d39a-688999747f-nhdx8\" (UID: \"c9137f39-1b8b-4d84-b436-f208db71d773\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d39a-688999747f-nhdx8"
Jul 20 16:01:20.781108 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:20.781096    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9137f39-1b8b-4d84-b436-f208db71d773-maas-api-tls\") pod \"maas-api-e2e-inf-b-f9d39a-688999747f-nhdx8\" (UID: \"c9137f39-1b8b-4d84-b436-f208db71d773\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d39a-688999747f-nhdx8"
Jul 20 16:01:20.832983 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:20.832955    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:01:20.882343 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:20.882312    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z5c8\" (UniqueName: \"kubernetes.io/projected/c9137f39-1b8b-4d84-b436-f208db71d773-kube-api-access-8z5c8\") pod \"maas-api-e2e-inf-b-f9d39a-688999747f-nhdx8\" (UID: \"c9137f39-1b8b-4d84-b436-f208db71d773\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d39a-688999747f-nhdx8"
Jul 20 16:01:20.882482 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:20.882394    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9137f39-1b8b-4d84-b436-f208db71d773-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f9d39a-688999747f-nhdx8\" (UID: \"c9137f39-1b8b-4d84-b436-f208db71d773\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d39a-688999747f-nhdx8"
Jul 20 16:01:20.882482 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:20.882463    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9137f39-1b8b-4d84-b436-f208db71d773-maas-api-tls\") pod \"maas-api-e2e-inf-b-f9d39a-688999747f-nhdx8\" (UID: \"c9137f39-1b8b-4d84-b436-f208db71d773\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d39a-688999747f-nhdx8"
Jul 20 16:01:20.883103 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:20.883082    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9137f39-1b8b-4d84-b436-f208db71d773-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f9d39a-688999747f-nhdx8\" (UID: \"c9137f39-1b8b-4d84-b436-f208db71d773\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d39a-688999747f-nhdx8"
Jul 20 16:01:20.884826 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:20.884806    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9137f39-1b8b-4d84-b436-f208db71d773-maas-api-tls\") pod \"maas-api-e2e-inf-b-f9d39a-688999747f-nhdx8\" (UID: \"c9137f39-1b8b-4d84-b436-f208db71d773\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d39a-688999747f-nhdx8"
Jul 20 16:01:20.892983 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:20.892957    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8z5c8\" (UniqueName: \"kubernetes.io/projected/c9137f39-1b8b-4d84-b436-f208db71d773-kube-api-access-8z5c8\") pod \"maas-api-e2e-inf-b-f9d39a-688999747f-nhdx8\" (UID: \"c9137f39-1b8b-4d84-b436-f208db71d773\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d39a-688999747f-nhdx8"
Jul 20 16:01:20.933147 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:20.933126    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:01:21.022826 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:21.022797    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d39a-688999747f-nhdx8"
Jul 20 16:01:21.161088 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:21.161062    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d39a-688999747f-nhdx8"]
Jul 20 16:01:21.162584 ip-10-0-130-19 kubenswrapper[2559]: W0720 16:01:21.162554    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9137f39_1b8b_4d84_b436_f208db71d773.slice/crio-6691a4f1da0e3b0c0d8002f1aafa40a60d35ed0a230416d6561ce8d199207f56 WatchSource:0}: Error finding container 6691a4f1da0e3b0c0d8002f1aafa40a60d35ed0a230416d6561ce8d199207f56: Status 404 returned error can't find the container with id 6691a4f1da0e3b0c0d8002f1aafa40a60d35ed0a230416d6561ce8d199207f56
Jul 20 16:01:21.349459 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:21.349378    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:01:21.459929 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:21.459895    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:01:21.517776 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:21.517744    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d39a-688999747f-nhdx8" event={"ID":"c9137f39-1b8b-4d84-b436-f208db71d773","Type":"ContainerStarted","Data":"6691a4f1da0e3b0c0d8002f1aafa40a60d35ed0a230416d6561ce8d199207f56"}
Jul 20 16:01:22.524719 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:22.524690    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d39a-688999747f-nhdx8" event={"ID":"c9137f39-1b8b-4d84-b436-f208db71d773","Type":"ContainerStarted","Data":"e77af26d49ea7975f83f15a9c2ff993099de66e5f61599702352086888caca76"}
Jul 20 16:01:22.525090 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:22.524797    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d39a-688999747f-nhdx8"
Jul 20 16:01:22.545341 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:22.545281    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d39a-688999747f-nhdx8" podStartSLOduration=1.501298427 podStartE2EDuration="2.545266958s" totalImagesPullingTime="1.043968531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:01:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:01:22.543256451 +0000 UTC m=+2065.552878112" watchObservedRunningTime="2026-07-20 16:01:22.545266958 +0000 UTC m=+2065.554888620"
Jul 20 16:01:25.089451 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:25.089418    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:01:28.535064 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:28.535034    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d39a-688999747f-nhdx8"
Jul 20 16:01:35.132670 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:35.132644    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:01:35.232072 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:35.232032    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:01:38.598411 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:38.598375    2559 generic.go:356] "Generic (PLEG): container finished" podID="92ccc397-b2d7-49c1-b915-54b4742c9c8e" containerID="deaa11bf8bf0ae820d1a780e57846341328e7420c3e2510ef1493583b862be29" exitCode=0
Jul 20 16:01:38.598835 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:38.598462    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf" event={"ID":"92ccc397-b2d7-49c1-b915-54b4742c9c8e","Type":"ContainerDied","Data":"deaa11bf8bf0ae820d1a780e57846341328e7420c3e2510ef1493583b862be29"}
Jul 20 16:01:38.598835 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:38.598796    2559 scope.go:117] "RemoveContainer" containerID="deaa11bf8bf0ae820d1a780e57846341328e7420c3e2510ef1493583b862be29"
Jul 20 16:01:38.599742 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:38.599717    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 16:01:38.599803 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:38.599782    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 16:01:38.599836 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:38.599812    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 16:01:39.419895 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:39.419857    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-48c284/test-model-48c284-kserve-79b958f95-d46sh"]
Jul 20 16:01:39.422616 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:39.422594    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-48c284/test-model-48c284-kserve-79b958f95-d46sh"
Jul 20 16:01:39.425688 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:39.425670    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-48c284\"/\"kube-root-ca.crt\""
Jul 20 16:01:39.426831 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:39.426808    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-48c284\"/\"default-dockercfg-d6zwc\""
Jul 20 16:01:39.426948 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:39.426848    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-48c284\"/\"test-model-48c284-kserve-self-signed-certs\""
Jul 20 16:01:39.426948 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:39.426819    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-48c284\"/\"openshift-service-ca.crt\""
Jul 20 16:01:39.437239 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:39.437210    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-48c284/test-model-48c284-kserve-79b958f95-d46sh"]
Jul 20 16:01:39.551317 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:39.551252    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c78c3d61-4968-4979-ac22-b2ac715c345e-tls-certs\") pod \"test-model-48c284-kserve-79b958f95-d46sh\" (UID: \"c78c3d61-4968-4979-ac22-b2ac715c345e\") " pod="ai-tenant-e2e-inf-a-48c284/test-model-48c284-kserve-79b958f95-d46sh"
Jul 20 16:01:39.551519 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:39.551335    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sv84v\" (UniqueName: \"kubernetes.io/projected/c78c3d61-4968-4979-ac22-b2ac715c345e-kube-api-access-sv84v\") pod \"test-model-48c284-kserve-79b958f95-d46sh\" (UID: \"c78c3d61-4968-4979-ac22-b2ac715c345e\") " pod="ai-tenant-e2e-inf-a-48c284/test-model-48c284-kserve-79b958f95-d46sh"
Jul 20 16:01:39.551519 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:39.551411    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c78c3d61-4968-4979-ac22-b2ac715c345e-home\") pod \"test-model-48c284-kserve-79b958f95-d46sh\" (UID: \"c78c3d61-4968-4979-ac22-b2ac715c345e\") " pod="ai-tenant-e2e-inf-a-48c284/test-model-48c284-kserve-79b958f95-d46sh"
Jul 20 16:01:39.551519 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:39.551452    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c78c3d61-4968-4979-ac22-b2ac715c345e-tmp-dir\") pod \"test-model-48c284-kserve-79b958f95-d46sh\" (UID: \"c78c3d61-4968-4979-ac22-b2ac715c345e\") " pod="ai-tenant-e2e-inf-a-48c284/test-model-48c284-kserve-79b958f95-d46sh"
Jul 20 16:01:39.551519 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:39.551490    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c78c3d61-4968-4979-ac22-b2ac715c345e-model-cache\") pod \"test-model-48c284-kserve-79b958f95-d46sh\" (UID: \"c78c3d61-4968-4979-ac22-b2ac715c345e\") " pod="ai-tenant-e2e-inf-a-48c284/test-model-48c284-kserve-79b958f95-d46sh"
Jul 20 16:01:39.551758 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:39.551585    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c78c3d61-4968-4979-ac22-b2ac715c345e-dshm\") pod \"test-model-48c284-kserve-79b958f95-d46sh\" (UID: \"c78c3d61-4968-4979-ac22-b2ac715c345e\") " pod="ai-tenant-e2e-inf-a-48c284/test-model-48c284-kserve-79b958f95-d46sh"
Jul 20 16:01:39.605273 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:39.605240    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf" event={"ID":"92ccc397-b2d7-49c1-b915-54b4742c9c8e","Type":"ContainerStarted","Data":"ce14c51bfe3f9bffc9993f8b90ea8b8a6019dfb70bece89cc0d7d8ea2bfeb339"}
Jul 20 16:01:39.624401 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:39.624374    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:01:39.653651 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:39.653617    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c78c3d61-4968-4979-ac22-b2ac715c345e-tls-certs\") pod \"test-model-48c284-kserve-79b958f95-d46sh\" (UID: \"c78c3d61-4968-4979-ac22-b2ac715c345e\") " pod="ai-tenant-e2e-inf-a-48c284/test-model-48c284-kserve-79b958f95-d46sh"
Jul 20 16:01:39.654642 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:39.654616    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv84v\" (UniqueName: \"kubernetes.io/projected/c78c3d61-4968-4979-ac22-b2ac715c345e-kube-api-access-sv84v\") pod \"test-model-48c284-kserve-79b958f95-d46sh\" (UID: \"c78c3d61-4968-4979-ac22-b2ac715c345e\") " pod="ai-tenant-e2e-inf-a-48c284/test-model-48c284-kserve-79b958f95-d46sh"
Jul 20 16:01:39.654863 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:39.654846    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c78c3d61-4968-4979-ac22-b2ac715c345e-home\") pod \"test-model-48c284-kserve-79b958f95-d46sh\" (UID: \"c78c3d61-4968-4979-ac22-b2ac715c345e\") " pod="ai-tenant-e2e-inf-a-48c284/test-model-48c284-kserve-79b958f95-d46sh"
Jul 20 16:01:39.655223 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:39.655197    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c78c3d61-4968-4979-ac22-b2ac715c345e-tmp-dir\") pod \"test-model-48c284-kserve-79b958f95-d46sh\" (UID: \"c78c3d61-4968-4979-ac22-b2ac715c345e\") " pod="ai-tenant-e2e-inf-a-48c284/test-model-48c284-kserve-79b958f95-d46sh"
Jul 20 16:01:39.655403 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:39.655350    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c78c3d61-4968-4979-ac22-b2ac715c345e-home\") pod \"test-model-48c284-kserve-79b958f95-d46sh\" (UID: \"c78c3d61-4968-4979-ac22-b2ac715c345e\") " pod="ai-tenant-e2e-inf-a-48c284/test-model-48c284-kserve-79b958f95-d46sh"
Jul 20 16:01:39.655403 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:39.655393    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c78c3d61-4968-4979-ac22-b2ac715c345e-model-cache\") pod \"test-model-48c284-kserve-79b958f95-d46sh\" (UID: \"c78c3d61-4968-4979-ac22-b2ac715c345e\") " pod="ai-tenant-e2e-inf-a-48c284/test-model-48c284-kserve-79b958f95-d46sh"
Jul 20 16:01:39.655593 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:39.655479    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c78c3d61-4968-4979-ac22-b2ac715c345e-dshm\") pod \"test-model-48c284-kserve-79b958f95-d46sh\" (UID: \"c78c3d61-4968-4979-ac22-b2ac715c345e\") " pod="ai-tenant-e2e-inf-a-48c284/test-model-48c284-kserve-79b958f95-d46sh"
Jul 20 16:01:39.655593 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:39.655516    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c78c3d61-4968-4979-ac22-b2ac715c345e-tmp-dir\") pod \"test-model-48c284-kserve-79b958f95-d46sh\" (UID: \"c78c3d61-4968-4979-ac22-b2ac715c345e\") " pod="ai-tenant-e2e-inf-a-48c284/test-model-48c284-kserve-79b958f95-d46sh"
Jul 20 16:01:39.655828 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:39.655803    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c78c3d61-4968-4979-ac22-b2ac715c345e-model-cache\") pod \"test-model-48c284-kserve-79b958f95-d46sh\" (UID: \"c78c3d61-4968-4979-ac22-b2ac715c345e\") " pod="ai-tenant-e2e-inf-a-48c284/test-model-48c284-kserve-79b958f95-d46sh"
Jul 20 16:01:39.659787 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:39.659762    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c78c3d61-4968-4979-ac22-b2ac715c345e-tls-certs\") pod \"test-model-48c284-kserve-79b958f95-d46sh\" (UID: \"c78c3d61-4968-4979-ac22-b2ac715c345e\") " pod="ai-tenant-e2e-inf-a-48c284/test-model-48c284-kserve-79b958f95-d46sh"
Jul 20 16:01:39.660839 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:39.660816    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c78c3d61-4968-4979-ac22-b2ac715c345e-dshm\") pod \"test-model-48c284-kserve-79b958f95-d46sh\" (UID: \"c78c3d61-4968-4979-ac22-b2ac715c345e\") " pod="ai-tenant-e2e-inf-a-48c284/test-model-48c284-kserve-79b958f95-d46sh"
Jul 20 16:01:39.666749 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:39.666713    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sv84v\" (UniqueName: \"kubernetes.io/projected/c78c3d61-4968-4979-ac22-b2ac715c345e-kube-api-access-sv84v\") pod \"test-model-48c284-kserve-79b958f95-d46sh\" (UID: \"c78c3d61-4968-4979-ac22-b2ac715c345e\") " pod="ai-tenant-e2e-inf-a-48c284/test-model-48c284-kserve-79b958f95-d46sh"
Jul 20 16:01:39.733991 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:39.733906    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-48c284/test-model-48c284-kserve-79b958f95-d46sh"
Jul 20 16:01:39.906663 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:39.906632    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-48c284/test-model-48c284-kserve-79b958f95-d46sh"]
Jul 20 16:01:39.908824 ip-10-0-130-19 kubenswrapper[2559]: W0720 16:01:39.908793    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc78c3d61_4968_4979_ac22_b2ac715c345e.slice/crio-922de8eeccef85068aef192b3deb2b662856acd1ca5b864777524b89c23c80df WatchSource:0}: Error finding container 922de8eeccef85068aef192b3deb2b662856acd1ca5b864777524b89c23c80df: Status 404 returned error can't find the container with id 922de8eeccef85068aef192b3deb2b662856acd1ca5b864777524b89c23c80df
Jul 20 16:01:40.562394 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:40.562357    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"
Jul 20 16:01:40.562635 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:40.562612    2559 patch_prober.go:28] interesting pod/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 20 16:01:40.562687 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:40.562666    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf" podUID="92ccc397-b2d7-49c1-b915-54b4742c9c8e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 20 16:01:40.610980 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:40.610946    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-48c284/test-model-48c284-kserve-79b958f95-d46sh" event={"ID":"c78c3d61-4968-4979-ac22-b2ac715c345e","Type":"ContainerStarted","Data":"29e060f7db975c1102860c7fb2ce45275b8fb203aeaf6ae62ebd3027cadc6f50"}
Jul 20 16:01:40.611339 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:40.610992    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-48c284/test-model-48c284-kserve-79b958f95-d46sh" event={"ID":"c78c3d61-4968-4979-ac22-b2ac715c345e","Type":"ContainerStarted","Data":"922de8eeccef85068aef192b3deb2b662856acd1ca5b864777524b89c23c80df"}
Jul 20 16:01:40.644100 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:40.644032    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-48c284/test-model-48c284-kserve-79b958f95-d46sh" podStartSLOduration=1.644011022 podStartE2EDuration="1.644011022s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:01:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:01:40.643105958 +0000 UTC m=+2083.652727621" watchObservedRunningTime="2026-07-20 16:01:40.644011022 +0000 UTC m=+2083.653632685"
Jul 20 16:01:41.563136 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:41.563098    2559 patch_prober.go:28] interesting pod/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 20 16:01:41.563388 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:41.563160    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf" podUID="92ccc397-b2d7-49c1-b915-54b4742c9c8e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 20 16:01:42.353595 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:42.353561    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:01:42.457576 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:42.457545    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:01:42.563134 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:42.563090    2559 patch_prober.go:28] interesting pod/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 20 16:01:42.563342 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:42.563158    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf" podUID="92ccc397-b2d7-49c1-b915-54b4742c9c8e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 20 16:01:43.566080 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:43.566051    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"
Jul 20 16:01:43.566508 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:43.566490    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"
Jul 20 16:01:43.567187 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:43.567166    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"
Jul 20 16:01:49.735011 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:49.734975    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-48c284/test-model-48c284-kserve-79b958f95-d46sh"
Jul 20 16:01:49.735481 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:49.735248    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-48c284/test-model-48c284-kserve-79b958f95-d46sh"
Jul 20 16:01:49.747515 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:49.747492    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-48c284/test-model-48c284-kserve-79b958f95-d46sh"
Jul 20 16:01:50.669904 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:01:50.669873    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-48c284/test-model-48c284-kserve-79b958f95-d46sh"
Jul 20 16:02:05.102968 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:05.102937    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-f9d39a/test-model-f9d39a-kserve-76fdbf5689-9wssr"]
Jul 20 16:02:05.105381 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:05.105360    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f9d39a/test-model-f9d39a-kserve-76fdbf5689-9wssr"
Jul 20 16:02:05.108336 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:05.108310    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-f9d39a\"/\"openshift-service-ca.crt\""
Jul 20 16:02:05.108460 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:05.108337    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-f9d39a\"/\"kube-root-ca.crt\""
Jul 20 16:02:05.109712 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:05.109691    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-f9d39a\"/\"default-dockercfg-8svvv\""
Jul 20 16:02:05.109823 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:05.109742    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-f9d39a\"/\"test-model-f9d39a-kserve-self-signed-certs\""
Jul 20 16:02:05.118153 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:05.118134    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-f9d39a/test-model-f9d39a-kserve-76fdbf5689-9wssr"]
Jul 20 16:02:05.150998 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:05.150964    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:02:05.281896 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:05.281874    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6f7s\" (UniqueName: \"kubernetes.io/projected/97d5ae75-6120-44fb-9e75-6707d9afcfc6-kube-api-access-f6f7s\") pod \"test-model-f9d39a-kserve-76fdbf5689-9wssr\" (UID: \"97d5ae75-6120-44fb-9e75-6707d9afcfc6\") " pod="ai-tenant-e2e-inf-b-f9d39a/test-model-f9d39a-kserve-76fdbf5689-9wssr"
Jul 20 16:02:05.282034 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:05.281931    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97d5ae75-6120-44fb-9e75-6707d9afcfc6-model-cache\") pod \"test-model-f9d39a-kserve-76fdbf5689-9wssr\" (UID: \"97d5ae75-6120-44fb-9e75-6707d9afcfc6\") " pod="ai-tenant-e2e-inf-b-f9d39a/test-model-f9d39a-kserve-76fdbf5689-9wssr"
Jul 20 16:02:05.282034 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:05.281997    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97d5ae75-6120-44fb-9e75-6707d9afcfc6-tls-certs\") pod \"test-model-f9d39a-kserve-76fdbf5689-9wssr\" (UID: \"97d5ae75-6120-44fb-9e75-6707d9afcfc6\") " pod="ai-tenant-e2e-inf-b-f9d39a/test-model-f9d39a-kserve-76fdbf5689-9wssr"
Jul 20 16:02:05.282164 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:05.282056    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97d5ae75-6120-44fb-9e75-6707d9afcfc6-home\") pod \"test-model-f9d39a-kserve-76fdbf5689-9wssr\" (UID: \"97d5ae75-6120-44fb-9e75-6707d9afcfc6\") " pod="ai-tenant-e2e-inf-b-f9d39a/test-model-f9d39a-kserve-76fdbf5689-9wssr"
Jul 20 16:02:05.282164 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:05.282120    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97d5ae75-6120-44fb-9e75-6707d9afcfc6-tmp-dir\") pod \"test-model-f9d39a-kserve-76fdbf5689-9wssr\" (UID: \"97d5ae75-6120-44fb-9e75-6707d9afcfc6\") " pod="ai-tenant-e2e-inf-b-f9d39a/test-model-f9d39a-kserve-76fdbf5689-9wssr"
Jul 20 16:02:05.282244 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:05.282167    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97d5ae75-6120-44fb-9e75-6707d9afcfc6-dshm\") pod \"test-model-f9d39a-kserve-76fdbf5689-9wssr\" (UID: \"97d5ae75-6120-44fb-9e75-6707d9afcfc6\") " pod="ai-tenant-e2e-inf-b-f9d39a/test-model-f9d39a-kserve-76fdbf5689-9wssr"
Jul 20 16:02:05.383673 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:05.383602    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6f7s\" (UniqueName: \"kubernetes.io/projected/97d5ae75-6120-44fb-9e75-6707d9afcfc6-kube-api-access-f6f7s\") pod \"test-model-f9d39a-kserve-76fdbf5689-9wssr\" (UID: \"97d5ae75-6120-44fb-9e75-6707d9afcfc6\") " pod="ai-tenant-e2e-inf-b-f9d39a/test-model-f9d39a-kserve-76fdbf5689-9wssr"
Jul 20 16:02:05.383673 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:05.383663    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97d5ae75-6120-44fb-9e75-6707d9afcfc6-model-cache\") pod \"test-model-f9d39a-kserve-76fdbf5689-9wssr\" (UID: \"97d5ae75-6120-44fb-9e75-6707d9afcfc6\") " pod="ai-tenant-e2e-inf-b-f9d39a/test-model-f9d39a-kserve-76fdbf5689-9wssr"
Jul 20 16:02:05.383831 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:05.383697    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97d5ae75-6120-44fb-9e75-6707d9afcfc6-tls-certs\") pod \"test-model-f9d39a-kserve-76fdbf5689-9wssr\" (UID: \"97d5ae75-6120-44fb-9e75-6707d9afcfc6\") " pod="ai-tenant-e2e-inf-b-f9d39a/test-model-f9d39a-kserve-76fdbf5689-9wssr"
Jul 20 16:02:05.383831 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:05.383723    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97d5ae75-6120-44fb-9e75-6707d9afcfc6-home\") pod \"test-model-f9d39a-kserve-76fdbf5689-9wssr\" (UID: \"97d5ae75-6120-44fb-9e75-6707d9afcfc6\") " pod="ai-tenant-e2e-inf-b-f9d39a/test-model-f9d39a-kserve-76fdbf5689-9wssr"
Jul 20 16:02:05.383831 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:05.383752    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97d5ae75-6120-44fb-9e75-6707d9afcfc6-tmp-dir\") pod \"test-model-f9d39a-kserve-76fdbf5689-9wssr\" (UID: \"97d5ae75-6120-44fb-9e75-6707d9afcfc6\") " pod="ai-tenant-e2e-inf-b-f9d39a/test-model-f9d39a-kserve-76fdbf5689-9wssr"
Jul 20 16:02:05.383831 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:05.383785    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97d5ae75-6120-44fb-9e75-6707d9afcfc6-dshm\") pod \"test-model-f9d39a-kserve-76fdbf5689-9wssr\" (UID: \"97d5ae75-6120-44fb-9e75-6707d9afcfc6\") " pod="ai-tenant-e2e-inf-b-f9d39a/test-model-f9d39a-kserve-76fdbf5689-9wssr"
Jul 20 16:02:05.384122 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:05.384104    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97d5ae75-6120-44fb-9e75-6707d9afcfc6-model-cache\") pod \"test-model-f9d39a-kserve-76fdbf5689-9wssr\" (UID: \"97d5ae75-6120-44fb-9e75-6707d9afcfc6\") " pod="ai-tenant-e2e-inf-b-f9d39a/test-model-f9d39a-kserve-76fdbf5689-9wssr"
Jul 20 16:02:05.384211 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:05.384190    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97d5ae75-6120-44fb-9e75-6707d9afcfc6-home\") pod \"test-model-f9d39a-kserve-76fdbf5689-9wssr\" (UID: \"97d5ae75-6120-44fb-9e75-6707d9afcfc6\") " pod="ai-tenant-e2e-inf-b-f9d39a/test-model-f9d39a-kserve-76fdbf5689-9wssr"
Jul 20 16:02:05.384246 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:05.384220    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97d5ae75-6120-44fb-9e75-6707d9afcfc6-tmp-dir\") pod \"test-model-f9d39a-kserve-76fdbf5689-9wssr\" (UID: \"97d5ae75-6120-44fb-9e75-6707d9afcfc6\") " pod="ai-tenant-e2e-inf-b-f9d39a/test-model-f9d39a-kserve-76fdbf5689-9wssr"
Jul 20 16:02:05.386118 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:05.386087    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97d5ae75-6120-44fb-9e75-6707d9afcfc6-dshm\") pod \"test-model-f9d39a-kserve-76fdbf5689-9wssr\" (UID: \"97d5ae75-6120-44fb-9e75-6707d9afcfc6\") " pod="ai-tenant-e2e-inf-b-f9d39a/test-model-f9d39a-kserve-76fdbf5689-9wssr"
Jul 20 16:02:05.386310 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:05.386276    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97d5ae75-6120-44fb-9e75-6707d9afcfc6-tls-certs\") pod \"test-model-f9d39a-kserve-76fdbf5689-9wssr\" (UID: \"97d5ae75-6120-44fb-9e75-6707d9afcfc6\") " pod="ai-tenant-e2e-inf-b-f9d39a/test-model-f9d39a-kserve-76fdbf5689-9wssr"
Jul 20 16:02:05.393328 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:05.393303    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6f7s\" (UniqueName: \"kubernetes.io/projected/97d5ae75-6120-44fb-9e75-6707d9afcfc6-kube-api-access-f6f7s\") pod \"test-model-f9d39a-kserve-76fdbf5689-9wssr\" (UID: \"97d5ae75-6120-44fb-9e75-6707d9afcfc6\") " pod="ai-tenant-e2e-inf-b-f9d39a/test-model-f9d39a-kserve-76fdbf5689-9wssr"
Jul 20 16:02:05.415693 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:05.415672    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f9d39a/test-model-f9d39a-kserve-76fdbf5689-9wssr"
Jul 20 16:02:05.543691 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:05.543669    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-f9d39a/test-model-f9d39a-kserve-76fdbf5689-9wssr"]
Jul 20 16:02:05.545758 ip-10-0-130-19 kubenswrapper[2559]: W0720 16:02:05.545719    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod97d5ae75_6120_44fb_9e75_6707d9afcfc6.slice/crio-d4a74adb3f78e136211e23b423c6dff1def23af62138234e44a358f6d4f2e75c WatchSource:0}: Error finding container d4a74adb3f78e136211e23b423c6dff1def23af62138234e44a358f6d4f2e75c: Status 404 returned error can't find the container with id d4a74adb3f78e136211e23b423c6dff1def23af62138234e44a358f6d4f2e75c
Jul 20 16:02:05.728661 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:05.728577    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f9d39a/test-model-f9d39a-kserve-76fdbf5689-9wssr" event={"ID":"97d5ae75-6120-44fb-9e75-6707d9afcfc6","Type":"ContainerStarted","Data":"a729323c754dd268dbf79bd76387a994d378fe966dd62fded7f1aff53212697b"}
Jul 20 16:02:05.728992 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:05.728948    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f9d39a/test-model-f9d39a-kserve-76fdbf5689-9wssr" event={"ID":"97d5ae75-6120-44fb-9e75-6707d9afcfc6","Type":"ContainerStarted","Data":"d4a74adb3f78e136211e23b423c6dff1def23af62138234e44a358f6d4f2e75c"}
Jul 20 16:02:05.748436 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:05.748393    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-f9d39a/test-model-f9d39a-kserve-76fdbf5689-9wssr" podStartSLOduration=0.748380592 podStartE2EDuration="748.380592ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:02:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:02:05.747974927 +0000 UTC m=+2108.757596590" watchObservedRunningTime="2026-07-20 16:02:05.748380592 +0000 UTC m=+2108.758002254"
Jul 20 16:02:10.095745 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:10.095693    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:02:10.196580 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:10.196552    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:02:15.416779 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:15.416728    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-f9d39a/test-model-f9d39a-kserve-76fdbf5689-9wssr"
Jul 20 16:02:15.416779 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:15.416785    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-f9d39a/test-model-f9d39a-kserve-76fdbf5689-9wssr"
Jul 20 16:02:15.429055 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:15.429031    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-f9d39a/test-model-f9d39a-kserve-76fdbf5689-9wssr"
Jul 20 16:02:15.787096 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:15.787058    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-f9d39a/test-model-f9d39a-kserve-76fdbf5689-9wssr"
Jul 20 16:02:24.072063 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:24.072013    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:02:44.798846 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:44.798811    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-48c284/test-model-48c284-kserve-79b958f95-d46sh"]
Jul 20 16:02:44.835591 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:44.835552    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:02:45.753090 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:45.753049    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-f9d39a/test-model-f9d39a-kserve-76fdbf5689-9wssr"]
Jul 20 16:02:46.942753 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:46.942725    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:02:56.468885 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:56.468808    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-48c284-6db878bdd7-2chxw"]
Jul 20 16:02:56.469316 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:56.469002    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-48c284-6db878bdd7-2chxw" podUID="9f5e48d8-f896-47af-adee-9927a8819970" containerName="maas-api" containerID="cri-o://be13992e66541fca3f11c688d62fd4b026969ff2869a3510d92fff9e12ff2aae" gracePeriod=30
Jul 20 16:02:56.716886 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:56.716865    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-48c284-6db878bdd7-2chxw"
Jul 20 16:02:56.724185 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:56.724125    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f5e48d8-f896-47af-adee-9927a8819970-maas-api-tls\") pod \"9f5e48d8-f896-47af-adee-9927a8819970\" (UID: \"9f5e48d8-f896-47af-adee-9927a8819970\") "
Jul 20 16:02:56.724272 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:56.724203    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f5e48d8-f896-47af-adee-9927a8819970-openshift-service-ca\") pod \"9f5e48d8-f896-47af-adee-9927a8819970\" (UID: \"9f5e48d8-f896-47af-adee-9927a8819970\") "
Jul 20 16:02:56.724272 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:56.724257    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-spbv8\" (UniqueName: \"kubernetes.io/projected/9f5e48d8-f896-47af-adee-9927a8819970-kube-api-access-spbv8\") pod \"9f5e48d8-f896-47af-adee-9927a8819970\" (UID: \"9f5e48d8-f896-47af-adee-9927a8819970\") "
Jul 20 16:02:56.724620 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:56.724591    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f5e48d8-f896-47af-adee-9927a8819970-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9f5e48d8-f896-47af-adee-9927a8819970" (UID: "9f5e48d8-f896-47af-adee-9927a8819970"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:02:56.726103 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:56.726082    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f5e48d8-f896-47af-adee-9927a8819970-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9f5e48d8-f896-47af-adee-9927a8819970" (UID: "9f5e48d8-f896-47af-adee-9927a8819970"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:02:56.726203 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:56.726184    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f5e48d8-f896-47af-adee-9927a8819970-kube-api-access-spbv8" (OuterVolumeSpecName: "kube-api-access-spbv8") pod "9f5e48d8-f896-47af-adee-9927a8819970" (UID: "9f5e48d8-f896-47af-adee-9927a8819970"). InnerVolumeSpecName "kube-api-access-spbv8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:02:56.748737 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:56.748711    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:02:56.824951 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:56.824928    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f5e48d8-f896-47af-adee-9927a8819970-openshift-service-ca\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:02:56.824951 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:56.824948    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-spbv8\" (UniqueName: \"kubernetes.io/projected/9f5e48d8-f896-47af-adee-9927a8819970-kube-api-access-spbv8\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:02:56.825092 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:56.824958    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f5e48d8-f896-47af-adee-9927a8819970-maas-api-tls\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:02:56.967152 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:56.967119    2559 generic.go:356] "Generic (PLEG): container finished" podID="9f5e48d8-f896-47af-adee-9927a8819970" containerID="be13992e66541fca3f11c688d62fd4b026969ff2869a3510d92fff9e12ff2aae" exitCode=0
Jul 20 16:02:56.967312 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:56.967160    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-48c284-6db878bdd7-2chxw"
Jul 20 16:02:56.967312 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:56.967154    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-48c284-6db878bdd7-2chxw" event={"ID":"9f5e48d8-f896-47af-adee-9927a8819970","Type":"ContainerDied","Data":"be13992e66541fca3f11c688d62fd4b026969ff2869a3510d92fff9e12ff2aae"}
Jul 20 16:02:56.967312 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:56.967278    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-48c284-6db878bdd7-2chxw" event={"ID":"9f5e48d8-f896-47af-adee-9927a8819970","Type":"ContainerDied","Data":"e7fc2e02780dea65f5bb30824a4a669cab72a3011f6c6d02ee253719641dc756"}
Jul 20 16:02:56.967431 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:56.967320    2559 scope.go:117] "RemoveContainer" containerID="be13992e66541fca3f11c688d62fd4b026969ff2869a3510d92fff9e12ff2aae"
Jul 20 16:02:56.983362 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:56.983342    2559 scope.go:117] "RemoveContainer" containerID="be13992e66541fca3f11c688d62fd4b026969ff2869a3510d92fff9e12ff2aae"
Jul 20 16:02:56.983611 ip-10-0-130-19 kubenswrapper[2559]: E0720 16:02:56.983594    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be13992e66541fca3f11c688d62fd4b026969ff2869a3510d92fff9e12ff2aae\": container with ID starting with be13992e66541fca3f11c688d62fd4b026969ff2869a3510d92fff9e12ff2aae not found: ID does not exist" containerID="be13992e66541fca3f11c688d62fd4b026969ff2869a3510d92fff9e12ff2aae"
Jul 20 16:02:56.983677 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:56.983618    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be13992e66541fca3f11c688d62fd4b026969ff2869a3510d92fff9e12ff2aae"} err="failed to get container status \"be13992e66541fca3f11c688d62fd4b026969ff2869a3510d92fff9e12ff2aae\": rpc error: code = NotFound desc = could not find container \"be13992e66541fca3f11c688d62fd4b026969ff2869a3510d92fff9e12ff2aae\": container with ID starting with be13992e66541fca3f11c688d62fd4b026969ff2869a3510d92fff9e12ff2aae not found: ID does not exist"
Jul 20 16:02:57.000578 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:57.000548    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-48c284-6db878bdd7-2chxw"]
Jul 20 16:02:57.005100 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:57.005079    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-48c284-6db878bdd7-2chxw"]
Jul 20 16:02:57.515304 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:57.515259    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f5e48d8-f896-47af-adee-9927a8819970" path="/var/lib/kubelet/pods/9f5e48d8-f896-47af-adee-9927a8819970/volumes"
Jul 20 16:02:59.818819 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:59.818780    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-48c284/test-model-48c284-kserve-79b958f95-d46sh" podUID="c78c3d61-4968-4979-ac22-b2ac715c345e" containerName="main" containerID="cri-o://29e060f7db975c1102860c7fb2ce45275b8fb203aeaf6ae62ebd3027cadc6f50" gracePeriod=45
Jul 20 16:02:59.983476 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:59.983432    2559 generic.go:356] "Generic (PLEG): container finished" podID="c78c3d61-4968-4979-ac22-b2ac715c345e" containerID="29e060f7db975c1102860c7fb2ce45275b8fb203aeaf6ae62ebd3027cadc6f50" exitCode=0
Jul 20 16:02:59.983623 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:02:59.983490    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-48c284/test-model-48c284-kserve-79b958f95-d46sh" event={"ID":"c78c3d61-4968-4979-ac22-b2ac715c345e","Type":"ContainerDied","Data":"29e060f7db975c1102860c7fb2ce45275b8fb203aeaf6ae62ebd3027cadc6f50"}
Jul 20 16:03:00.076540 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:00.076472    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-48c284/test-model-48c284-kserve-79b958f95-d46sh"
Jul 20 16:03:00.156585 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:00.156557    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sv84v\" (UniqueName: \"kubernetes.io/projected/c78c3d61-4968-4979-ac22-b2ac715c345e-kube-api-access-sv84v\") pod \"c78c3d61-4968-4979-ac22-b2ac715c345e\" (UID: \"c78c3d61-4968-4979-ac22-b2ac715c345e\") "
Jul 20 16:03:00.156736 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:00.156619    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c78c3d61-4968-4979-ac22-b2ac715c345e-tmp-dir\") pod \"c78c3d61-4968-4979-ac22-b2ac715c345e\" (UID: \"c78c3d61-4968-4979-ac22-b2ac715c345e\") "
Jul 20 16:03:00.156736 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:00.156647    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c78c3d61-4968-4979-ac22-b2ac715c345e-model-cache\") pod \"c78c3d61-4968-4979-ac22-b2ac715c345e\" (UID: \"c78c3d61-4968-4979-ac22-b2ac715c345e\") "
Jul 20 16:03:00.156736 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:00.156692    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c78c3d61-4968-4979-ac22-b2ac715c345e-dshm\") pod \"c78c3d61-4968-4979-ac22-b2ac715c345e\" (UID: \"c78c3d61-4968-4979-ac22-b2ac715c345e\") "
Jul 20 16:03:00.156736 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:00.156718    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c78c3d61-4968-4979-ac22-b2ac715c345e-tls-certs\") pod \"c78c3d61-4968-4979-ac22-b2ac715c345e\" (UID: \"c78c3d61-4968-4979-ac22-b2ac715c345e\") "
Jul 20 16:03:00.156932 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:00.156740    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c78c3d61-4968-4979-ac22-b2ac715c345e-home\") pod \"c78c3d61-4968-4979-ac22-b2ac715c345e\" (UID: \"c78c3d61-4968-4979-ac22-b2ac715c345e\") "
Jul 20 16:03:00.156991 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:00.156950    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c78c3d61-4968-4979-ac22-b2ac715c345e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c78c3d61-4968-4979-ac22-b2ac715c345e" (UID: "c78c3d61-4968-4979-ac22-b2ac715c345e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:03:00.156991 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:00.156963    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c78c3d61-4968-4979-ac22-b2ac715c345e-model-cache" (OuterVolumeSpecName: "model-cache") pod "c78c3d61-4968-4979-ac22-b2ac715c345e" (UID: "c78c3d61-4968-4979-ac22-b2ac715c345e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:03:00.157104 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:00.157033    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c78c3d61-4968-4979-ac22-b2ac715c345e-tmp-dir\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:03:00.157104 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:00.157050    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c78c3d61-4968-4979-ac22-b2ac715c345e-model-cache\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:03:00.157104 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:00.157066    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c78c3d61-4968-4979-ac22-b2ac715c345e-home" (OuterVolumeSpecName: "home") pod "c78c3d61-4968-4979-ac22-b2ac715c345e" (UID: "c78c3d61-4968-4979-ac22-b2ac715c345e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:03:00.158933 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:00.158908    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c78c3d61-4968-4979-ac22-b2ac715c345e-dshm" (OuterVolumeSpecName: "dshm") pod "c78c3d61-4968-4979-ac22-b2ac715c345e" (UID: "c78c3d61-4968-4979-ac22-b2ac715c345e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:03:00.158933 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:00.158925    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c78c3d61-4968-4979-ac22-b2ac715c345e-kube-api-access-sv84v" (OuterVolumeSpecName: "kube-api-access-sv84v") pod "c78c3d61-4968-4979-ac22-b2ac715c345e" (UID: "c78c3d61-4968-4979-ac22-b2ac715c345e"). InnerVolumeSpecName "kube-api-access-sv84v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:03:00.159073 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:00.158911    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c78c3d61-4968-4979-ac22-b2ac715c345e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c78c3d61-4968-4979-ac22-b2ac715c345e" (UID: "c78c3d61-4968-4979-ac22-b2ac715c345e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:03:00.257494 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:00.257466    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sv84v\" (UniqueName: \"kubernetes.io/projected/c78c3d61-4968-4979-ac22-b2ac715c345e-kube-api-access-sv84v\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:03:00.257494 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:00.257491    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c78c3d61-4968-4979-ac22-b2ac715c345e-dshm\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:03:00.257659 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:00.257502    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c78c3d61-4968-4979-ac22-b2ac715c345e-tls-certs\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:03:00.257659 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:00.257511    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c78c3d61-4968-4979-ac22-b2ac715c345e-home\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:03:00.773645 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:00.773608    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-f9d39a/test-model-f9d39a-kserve-76fdbf5689-9wssr" podUID="97d5ae75-6120-44fb-9e75-6707d9afcfc6" containerName="main" containerID="cri-o://a729323c754dd268dbf79bd76387a994d378fe966dd62fded7f1aff53212697b" gracePeriod=45
Jul 20 16:03:00.989246 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:00.989209    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-48c284/test-model-48c284-kserve-79b958f95-d46sh"
Jul 20 16:03:00.989708 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:00.989238    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-48c284/test-model-48c284-kserve-79b958f95-d46sh" event={"ID":"c78c3d61-4968-4979-ac22-b2ac715c345e","Type":"ContainerDied","Data":"922de8eeccef85068aef192b3deb2b662856acd1ca5b864777524b89c23c80df"}
Jul 20 16:03:00.989708 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:00.989314    2559 scope.go:117] "RemoveContainer" containerID="29e060f7db975c1102860c7fb2ce45275b8fb203aeaf6ae62ebd3027cadc6f50"
Jul 20 16:03:00.990927 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:00.990905    2559 generic.go:356] "Generic (PLEG): container finished" podID="97d5ae75-6120-44fb-9e75-6707d9afcfc6" containerID="a729323c754dd268dbf79bd76387a994d378fe966dd62fded7f1aff53212697b" exitCode=0
Jul 20 16:03:00.991071 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:00.991050    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f9d39a/test-model-f9d39a-kserve-76fdbf5689-9wssr" event={"ID":"97d5ae75-6120-44fb-9e75-6707d9afcfc6","Type":"ContainerDied","Data":"a729323c754dd268dbf79bd76387a994d378fe966dd62fded7f1aff53212697b"}
Jul 20 16:03:00.995749 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:00.995730    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:03:01.013381 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:01.013355    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-48c284/test-model-48c284-kserve-79b958f95-d46sh"]
Jul 20 16:03:01.018933 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:01.018913    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-48c284/test-model-48c284-kserve-79b958f95-d46sh"]
Jul 20 16:03:01.031088 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:01.031042    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f9d39a/test-model-f9d39a-kserve-76fdbf5689-9wssr"
Jul 20 16:03:01.063640 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:01.063617    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6f7s\" (UniqueName: \"kubernetes.io/projected/97d5ae75-6120-44fb-9e75-6707d9afcfc6-kube-api-access-f6f7s\") pod \"97d5ae75-6120-44fb-9e75-6707d9afcfc6\" (UID: \"97d5ae75-6120-44fb-9e75-6707d9afcfc6\") "
Jul 20 16:03:01.063771 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:01.063712    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97d5ae75-6120-44fb-9e75-6707d9afcfc6-tmp-dir\") pod \"97d5ae75-6120-44fb-9e75-6707d9afcfc6\" (UID: \"97d5ae75-6120-44fb-9e75-6707d9afcfc6\") "
Jul 20 16:03:01.063771 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:01.063753    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97d5ae75-6120-44fb-9e75-6707d9afcfc6-model-cache\") pod \"97d5ae75-6120-44fb-9e75-6707d9afcfc6\" (UID: \"97d5ae75-6120-44fb-9e75-6707d9afcfc6\") "
Jul 20 16:03:01.063888 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:01.063790    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97d5ae75-6120-44fb-9e75-6707d9afcfc6-home\") pod \"97d5ae75-6120-44fb-9e75-6707d9afcfc6\" (UID: \"97d5ae75-6120-44fb-9e75-6707d9afcfc6\") "
Jul 20 16:03:01.063888 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:01.063854    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97d5ae75-6120-44fb-9e75-6707d9afcfc6-tls-certs\") pod \"97d5ae75-6120-44fb-9e75-6707d9afcfc6\" (UID: \"97d5ae75-6120-44fb-9e75-6707d9afcfc6\") "
Jul 20 16:03:01.063888 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:01.063880    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97d5ae75-6120-44fb-9e75-6707d9afcfc6-dshm\") pod \"97d5ae75-6120-44fb-9e75-6707d9afcfc6\" (UID: \"97d5ae75-6120-44fb-9e75-6707d9afcfc6\") "
Jul 20 16:03:01.064027 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:01.063934    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97d5ae75-6120-44fb-9e75-6707d9afcfc6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "97d5ae75-6120-44fb-9e75-6707d9afcfc6" (UID: "97d5ae75-6120-44fb-9e75-6707d9afcfc6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:03:01.064206 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:01.064183    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97d5ae75-6120-44fb-9e75-6707d9afcfc6-model-cache" (OuterVolumeSpecName: "model-cache") pod "97d5ae75-6120-44fb-9e75-6707d9afcfc6" (UID: "97d5ae75-6120-44fb-9e75-6707d9afcfc6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:03:01.064206 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:01.064194    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97d5ae75-6120-44fb-9e75-6707d9afcfc6-home" (OuterVolumeSpecName: "home") pod "97d5ae75-6120-44fb-9e75-6707d9afcfc6" (UID: "97d5ae75-6120-44fb-9e75-6707d9afcfc6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:03:01.064537 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:01.064512    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97d5ae75-6120-44fb-9e75-6707d9afcfc6-model-cache\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:03:01.064638 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:01.064540    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97d5ae75-6120-44fb-9e75-6707d9afcfc6-home\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:03:01.064638 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:01.064552    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97d5ae75-6120-44fb-9e75-6707d9afcfc6-tmp-dir\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:03:01.066590 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:01.066568    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97d5ae75-6120-44fb-9e75-6707d9afcfc6-dshm" (OuterVolumeSpecName: "dshm") pod "97d5ae75-6120-44fb-9e75-6707d9afcfc6" (UID: "97d5ae75-6120-44fb-9e75-6707d9afcfc6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:03:01.066590 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:01.066582    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97d5ae75-6120-44fb-9e75-6707d9afcfc6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "97d5ae75-6120-44fb-9e75-6707d9afcfc6" (UID: "97d5ae75-6120-44fb-9e75-6707d9afcfc6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:03:01.066815 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:01.066797    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97d5ae75-6120-44fb-9e75-6707d9afcfc6-kube-api-access-f6f7s" (OuterVolumeSpecName: "kube-api-access-f6f7s") pod "97d5ae75-6120-44fb-9e75-6707d9afcfc6" (UID: "97d5ae75-6120-44fb-9e75-6707d9afcfc6"). InnerVolumeSpecName "kube-api-access-f6f7s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:03:01.165329 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:01.165282    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97d5ae75-6120-44fb-9e75-6707d9afcfc6-tls-certs\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:03:01.165329 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:01.165324    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97d5ae75-6120-44fb-9e75-6707d9afcfc6-dshm\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:03:01.165466 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:01.165333    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6f7s\" (UniqueName: \"kubernetes.io/projected/97d5ae75-6120-44fb-9e75-6707d9afcfc6-kube-api-access-f6f7s\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:03:01.404340 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:01.404264    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:03:01.514055 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:01.514027    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c78c3d61-4968-4979-ac22-b2ac715c345e" path="/var/lib/kubelet/pods/c78c3d61-4968-4979-ac22-b2ac715c345e/volumes"
Jul 20 16:03:01.998110 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:01.998072    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f9d39a/test-model-f9d39a-kserve-76fdbf5689-9wssr" event={"ID":"97d5ae75-6120-44fb-9e75-6707d9afcfc6","Type":"ContainerDied","Data":"d4a74adb3f78e136211e23b423c6dff1def23af62138234e44a358f6d4f2e75c"}
Jul 20 16:03:01.998599 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:01.998127    2559 scope.go:117] "RemoveContainer" containerID="a729323c754dd268dbf79bd76387a994d378fe966dd62fded7f1aff53212697b"
Jul 20 16:03:01.998599 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:01.998088    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f9d39a/test-model-f9d39a-kserve-76fdbf5689-9wssr"
Jul 20 16:03:02.018811 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:02.018780    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-f9d39a/test-model-f9d39a-kserve-76fdbf5689-9wssr"]
Jul 20 16:03:02.020975 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:02.020951    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-f9d39a/test-model-f9d39a-kserve-76fdbf5689-9wssr"]
Jul 20 16:03:03.515426 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:03.515394    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97d5ae75-6120-44fb-9e75-6707d9afcfc6" path="/var/lib/kubelet/pods/97d5ae75-6120-44fb-9e75-6707d9afcfc6/volumes"
Jul 20 16:03:07.797463 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:07.797427    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:03:07.886722 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:07.886692    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:03:08.687768 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:08.687738    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:03:08.856070 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:08.856038    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:03:09.367387 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:09.367355    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:03:09.509710 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:09.509681    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:03:28.015790 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:28.015760    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d39a-688999747f-nhdx8"]
Jul 20 16:03:28.016208 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:28.016034    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d39a-688999747f-nhdx8" podUID="c9137f39-1b8b-4d84-b436-f208db71d773" containerName="maas-api" containerID="cri-o://e77af26d49ea7975f83f15a9c2ff993099de66e5f61599702352086888caca76" gracePeriod=30
Jul 20 16:03:28.052184 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:28.052155    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:03:28.117498 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:28.117471    2559 generic.go:356] "Generic (PLEG): container finished" podID="4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f" containerID="1b546a43dea435b95f56a5aff967f317d24bd92342c527852dff8a268cc4b7c7" exitCode=1
Jul 20 16:03:28.117645 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:28.117544    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-srbk6" event={"ID":"4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f","Type":"ContainerDied","Data":"1b546a43dea435b95f56a5aff967f317d24bd92342c527852dff8a268cc4b7c7"}
Jul 20 16:03:28.117926 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:28.117908    2559 scope.go:117] "RemoveContainer" containerID="1b546a43dea435b95f56a5aff967f317d24bd92342c527852dff8a268cc4b7c7"
Jul 20 16:03:28.118190 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:28.118179    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 16:03:28.293751 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:28.293728    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d39a-688999747f-nhdx8"
Jul 20 16:03:28.397540 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:28.397498    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9137f39-1b8b-4d84-b436-f208db71d773-maas-api-tls\") pod \"c9137f39-1b8b-4d84-b436-f208db71d773\" (UID: \"c9137f39-1b8b-4d84-b436-f208db71d773\") "
Jul 20 16:03:28.397719 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:28.397564    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8z5c8\" (UniqueName: \"kubernetes.io/projected/c9137f39-1b8b-4d84-b436-f208db71d773-kube-api-access-8z5c8\") pod \"c9137f39-1b8b-4d84-b436-f208db71d773\" (UID: \"c9137f39-1b8b-4d84-b436-f208db71d773\") "
Jul 20 16:03:28.397719 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:28.397625    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9137f39-1b8b-4d84-b436-f208db71d773-openshift-service-ca\") pod \"c9137f39-1b8b-4d84-b436-f208db71d773\" (UID: \"c9137f39-1b8b-4d84-b436-f208db71d773\") "
Jul 20 16:03:28.398022 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:28.397996    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9137f39-1b8b-4d84-b436-f208db71d773-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c9137f39-1b8b-4d84-b436-f208db71d773" (UID: "c9137f39-1b8b-4d84-b436-f208db71d773"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:03:28.399600 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:28.399577    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9137f39-1b8b-4d84-b436-f208db71d773-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c9137f39-1b8b-4d84-b436-f208db71d773" (UID: "c9137f39-1b8b-4d84-b436-f208db71d773"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:03:28.399719 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:28.399703    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9137f39-1b8b-4d84-b436-f208db71d773-kube-api-access-8z5c8" (OuterVolumeSpecName: "kube-api-access-8z5c8") pod "c9137f39-1b8b-4d84-b436-f208db71d773" (UID: "c9137f39-1b8b-4d84-b436-f208db71d773"). InnerVolumeSpecName "kube-api-access-8z5c8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:03:28.498906 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:28.498872    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9137f39-1b8b-4d84-b436-f208db71d773-openshift-service-ca\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:03:28.498906 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:28.498902    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9137f39-1b8b-4d84-b436-f208db71d773-maas-api-tls\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:03:28.498906 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:28.498911    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8z5c8\" (UniqueName: \"kubernetes.io/projected/c9137f39-1b8b-4d84-b436-f208db71d773-kube-api-access-8z5c8\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:03:28.581507 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:28.581431    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:03:28.993779 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:28.993740    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:03:29.124216 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:29.124130    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-srbk6" event={"ID":"4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f","Type":"ContainerStarted","Data":"28f82d7b3e0b8fe36529fc4b2a7e266f07f5e4318b95051d9cced73e4a88b96a"}
Jul 20 16:03:29.124651 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:29.124442    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-srbk6"
Jul 20 16:03:29.125386 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:29.125364    2559 generic.go:356] "Generic (PLEG): container finished" podID="c9137f39-1b8b-4d84-b436-f208db71d773" containerID="e77af26d49ea7975f83f15a9c2ff993099de66e5f61599702352086888caca76" exitCode=0
Jul 20 16:03:29.125453 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:29.125406    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d39a-688999747f-nhdx8"
Jul 20 16:03:29.125490 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:29.125443    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d39a-688999747f-nhdx8" event={"ID":"c9137f39-1b8b-4d84-b436-f208db71d773","Type":"ContainerDied","Data":"e77af26d49ea7975f83f15a9c2ff993099de66e5f61599702352086888caca76"}
Jul 20 16:03:29.125525 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:29.125490    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d39a-688999747f-nhdx8" event={"ID":"c9137f39-1b8b-4d84-b436-f208db71d773","Type":"ContainerDied","Data":"6691a4f1da0e3b0c0d8002f1aafa40a60d35ed0a230416d6561ce8d199207f56"}
Jul 20 16:03:29.125525 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:29.125514    2559 scope.go:117] "RemoveContainer" containerID="e77af26d49ea7975f83f15a9c2ff993099de66e5f61599702352086888caca76"
Jul 20 16:03:29.133979 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:29.133959    2559 scope.go:117] "RemoveContainer" containerID="e77af26d49ea7975f83f15a9c2ff993099de66e5f61599702352086888caca76"
Jul 20 16:03:29.134224 ip-10-0-130-19 kubenswrapper[2559]: E0720 16:03:29.134206    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e77af26d49ea7975f83f15a9c2ff993099de66e5f61599702352086888caca76\": container with ID starting with e77af26d49ea7975f83f15a9c2ff993099de66e5f61599702352086888caca76 not found: ID does not exist" containerID="e77af26d49ea7975f83f15a9c2ff993099de66e5f61599702352086888caca76"
Jul 20 16:03:29.134428 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:29.134232    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e77af26d49ea7975f83f15a9c2ff993099de66e5f61599702352086888caca76"} err="failed to get container status \"e77af26d49ea7975f83f15a9c2ff993099de66e5f61599702352086888caca76\": rpc error: code = NotFound desc = could not find container \"e77af26d49ea7975f83f15a9c2ff993099de66e5f61599702352086888caca76\": container with ID starting with e77af26d49ea7975f83f15a9c2ff993099de66e5f61599702352086888caca76 not found: ID does not exist"
Jul 20 16:03:29.167781 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:29.167756    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d39a-688999747f-nhdx8"]
Jul 20 16:03:29.172660 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:29.172640    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d39a-688999747f-nhdx8"]
Jul 20 16:03:29.513917 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:29.513845    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9137f39-1b8b-4d84-b436-f208db71d773" path="/var/lib/kubelet/pods/c9137f39-1b8b-4d84-b436-f208db71d773/volumes"
Jul 20 16:03:31.420557 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:31.420526    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:03:31.534677 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:31.534646    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:03:35.981057 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:35.981030    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:03:36.085553 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:36.085511    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:03:36.269223 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:36.269192    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"]
Jul 20 16:03:36.269422 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:36.269401    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf" podUID="92ccc397-b2d7-49c1-b915-54b4742c9c8e" containerName="istio-proxy" containerID="cri-o://ce14c51bfe3f9bffc9993f8b90ea8b8a6019dfb70bece89cc0d7d8ea2bfeb339" gracePeriod=30
Jul 20 16:03:36.451017 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:36.450988    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:03:40.132454 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:40.132426    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-srbk6"
Jul 20 16:03:41.519506 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:41.519484    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"
Jul 20 16:03:41.707946 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:41.707882    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92ccc397-b2d7-49c1-b915-54b4742c9c8e-credential-socket\") pod \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\" (UID: \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\") "
Jul 20 16:03:41.707946 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:41.707917    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92ccc397-b2d7-49c1-b915-54b4742c9c8e-istiod-ca-cert\") pod \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\" (UID: \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\") "
Jul 20 16:03:41.707946 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:41.707938    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92ccc397-b2d7-49c1-b915-54b4742c9c8e-workload-socket\") pod \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\" (UID: \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\") "
Jul 20 16:03:41.708220 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:41.707956    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92ccc397-b2d7-49c1-b915-54b4742c9c8e-workload-certs\") pod \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\" (UID: \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\") "
Jul 20 16:03:41.708220 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:41.707984    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92ccc397-b2d7-49c1-b915-54b4742c9c8e-istio-token\") pod \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\" (UID: \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\") "
Jul 20 16:03:41.708220 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:41.708025    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92ccc397-b2d7-49c1-b915-54b4742c9c8e-istio-podinfo\") pod \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\" (UID: \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\") "
Jul 20 16:03:41.708220 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:41.708068    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-566hm\" (UniqueName: \"kubernetes.io/projected/92ccc397-b2d7-49c1-b915-54b4742c9c8e-kube-api-access-566hm\") pod \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\" (UID: \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\") "
Jul 20 16:03:41.708220 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:41.708094    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92ccc397-b2d7-49c1-b915-54b4742c9c8e-istio-envoy\") pod \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\" (UID: \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\") "
Jul 20 16:03:41.708501 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:41.708253    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92ccc397-b2d7-49c1-b915-54b4742c9c8e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "92ccc397-b2d7-49c1-b915-54b4742c9c8e" (UID: "92ccc397-b2d7-49c1-b915-54b4742c9c8e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:03:41.708501 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:41.708272    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92ccc397-b2d7-49c1-b915-54b4742c9c8e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "92ccc397-b2d7-49c1-b915-54b4742c9c8e" (UID: "92ccc397-b2d7-49c1-b915-54b4742c9c8e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:03:41.708501 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:41.708332    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92ccc397-b2d7-49c1-b915-54b4742c9c8e-istio-data\") pod \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\" (UID: \"92ccc397-b2d7-49c1-b915-54b4742c9c8e\") "
Jul 20 16:03:41.708501 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:41.708342    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92ccc397-b2d7-49c1-b915-54b4742c9c8e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "92ccc397-b2d7-49c1-b915-54b4742c9c8e" (UID: "92ccc397-b2d7-49c1-b915-54b4742c9c8e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:03:41.708711 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:41.708497    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92ccc397-b2d7-49c1-b915-54b4742c9c8e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "92ccc397-b2d7-49c1-b915-54b4742c9c8e" (UID: "92ccc397-b2d7-49c1-b915-54b4742c9c8e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:03:41.708711 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:41.708642    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92ccc397-b2d7-49c1-b915-54b4742c9c8e-credential-socket\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:03:41.708711 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:41.708660    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92ccc397-b2d7-49c1-b915-54b4742c9c8e-istiod-ca-cert\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:03:41.708711 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:41.708673    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92ccc397-b2d7-49c1-b915-54b4742c9c8e-workload-socket\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:03:41.708711 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:41.708684    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92ccc397-b2d7-49c1-b915-54b4742c9c8e-workload-certs\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:03:41.709183 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:41.709161    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92ccc397-b2d7-49c1-b915-54b4742c9c8e-istio-data" (OuterVolumeSpecName: "istio-data") pod "92ccc397-b2d7-49c1-b915-54b4742c9c8e" (UID: "92ccc397-b2d7-49c1-b915-54b4742c9c8e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:03:41.710318 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:41.710276    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92ccc397-b2d7-49c1-b915-54b4742c9c8e-kube-api-access-566hm" (OuterVolumeSpecName: "kube-api-access-566hm") pod "92ccc397-b2d7-49c1-b915-54b4742c9c8e" (UID: "92ccc397-b2d7-49c1-b915-54b4742c9c8e"). InnerVolumeSpecName "kube-api-access-566hm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:03:41.710417 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:41.710303    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92ccc397-b2d7-49c1-b915-54b4742c9c8e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "92ccc397-b2d7-49c1-b915-54b4742c9c8e" (UID: "92ccc397-b2d7-49c1-b915-54b4742c9c8e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:03:41.710678 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:41.710649    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92ccc397-b2d7-49c1-b915-54b4742c9c8e-istio-token" (OuterVolumeSpecName: "istio-token") pod "92ccc397-b2d7-49c1-b915-54b4742c9c8e" (UID: "92ccc397-b2d7-49c1-b915-54b4742c9c8e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:03:41.710678 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:41.710664    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/92ccc397-b2d7-49c1-b915-54b4742c9c8e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "92ccc397-b2d7-49c1-b915-54b4742c9c8e" (UID: "92ccc397-b2d7-49c1-b915-54b4742c9c8e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 20 16:03:41.809515 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:41.809469    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-566hm\" (UniqueName: \"kubernetes.io/projected/92ccc397-b2d7-49c1-b915-54b4742c9c8e-kube-api-access-566hm\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:03:41.809515 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:41.809513    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92ccc397-b2d7-49c1-b915-54b4742c9c8e-istio-envoy\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:03:41.809515 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:41.809522    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92ccc397-b2d7-49c1-b915-54b4742c9c8e-istio-data\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:03:41.809697 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:41.809530    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92ccc397-b2d7-49c1-b915-54b4742c9c8e-istio-token\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:03:41.809697 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:41.809538    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92ccc397-b2d7-49c1-b915-54b4742c9c8e-istio-podinfo\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:03:42.186905 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:42.186872    2559 generic.go:356] "Generic (PLEG): container finished" podID="92ccc397-b2d7-49c1-b915-54b4742c9c8e" containerID="ce14c51bfe3f9bffc9993f8b90ea8b8a6019dfb70bece89cc0d7d8ea2bfeb339" exitCode=0
Jul 20 16:03:42.187051 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:42.186927    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"
Jul 20 16:03:42.187051 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:42.186957    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf" event={"ID":"92ccc397-b2d7-49c1-b915-54b4742c9c8e","Type":"ContainerDied","Data":"ce14c51bfe3f9bffc9993f8b90ea8b8a6019dfb70bece89cc0d7d8ea2bfeb339"}
Jul 20 16:03:42.187051 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:42.187013    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf" event={"ID":"92ccc397-b2d7-49c1-b915-54b4742c9c8e","Type":"ContainerDied","Data":"87b2104bffa9f3ddd3fe4c813999cf8d939aa71d03091d15dff773ab1a9884de"}
Jul 20 16:03:42.187051 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:42.187037    2559 scope.go:117] "RemoveContainer" containerID="ce14c51bfe3f9bffc9993f8b90ea8b8a6019dfb70bece89cc0d7d8ea2bfeb339"
Jul 20 16:03:42.196171 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:42.196154    2559 scope.go:117] "RemoveContainer" containerID="deaa11bf8bf0ae820d1a780e57846341328e7420c3e2510ef1493583b862be29"
Jul 20 16:03:42.203811 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:42.203795    2559 scope.go:117] "RemoveContainer" containerID="ce14c51bfe3f9bffc9993f8b90ea8b8a6019dfb70bece89cc0d7d8ea2bfeb339"
Jul 20 16:03:42.204066 ip-10-0-130-19 kubenswrapper[2559]: E0720 16:03:42.204047    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce14c51bfe3f9bffc9993f8b90ea8b8a6019dfb70bece89cc0d7d8ea2bfeb339\": container with ID starting with ce14c51bfe3f9bffc9993f8b90ea8b8a6019dfb70bece89cc0d7d8ea2bfeb339 not found: ID does not exist" containerID="ce14c51bfe3f9bffc9993f8b90ea8b8a6019dfb70bece89cc0d7d8ea2bfeb339"
Jul 20 16:03:42.204117 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:42.204075    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce14c51bfe3f9bffc9993f8b90ea8b8a6019dfb70bece89cc0d7d8ea2bfeb339"} err="failed to get container status \"ce14c51bfe3f9bffc9993f8b90ea8b8a6019dfb70bece89cc0d7d8ea2bfeb339\": rpc error: code = NotFound desc = could not find container \"ce14c51bfe3f9bffc9993f8b90ea8b8a6019dfb70bece89cc0d7d8ea2bfeb339\": container with ID starting with ce14c51bfe3f9bffc9993f8b90ea8b8a6019dfb70bece89cc0d7d8ea2bfeb339 not found: ID does not exist"
Jul 20 16:03:42.204117 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:42.204094    2559 scope.go:117] "RemoveContainer" containerID="deaa11bf8bf0ae820d1a780e57846341328e7420c3e2510ef1493583b862be29"
Jul 20 16:03:42.204327 ip-10-0-130-19 kubenswrapper[2559]: E0720 16:03:42.204308    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"deaa11bf8bf0ae820d1a780e57846341328e7420c3e2510ef1493583b862be29\": container with ID starting with deaa11bf8bf0ae820d1a780e57846341328e7420c3e2510ef1493583b862be29 not found: ID does not exist" containerID="deaa11bf8bf0ae820d1a780e57846341328e7420c3e2510ef1493583b862be29"
Jul 20 16:03:42.204380 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:42.204335    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"deaa11bf8bf0ae820d1a780e57846341328e7420c3e2510ef1493583b862be29"} err="failed to get container status \"deaa11bf8bf0ae820d1a780e57846341328e7420c3e2510ef1493583b862be29\": rpc error: code = NotFound desc = could not find container \"deaa11bf8bf0ae820d1a780e57846341328e7420c3e2510ef1493583b862be29\": container with ID starting with deaa11bf8bf0ae820d1a780e57846341328e7420c3e2510ef1493583b862be29 not found: ID does not exist"
Jul 20 16:03:42.209448 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:42.209407    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"]
Jul 20 16:03:42.211216 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:42.211193    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-f9d39a-openshift-default-794ddc7f75-gmltf"]
Jul 20 16:03:43.513731 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:43.513694    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92ccc397-b2d7-49c1-b915-54b4742c9c8e" path="/var/lib/kubelet/pods/92ccc397-b2d7-49c1-b915-54b4742c9c8e/volumes"
Jul 20 16:03:43.787093 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:43.787062    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:03:44.012952 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:44.012923    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:03:44.383098 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:44.383069    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:03:44.531302 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:44.531259    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:03:47.959588 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:47.959548    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:03:48.080351 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:48.080322    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq"]
Jul 20 16:03:48.080686 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:48.080674    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9137f39-1b8b-4d84-b436-f208db71d773" containerName="maas-api"
Jul 20 16:03:48.080729 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:48.080687    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9137f39-1b8b-4d84-b436-f208db71d773" containerName="maas-api"
Jul 20 16:03:48.080729 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:48.080700    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c78c3d61-4968-4979-ac22-b2ac715c345e" containerName="main"
Jul 20 16:03:48.080729 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:48.080706    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="c78c3d61-4968-4979-ac22-b2ac715c345e" containerName="main"
Jul 20 16:03:48.080729 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:48.080721    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92ccc397-b2d7-49c1-b915-54b4742c9c8e" containerName="istio-proxy"
Jul 20 16:03:48.080729 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:48.080727    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="92ccc397-b2d7-49c1-b915-54b4742c9c8e" containerName="istio-proxy"
Jul 20 16:03:48.080884 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:48.080736    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f5e48d8-f896-47af-adee-9927a8819970" containerName="maas-api"
Jul 20 16:03:48.080884 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:48.080741    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f5e48d8-f896-47af-adee-9927a8819970" containerName="maas-api"
Jul 20 16:03:48.080884 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:48.080751    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97d5ae75-6120-44fb-9e75-6707d9afcfc6" containerName="main"
Jul 20 16:03:48.080884 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:48.080756    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="97d5ae75-6120-44fb-9e75-6707d9afcfc6" containerName="main"
Jul 20 16:03:48.080884 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:48.080805    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="92ccc397-b2d7-49c1-b915-54b4742c9c8e" containerName="istio-proxy"
Jul 20 16:03:48.080884 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:48.080811    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="c78c3d61-4968-4979-ac22-b2ac715c345e" containerName="main"
Jul 20 16:03:48.080884 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:48.080817    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9137f39-1b8b-4d84-b436-f208db71d773" containerName="maas-api"
Jul 20 16:03:48.080884 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:48.080825    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f5e48d8-f896-47af-adee-9927a8819970" containerName="maas-api"
Jul 20 16:03:48.080884 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:48.080831    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="92ccc397-b2d7-49c1-b915-54b4742c9c8e" containerName="istio-proxy"
Jul 20 16:03:48.080884 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:48.080836    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="97d5ae75-6120-44fb-9e75-6707d9afcfc6" containerName="main"
Jul 20 16:03:48.085803 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:48.085785    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq"
Jul 20 16:03:48.089476 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:48.089450    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-5a8c68-serving-cert\""
Jul 20 16:03:48.089593 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:48.089539    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-96kqc\""
Jul 20 16:03:48.094355 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:48.094331    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:03:48.101331 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:48.101279    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq"]
Jul 20 16:03:48.155178 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:48.155149    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/203ccc6c-38d8-463d-9302-57865132e8d4-openshift-service-ca\") pod \"maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq\" (UID: \"203ccc6c-38d8-463d-9302-57865132e8d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq"
Jul 20 16:03:48.155382 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:48.155191    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/203ccc6c-38d8-463d-9302-57865132e8d4-maas-api-tls\") pod \"maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq\" (UID: \"203ccc6c-38d8-463d-9302-57865132e8d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq"
Jul 20 16:03:48.155382 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:48.155222    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzkf4\" (UniqueName: \"kubernetes.io/projected/203ccc6c-38d8-463d-9302-57865132e8d4-kube-api-access-qzkf4\") pod \"maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq\" (UID: \"203ccc6c-38d8-463d-9302-57865132e8d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq"
Jul 20 16:03:48.211544 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:48.211462    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:03:48.256529 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:48.256492    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/203ccc6c-38d8-463d-9302-57865132e8d4-openshift-service-ca\") pod \"maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq\" (UID: \"203ccc6c-38d8-463d-9302-57865132e8d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq"
Jul 20 16:03:48.256780 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:48.256542    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/203ccc6c-38d8-463d-9302-57865132e8d4-maas-api-tls\") pod \"maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq\" (UID: \"203ccc6c-38d8-463d-9302-57865132e8d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq"
Jul 20 16:03:48.256780 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:48.256583    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzkf4\" (UniqueName: \"kubernetes.io/projected/203ccc6c-38d8-463d-9302-57865132e8d4-kube-api-access-qzkf4\") pod \"maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq\" (UID: \"203ccc6c-38d8-463d-9302-57865132e8d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq"
Jul 20 16:03:48.256780 ip-10-0-130-19 kubenswrapper[2559]: E0720 16:03:48.256737    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-5a8c68-serving-cert: secret "maas-api-e2e-api-a-5a8c68-serving-cert" not found
Jul 20 16:03:48.256994 ip-10-0-130-19 kubenswrapper[2559]: E0720 16:03:48.256809    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/203ccc6c-38d8-463d-9302-57865132e8d4-maas-api-tls podName:203ccc6c-38d8-463d-9302-57865132e8d4 nodeName:}" failed. No retries permitted until 2026-07-20 16:03:48.756791862 +0000 UTC m=+2211.766413643 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/203ccc6c-38d8-463d-9302-57865132e8d4-maas-api-tls") pod "maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq" (UID: "203ccc6c-38d8-463d-9302-57865132e8d4") : secret "maas-api-e2e-api-a-5a8c68-serving-cert" not found
Jul 20 16:03:48.257278 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:48.257254    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/203ccc6c-38d8-463d-9302-57865132e8d4-openshift-service-ca\") pod \"maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq\" (UID: \"203ccc6c-38d8-463d-9302-57865132e8d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq"
Jul 20 16:03:48.275954 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:48.275926    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzkf4\" (UniqueName: \"kubernetes.io/projected/203ccc6c-38d8-463d-9302-57865132e8d4-kube-api-access-qzkf4\") pod \"maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq\" (UID: \"203ccc6c-38d8-463d-9302-57865132e8d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq"
Jul 20 16:03:48.761079 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:48.761045    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/203ccc6c-38d8-463d-9302-57865132e8d4-maas-api-tls\") pod \"maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq\" (UID: \"203ccc6c-38d8-463d-9302-57865132e8d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq"
Jul 20 16:03:48.761248 ip-10-0-130-19 kubenswrapper[2559]: E0720 16:03:48.761155    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-5a8c68-serving-cert: secret "maas-api-e2e-api-a-5a8c68-serving-cert" not found
Jul 20 16:03:48.761248 ip-10-0-130-19 kubenswrapper[2559]: E0720 16:03:48.761207    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/203ccc6c-38d8-463d-9302-57865132e8d4-maas-api-tls podName:203ccc6c-38d8-463d-9302-57865132e8d4 nodeName:}" failed. No retries permitted until 2026-07-20 16:03:49.761192924 +0000 UTC m=+2212.770814566 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/203ccc6c-38d8-463d-9302-57865132e8d4-maas-api-tls") pod "maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq" (UID: "203ccc6c-38d8-463d-9302-57865132e8d4") : secret "maas-api-e2e-api-a-5a8c68-serving-cert" not found
Jul 20 16:03:48.993546 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:48.993506    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:03:49.152164 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:49.152126    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:03:49.280173 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:49.280143    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:03:49.770071 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:49.770034    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/203ccc6c-38d8-463d-9302-57865132e8d4-maas-api-tls\") pod \"maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq\" (UID: \"203ccc6c-38d8-463d-9302-57865132e8d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq"
Jul 20 16:03:49.772330 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:49.772299    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/203ccc6c-38d8-463d-9302-57865132e8d4-maas-api-tls\") pod \"maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq\" (UID: \"203ccc6c-38d8-463d-9302-57865132e8d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq"
Jul 20 16:03:49.896008 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:49.895970    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq"
Jul 20 16:03:50.048224 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:50.048140    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq"]
Jul 20 16:03:50.051089 ip-10-0-130-19 kubenswrapper[2559]: W0720 16:03:50.051057    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod203ccc6c_38d8_463d_9302_57865132e8d4.slice/crio-9708117c5049affb53bfc9d7f00a6fbe3c9d3463065db79f068b2f85b99c6fea WatchSource:0}: Error finding container 9708117c5049affb53bfc9d7f00a6fbe3c9d3463065db79f068b2f85b99c6fea: Status 404 returned error can't find the container with id 9708117c5049affb53bfc9d7f00a6fbe3c9d3463065db79f068b2f85b99c6fea
Jul 20 16:03:50.225715 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:50.225679    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq" event={"ID":"203ccc6c-38d8-463d-9302-57865132e8d4","Type":"ContainerStarted","Data":"9708117c5049affb53bfc9d7f00a6fbe3c9d3463065db79f068b2f85b99c6fea"}
Jul 20 16:03:51.231440 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:51.231408    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq" event={"ID":"203ccc6c-38d8-463d-9302-57865132e8d4","Type":"ContainerStarted","Data":"9fd22a7674c4f7d27304e1520d99cf6709eb5c7c33c6ff5de7f3d49e5d4e2c2d"}
Jul 20 16:03:51.231805 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:51.231473    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq"
Jul 20 16:03:51.251499 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:51.251452    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq" podStartSLOduration=2.575861289 podStartE2EDuration="3.251437918s" totalImagesPullingTime="675.576629ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:03:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:03:51.249513823 +0000 UTC m=+2214.259135487" watchObservedRunningTime="2026-07-20 16:03:51.251437918 +0000 UTC m=+2214.261059581"
Jul 20 16:03:52.419132 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:52.419102    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:03:52.924332 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:52.924275    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:03:53.206405 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.206323    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"]
Jul 20 16:03:53.206874 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.206856    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92ccc397-b2d7-49c1-b915-54b4742c9c8e" containerName="istio-proxy"
Jul 20 16:03:53.206874 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.206875    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="92ccc397-b2d7-49c1-b915-54b4742c9c8e" containerName="istio-proxy"
Jul 20 16:03:53.210315 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.210281    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"
Jul 20 16:03:53.214478 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.214459    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-4a2d96-openshift-default-dockercfg-zg7m9\""
Jul 20 16:03:53.216336 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.216314    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 20 16:03:53.227151 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.227126    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"]
Jul 20 16:03:53.254785 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.254733    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:03:53.297771 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.297739    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-np2pl\" (UniqueName: \"kubernetes.io/projected/dfb761b9-e224-49a8-81e6-058a358422a2-kube-api-access-np2pl\") pod \"e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8\" (UID: \"dfb761b9-e224-49a8-81e6-058a358422a2\") " pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"
Jul 20 16:03:53.297884 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.297782    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dfb761b9-e224-49a8-81e6-058a358422a2-istio-podinfo\") pod \"e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8\" (UID: \"dfb761b9-e224-49a8-81e6-058a358422a2\") " pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"
Jul 20 16:03:53.297884 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.297841    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dfb761b9-e224-49a8-81e6-058a358422a2-istio-data\") pod \"e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8\" (UID: \"dfb761b9-e224-49a8-81e6-058a358422a2\") " pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"
Jul 20 16:03:53.298005 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.297894    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfb761b9-e224-49a8-81e6-058a358422a2-credential-socket\") pod \"e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8\" (UID: \"dfb761b9-e224-49a8-81e6-058a358422a2\") " pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"
Jul 20 16:03:53.298005 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.297918    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dfb761b9-e224-49a8-81e6-058a358422a2-istio-envoy\") pod \"e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8\" (UID: \"dfb761b9-e224-49a8-81e6-058a358422a2\") " pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"
Jul 20 16:03:53.298005 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.297955    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dfb761b9-e224-49a8-81e6-058a358422a2-workload-certs\") pod \"e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8\" (UID: \"dfb761b9-e224-49a8-81e6-058a358422a2\") " pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"
Jul 20 16:03:53.298159 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.298015    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dfb761b9-e224-49a8-81e6-058a358422a2-istio-token\") pod \"e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8\" (UID: \"dfb761b9-e224-49a8-81e6-058a358422a2\") " pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"
Jul 20 16:03:53.298159 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.298044    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dfb761b9-e224-49a8-81e6-058a358422a2-istiod-ca-cert\") pod \"e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8\" (UID: \"dfb761b9-e224-49a8-81e6-058a358422a2\") " pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"
Jul 20 16:03:53.298159 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.298098    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfb761b9-e224-49a8-81e6-058a358422a2-workload-socket\") pod \"e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8\" (UID: \"dfb761b9-e224-49a8-81e6-058a358422a2\") " pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"
Jul 20 16:03:53.399374 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.399339    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dfb761b9-e224-49a8-81e6-058a358422a2-istio-token\") pod \"e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8\" (UID: \"dfb761b9-e224-49a8-81e6-058a358422a2\") " pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"
Jul 20 16:03:53.399374 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.399378    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dfb761b9-e224-49a8-81e6-058a358422a2-istiod-ca-cert\") pod \"e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8\" (UID: \"dfb761b9-e224-49a8-81e6-058a358422a2\") " pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"
Jul 20 16:03:53.399600 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.399399    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfb761b9-e224-49a8-81e6-058a358422a2-workload-socket\") pod \"e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8\" (UID: \"dfb761b9-e224-49a8-81e6-058a358422a2\") " pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"
Jul 20 16:03:53.399600 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.399530    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-np2pl\" (UniqueName: \"kubernetes.io/projected/dfb761b9-e224-49a8-81e6-058a358422a2-kube-api-access-np2pl\") pod \"e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8\" (UID: \"dfb761b9-e224-49a8-81e6-058a358422a2\") " pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"
Jul 20 16:03:53.399600 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.399578    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dfb761b9-e224-49a8-81e6-058a358422a2-istio-podinfo\") pod \"e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8\" (UID: \"dfb761b9-e224-49a8-81e6-058a358422a2\") " pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"
Jul 20 16:03:53.399758 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.399628    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dfb761b9-e224-49a8-81e6-058a358422a2-istio-data\") pod \"e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8\" (UID: \"dfb761b9-e224-49a8-81e6-058a358422a2\") " pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"
Jul 20 16:03:53.399758 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.399663    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfb761b9-e224-49a8-81e6-058a358422a2-credential-socket\") pod \"e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8\" (UID: \"dfb761b9-e224-49a8-81e6-058a358422a2\") " pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"
Jul 20 16:03:53.399758 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.399697    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dfb761b9-e224-49a8-81e6-058a358422a2-istio-envoy\") pod \"e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8\" (UID: \"dfb761b9-e224-49a8-81e6-058a358422a2\") " pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"
Jul 20 16:03:53.399758 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.399724    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dfb761b9-e224-49a8-81e6-058a358422a2-workload-certs\") pod \"e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8\" (UID: \"dfb761b9-e224-49a8-81e6-058a358422a2\") " pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"
Jul 20 16:03:53.399908 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.399834    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfb761b9-e224-49a8-81e6-058a358422a2-workload-socket\") pod \"e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8\" (UID: \"dfb761b9-e224-49a8-81e6-058a358422a2\") " pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"
Jul 20 16:03:53.400069 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.400048    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dfb761b9-e224-49a8-81e6-058a358422a2-istio-data\") pod \"e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8\" (UID: \"dfb761b9-e224-49a8-81e6-058a358422a2\") " pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"
Jul 20 16:03:53.400143 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.400110    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dfb761b9-e224-49a8-81e6-058a358422a2-istiod-ca-cert\") pod \"e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8\" (UID: \"dfb761b9-e224-49a8-81e6-058a358422a2\") " pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"
Jul 20 16:03:53.400143 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.400127    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dfb761b9-e224-49a8-81e6-058a358422a2-workload-certs\") pod \"e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8\" (UID: \"dfb761b9-e224-49a8-81e6-058a358422a2\") " pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"
Jul 20 16:03:53.400143 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.400133    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfb761b9-e224-49a8-81e6-058a358422a2-credential-socket\") pod \"e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8\" (UID: \"dfb761b9-e224-49a8-81e6-058a358422a2\") " pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"
Jul 20 16:03:53.402026 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.401989    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dfb761b9-e224-49a8-81e6-058a358422a2-istio-envoy\") pod \"e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8\" (UID: \"dfb761b9-e224-49a8-81e6-058a358422a2\") " pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"
Jul 20 16:03:53.402151 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.402129    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dfb761b9-e224-49a8-81e6-058a358422a2-istio-podinfo\") pod \"e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8\" (UID: \"dfb761b9-e224-49a8-81e6-058a358422a2\") " pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"
Jul 20 16:03:53.407455 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.407430    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-np2pl\" (UniqueName: \"kubernetes.io/projected/dfb761b9-e224-49a8-81e6-058a358422a2-kube-api-access-np2pl\") pod \"e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8\" (UID: \"dfb761b9-e224-49a8-81e6-058a358422a2\") " pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"
Jul 20 16:03:53.407532 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.407491    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dfb761b9-e224-49a8-81e6-058a358422a2-istio-token\") pod \"e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8\" (UID: \"dfb761b9-e224-49a8-81e6-058a358422a2\") " pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"
Jul 20 16:03:53.520540 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.520517    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"
Jul 20 16:03:53.629443 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.629408    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:03:53.658817 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.658796    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"]
Jul 20 16:03:53.660571 ip-10-0-130-19 kubenswrapper[2559]: W0720 16:03:53.660541    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfb761b9_e224_49a8_81e6_058a358422a2.slice/crio-16159a1c6a00540d0f3759fc36ef9a3dc281acf3811de7252eb44159b88e7af4 WatchSource:0}: Error finding container 16159a1c6a00540d0f3759fc36ef9a3dc281acf3811de7252eb44159b88e7af4: Status 404 returned error can't find the container with id 16159a1c6a00540d0f3759fc36ef9a3dc281acf3811de7252eb44159b88e7af4
Jul 20 16:03:53.662831 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.662799    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 16:03:53.662915 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.662865    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 16:03:53.662954 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:53.662913    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 16:03:54.251491 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:54.251452    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8" event={"ID":"dfb761b9-e224-49a8-81e6-058a358422a2","Type":"ContainerStarted","Data":"86d82c4d58374a1e21940f18449dcddac74a34791bb3cab810b3a84caf551582"}
Jul 20 16:03:54.251668 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:54.251537    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8" event={"ID":"dfb761b9-e224-49a8-81e6-058a358422a2","Type":"ContainerStarted","Data":"16159a1c6a00540d0f3759fc36ef9a3dc281acf3811de7252eb44159b88e7af4"}
Jul 20 16:03:54.278715 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:54.278654    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8" podStartSLOduration=1.278640018 podStartE2EDuration="1.278640018s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:03:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:03:54.277698771 +0000 UTC m=+2217.287320433" watchObservedRunningTime="2026-07-20 16:03:54.278640018 +0000 UTC m=+2217.288261679"
Jul 20 16:03:54.521325 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:54.521282    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"
Jul 20 16:03:54.525936 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:54.525912    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"
Jul 20 16:03:55.255851 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:55.255815    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"
Jul 20 16:03:55.256759 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:55.256738    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"
Jul 20 16:03:56.238316 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:56.238262    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:03:56.314732 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:56.314704    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4a2d96-56d96c454-l7wfp"]
Jul 20 16:03:56.318406 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:56.318385    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4a2d96-56d96c454-l7wfp"
Jul 20 16:03:56.322332 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:56.322314    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-4a2d96-serving-cert\""
Jul 20 16:03:56.332874 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:56.332851    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4a2d96-56d96c454-l7wfp"]
Jul 20 16:03:56.334459 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:56.334434    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:03:56.426691 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:56.426655    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceb8e0fd-a233-492e-90af-52fbc4a7a9b4-openshift-service-ca\") pod \"maas-api-e2e-api-b-4a2d96-56d96c454-l7wfp\" (UID: \"ceb8e0fd-a233-492e-90af-52fbc4a7a9b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4a2d96-56d96c454-l7wfp"
Jul 20 16:03:56.426843 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:56.426772    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd9qs\" (UniqueName: \"kubernetes.io/projected/ceb8e0fd-a233-492e-90af-52fbc4a7a9b4-kube-api-access-dd9qs\") pod \"maas-api-e2e-api-b-4a2d96-56d96c454-l7wfp\" (UID: \"ceb8e0fd-a233-492e-90af-52fbc4a7a9b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4a2d96-56d96c454-l7wfp"
Jul 20 16:03:56.426843 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:56.426817    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ceb8e0fd-a233-492e-90af-52fbc4a7a9b4-maas-api-tls\") pod \"maas-api-e2e-api-b-4a2d96-56d96c454-l7wfp\" (UID: \"ceb8e0fd-a233-492e-90af-52fbc4a7a9b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4a2d96-56d96c454-l7wfp"
Jul 20 16:03:56.433771 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:56.433749    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:03:56.527962 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:56.527938    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceb8e0fd-a233-492e-90af-52fbc4a7a9b4-openshift-service-ca\") pod \"maas-api-e2e-api-b-4a2d96-56d96c454-l7wfp\" (UID: \"ceb8e0fd-a233-492e-90af-52fbc4a7a9b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4a2d96-56d96c454-l7wfp"
Jul 20 16:03:56.528091 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:56.527988    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd9qs\" (UniqueName: \"kubernetes.io/projected/ceb8e0fd-a233-492e-90af-52fbc4a7a9b4-kube-api-access-dd9qs\") pod \"maas-api-e2e-api-b-4a2d96-56d96c454-l7wfp\" (UID: \"ceb8e0fd-a233-492e-90af-52fbc4a7a9b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4a2d96-56d96c454-l7wfp"
Jul 20 16:03:56.528091 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:56.528014    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ceb8e0fd-a233-492e-90af-52fbc4a7a9b4-maas-api-tls\") pod \"maas-api-e2e-api-b-4a2d96-56d96c454-l7wfp\" (UID: \"ceb8e0fd-a233-492e-90af-52fbc4a7a9b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4a2d96-56d96c454-l7wfp"
Jul 20 16:03:56.528637 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:56.528599    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceb8e0fd-a233-492e-90af-52fbc4a7a9b4-openshift-service-ca\") pod \"maas-api-e2e-api-b-4a2d96-56d96c454-l7wfp\" (UID: \"ceb8e0fd-a233-492e-90af-52fbc4a7a9b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4a2d96-56d96c454-l7wfp"
Jul 20 16:03:56.530341 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:56.530316    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ceb8e0fd-a233-492e-90af-52fbc4a7a9b4-maas-api-tls\") pod \"maas-api-e2e-api-b-4a2d96-56d96c454-l7wfp\" (UID: \"ceb8e0fd-a233-492e-90af-52fbc4a7a9b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4a2d96-56d96c454-l7wfp"
Jul 20 16:03:56.539060 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:56.539044    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd9qs\" (UniqueName: \"kubernetes.io/projected/ceb8e0fd-a233-492e-90af-52fbc4a7a9b4-kube-api-access-dd9qs\") pod \"maas-api-e2e-api-b-4a2d96-56d96c454-l7wfp\" (UID: \"ceb8e0fd-a233-492e-90af-52fbc4a7a9b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4a2d96-56d96c454-l7wfp"
Jul 20 16:03:56.605225 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:56.605194    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:03:56.629326 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:56.629278    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4a2d96-56d96c454-l7wfp"
Jul 20 16:03:56.760839 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:56.760818    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4a2d96-56d96c454-l7wfp"]
Jul 20 16:03:56.762558 ip-10-0-130-19 kubenswrapper[2559]: W0720 16:03:56.762527    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podceb8e0fd_a233_492e_90af_52fbc4a7a9b4.slice/crio-b4eeb46b5390bfba4498f9d20f90aa2ac7921f3d5e474d4172136f7e78d0acf6 WatchSource:0}: Error finding container b4eeb46b5390bfba4498f9d20f90aa2ac7921f3d5e474d4172136f7e78d0acf6: Status 404 returned error can't find the container with id b4eeb46b5390bfba4498f9d20f90aa2ac7921f3d5e474d4172136f7e78d0acf6
Jul 20 16:03:56.916163 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:56.916100    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:03:57.013827 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:57.013805    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:03:57.242348 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:57.242330    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq"
Jul 20 16:03:57.266061 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:57.266032    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4a2d96-56d96c454-l7wfp" event={"ID":"ceb8e0fd-a233-492e-90af-52fbc4a7a9b4","Type":"ContainerStarted","Data":"ead3f714fb7c9afe482ab6daa9ce94bf81132a104f67661f1ce7d95f9f11de76"}
Jul 20 16:03:57.266176 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:57.266085    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4a2d96-56d96c454-l7wfp" event={"ID":"ceb8e0fd-a233-492e-90af-52fbc4a7a9b4","Type":"ContainerStarted","Data":"b4eeb46b5390bfba4498f9d20f90aa2ac7921f3d5e474d4172136f7e78d0acf6"}
Jul 20 16:03:57.266361 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:57.266339    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4a2d96-56d96c454-l7wfp"
Jul 20 16:03:57.360852 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:03:57.360795    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4a2d96-56d96c454-l7wfp" podStartSLOduration=0.9406803 podStartE2EDuration="1.360779013s" totalImagesPullingTime="420.098713ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:03:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:03:57.284302667 +0000 UTC m=+2220.293924326" watchObservedRunningTime="2026-07-20 16:03:57.360779013 +0000 UTC m=+2220.370400675"
Jul 20 16:04:00.715764 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:00.715725    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:04:03.276448 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:03.276416    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4a2d96-56d96c454-l7wfp"
Jul 20 16:04:04.346272 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:04.346217    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:04:08.316847 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:08.316811    2559 generic.go:356] "Generic (PLEG): container finished" podID="dfb761b9-e224-49a8-81e6-058a358422a2" containerID="86d82c4d58374a1e21940f18449dcddac74a34791bb3cab810b3a84caf551582" exitCode=0
Jul 20 16:04:08.317258 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:08.316886    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8" event={"ID":"dfb761b9-e224-49a8-81e6-058a358422a2","Type":"ContainerDied","Data":"86d82c4d58374a1e21940f18449dcddac74a34791bb3cab810b3a84caf551582"}
Jul 20 16:04:08.317334 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:08.317281    2559 scope.go:117] "RemoveContainer" containerID="86d82c4d58374a1e21940f18449dcddac74a34791bb3cab810b3a84caf551582"
Jul 20 16:04:08.318306 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:08.318262    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 16:04:08.318371 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:08.318354    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 16:04:08.318413 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:08.318402    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 16:04:09.323594 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:09.323559    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8" event={"ID":"dfb761b9-e224-49a8-81e6-058a358422a2","Type":"ContainerStarted","Data":"d1c2405df79c8e91ce1822c675e4d008ac7faa74b9167d82a8b54743a8eddb64"}
Jul 20 16:04:09.520654 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:09.520623    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"
Jul 20 16:04:09.521843 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:09.521822    2559 patch_prober.go:28] interesting pod/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 20 16:04:09.521901 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:09.521879    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8" podUID="dfb761b9-e224-49a8-81e6-058a358422a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 20 16:04:09.569245 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:09.569215    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:04:09.715585 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:09.715515    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:04:09.815664 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:09.815633    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:04:10.256162 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:10.256125    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"
Jul 20 16:04:10.521189 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:10.521160    2559 patch_prober.go:28] interesting pod/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 20 16:04:10.521578 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:10.521214    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8" podUID="dfb761b9-e224-49a8-81e6-058a358422a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 20 16:04:11.521333 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:11.521296    2559 patch_prober.go:28] interesting pod/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 20 16:04:11.521782 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:11.521354    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8" podUID="dfb761b9-e224-49a8-81e6-058a358422a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 20 16:04:12.521482 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:12.521443    2559 patch_prober.go:28] interesting pod/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 20 16:04:12.521863 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:12.521513    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8" podUID="dfb761b9-e224-49a8-81e6-058a358422a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 20 16:04:13.525470 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:13.525439    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"
Jul 20 16:04:13.526367 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:13.526349    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"
Jul 20 16:04:17.828181 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:17.828144    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:04:17.928574 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:17.928541    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:04:19.598702 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:19.598620    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:04:19.698892 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:19.698855    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:04:24.870968 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:24.870932    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq"]
Jul 20 16:04:24.871392 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:24.871129    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq" podUID="203ccc6c-38d8-463d-9302-57865132e8d4" containerName="maas-api" containerID="cri-o://9fd22a7674c4f7d27304e1520d99cf6709eb5c7c33c6ff5de7f3d49e5d4e2c2d" gracePeriod=30
Jul 20 16:04:24.914568 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:24.914536    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:04:25.121806 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:25.121750    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq"
Jul 20 16:04:25.295701 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:25.295672    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/203ccc6c-38d8-463d-9302-57865132e8d4-openshift-service-ca\") pod \"203ccc6c-38d8-463d-9302-57865132e8d4\" (UID: \"203ccc6c-38d8-463d-9302-57865132e8d4\") "
Jul 20 16:04:25.295861 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:25.295745    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qzkf4\" (UniqueName: \"kubernetes.io/projected/203ccc6c-38d8-463d-9302-57865132e8d4-kube-api-access-qzkf4\") pod \"203ccc6c-38d8-463d-9302-57865132e8d4\" (UID: \"203ccc6c-38d8-463d-9302-57865132e8d4\") "
Jul 20 16:04:25.295916 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:25.295896    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/203ccc6c-38d8-463d-9302-57865132e8d4-maas-api-tls\") pod \"203ccc6c-38d8-463d-9302-57865132e8d4\" (UID: \"203ccc6c-38d8-463d-9302-57865132e8d4\") "
Jul 20 16:04:25.296054 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:25.296031    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/203ccc6c-38d8-463d-9302-57865132e8d4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "203ccc6c-38d8-463d-9302-57865132e8d4" (UID: "203ccc6c-38d8-463d-9302-57865132e8d4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:04:25.296172 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:25.296151    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/203ccc6c-38d8-463d-9302-57865132e8d4-openshift-service-ca\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:04:25.297792 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:25.297770    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/203ccc6c-38d8-463d-9302-57865132e8d4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "203ccc6c-38d8-463d-9302-57865132e8d4" (UID: "203ccc6c-38d8-463d-9302-57865132e8d4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:04:25.297893 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:25.297798    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/203ccc6c-38d8-463d-9302-57865132e8d4-kube-api-access-qzkf4" (OuterVolumeSpecName: "kube-api-access-qzkf4") pod "203ccc6c-38d8-463d-9302-57865132e8d4" (UID: "203ccc6c-38d8-463d-9302-57865132e8d4"). InnerVolumeSpecName "kube-api-access-qzkf4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:04:25.397673 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:25.397589    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qzkf4\" (UniqueName: \"kubernetes.io/projected/203ccc6c-38d8-463d-9302-57865132e8d4-kube-api-access-qzkf4\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:04:25.397673 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:25.397620    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/203ccc6c-38d8-463d-9302-57865132e8d4-maas-api-tls\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:04:25.410519 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:25.410489    2559 generic.go:356] "Generic (PLEG): container finished" podID="203ccc6c-38d8-463d-9302-57865132e8d4" containerID="9fd22a7674c4f7d27304e1520d99cf6709eb5c7c33c6ff5de7f3d49e5d4e2c2d" exitCode=0
Jul 20 16:04:25.410639 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:25.410533    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq"
Jul 20 16:04:25.410639 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:25.410576    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq" event={"ID":"203ccc6c-38d8-463d-9302-57865132e8d4","Type":"ContainerDied","Data":"9fd22a7674c4f7d27304e1520d99cf6709eb5c7c33c6ff5de7f3d49e5d4e2c2d"}
Jul 20 16:04:25.410724 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:25.410650    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq" event={"ID":"203ccc6c-38d8-463d-9302-57865132e8d4","Type":"ContainerDied","Data":"9708117c5049affb53bfc9d7f00a6fbe3c9d3463065db79f068b2f85b99c6fea"}
Jul 20 16:04:25.410724 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:25.410672    2559 scope.go:117] "RemoveContainer" containerID="9fd22a7674c4f7d27304e1520d99cf6709eb5c7c33c6ff5de7f3d49e5d4e2c2d"
Jul 20 16:04:25.419805 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:25.419788    2559 scope.go:117] "RemoveContainer" containerID="9fd22a7674c4f7d27304e1520d99cf6709eb5c7c33c6ff5de7f3d49e5d4e2c2d"
Jul 20 16:04:25.420057 ip-10-0-130-19 kubenswrapper[2559]: E0720 16:04:25.420041    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fd22a7674c4f7d27304e1520d99cf6709eb5c7c33c6ff5de7f3d49e5d4e2c2d\": container with ID starting with 9fd22a7674c4f7d27304e1520d99cf6709eb5c7c33c6ff5de7f3d49e5d4e2c2d not found: ID does not exist" containerID="9fd22a7674c4f7d27304e1520d99cf6709eb5c7c33c6ff5de7f3d49e5d4e2c2d"
Jul 20 16:04:25.420108 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:25.420064    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fd22a7674c4f7d27304e1520d99cf6709eb5c7c33c6ff5de7f3d49e5d4e2c2d"} err="failed to get container status \"9fd22a7674c4f7d27304e1520d99cf6709eb5c7c33c6ff5de7f3d49e5d4e2c2d\": rpc error: code = NotFound desc = could not find container \"9fd22a7674c4f7d27304e1520d99cf6709eb5c7c33c6ff5de7f3d49e5d4e2c2d\": container with ID starting with 9fd22a7674c4f7d27304e1520d99cf6709eb5c7c33c6ff5de7f3d49e5d4e2c2d not found: ID does not exist"
Jul 20 16:04:25.436275 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:25.436251    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq"]
Jul 20 16:04:25.440174 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:25.440153    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5a8c68-966cd99f8-r4qkq"]
Jul 20 16:04:25.515042 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:25.515012    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="203ccc6c-38d8-463d-9302-57865132e8d4" path="/var/lib/kubelet/pods/203ccc6c-38d8-463d-9302-57865132e8d4/volumes"
Jul 20 16:04:28.984124 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:28.984086    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:04:29.364331 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:29.364276    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:04:35.626283 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:35.626251    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:04:35.739196 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:35.739165    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:04:35.893481 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:35.893402    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:04:36.068889 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:36.068859    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:04:36.821895 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:36.821862    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:04:36.922724 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:36.922698    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:04:52.038539 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:52.038502    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4a2d96-56d96c454-l7wfp"]
Jul 20 16:04:52.038946 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:52.038733    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4a2d96-56d96c454-l7wfp" podUID="ceb8e0fd-a233-492e-90af-52fbc4a7a9b4" containerName="maas-api" containerID="cri-o://ead3f714fb7c9afe482ab6daa9ce94bf81132a104f67661f1ce7d95f9f11de76" gracePeriod=30
Jul 20 16:04:52.115472 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:52.115438    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:04:52.285913 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:52.285889    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4a2d96-56d96c454-l7wfp"
Jul 20 16:04:52.429614 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:52.429538    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dd9qs\" (UniqueName: \"kubernetes.io/projected/ceb8e0fd-a233-492e-90af-52fbc4a7a9b4-kube-api-access-dd9qs\") pod \"ceb8e0fd-a233-492e-90af-52fbc4a7a9b4\" (UID: \"ceb8e0fd-a233-492e-90af-52fbc4a7a9b4\") "
Jul 20 16:04:52.429614 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:52.429583    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceb8e0fd-a233-492e-90af-52fbc4a7a9b4-openshift-service-ca\") pod \"ceb8e0fd-a233-492e-90af-52fbc4a7a9b4\" (UID: \"ceb8e0fd-a233-492e-90af-52fbc4a7a9b4\") "
Jul 20 16:04:52.429614 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:52.429615    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ceb8e0fd-a233-492e-90af-52fbc4a7a9b4-maas-api-tls\") pod \"ceb8e0fd-a233-492e-90af-52fbc4a7a9b4\" (UID: \"ceb8e0fd-a233-492e-90af-52fbc4a7a9b4\") "
Jul 20 16:04:52.429998 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:52.429967    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ceb8e0fd-a233-492e-90af-52fbc4a7a9b4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ceb8e0fd-a233-492e-90af-52fbc4a7a9b4" (UID: "ceb8e0fd-a233-492e-90af-52fbc4a7a9b4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:04:52.431767 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:52.431737    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ceb8e0fd-a233-492e-90af-52fbc4a7a9b4-kube-api-access-dd9qs" (OuterVolumeSpecName: "kube-api-access-dd9qs") pod "ceb8e0fd-a233-492e-90af-52fbc4a7a9b4" (UID: "ceb8e0fd-a233-492e-90af-52fbc4a7a9b4"). InnerVolumeSpecName "kube-api-access-dd9qs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:04:52.431879 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:52.431784    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ceb8e0fd-a233-492e-90af-52fbc4a7a9b4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ceb8e0fd-a233-492e-90af-52fbc4a7a9b4" (UID: "ceb8e0fd-a233-492e-90af-52fbc4a7a9b4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:04:52.530829 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:52.530788    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dd9qs\" (UniqueName: \"kubernetes.io/projected/ceb8e0fd-a233-492e-90af-52fbc4a7a9b4-kube-api-access-dd9qs\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:04:52.530829 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:52.530826    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceb8e0fd-a233-492e-90af-52fbc4a7a9b4-openshift-service-ca\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:04:52.531028 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:52.530840    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ceb8e0fd-a233-492e-90af-52fbc4a7a9b4-maas-api-tls\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:04:52.537754 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:52.537725    2559 generic.go:356] "Generic (PLEG): container finished" podID="ceb8e0fd-a233-492e-90af-52fbc4a7a9b4" containerID="ead3f714fb7c9afe482ab6daa9ce94bf81132a104f67661f1ce7d95f9f11de76" exitCode=0
Jul 20 16:04:52.537913 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:52.537781    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4a2d96-56d96c454-l7wfp"
Jul 20 16:04:52.537913 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:52.537792    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4a2d96-56d96c454-l7wfp" event={"ID":"ceb8e0fd-a233-492e-90af-52fbc4a7a9b4","Type":"ContainerDied","Data":"ead3f714fb7c9afe482ab6daa9ce94bf81132a104f67661f1ce7d95f9f11de76"}
Jul 20 16:04:52.537913 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:52.537844    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4a2d96-56d96c454-l7wfp" event={"ID":"ceb8e0fd-a233-492e-90af-52fbc4a7a9b4","Type":"ContainerDied","Data":"b4eeb46b5390bfba4498f9d20f90aa2ac7921f3d5e474d4172136f7e78d0acf6"}
Jul 20 16:04:52.537913 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:52.537872    2559 scope.go:117] "RemoveContainer" containerID="ead3f714fb7c9afe482ab6daa9ce94bf81132a104f67661f1ce7d95f9f11de76"
Jul 20 16:04:52.546393 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:52.546372    2559 scope.go:117] "RemoveContainer" containerID="ead3f714fb7c9afe482ab6daa9ce94bf81132a104f67661f1ce7d95f9f11de76"
Jul 20 16:04:52.546652 ip-10-0-130-19 kubenswrapper[2559]: E0720 16:04:52.546628    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ead3f714fb7c9afe482ab6daa9ce94bf81132a104f67661f1ce7d95f9f11de76\": container with ID starting with ead3f714fb7c9afe482ab6daa9ce94bf81132a104f67661f1ce7d95f9f11de76 not found: ID does not exist" containerID="ead3f714fb7c9afe482ab6daa9ce94bf81132a104f67661f1ce7d95f9f11de76"
Jul 20 16:04:52.546700 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:52.546661    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ead3f714fb7c9afe482ab6daa9ce94bf81132a104f67661f1ce7d95f9f11de76"} err="failed to get container status \"ead3f714fb7c9afe482ab6daa9ce94bf81132a104f67661f1ce7d95f9f11de76\": rpc error: code = NotFound desc = could not find container \"ead3f714fb7c9afe482ab6daa9ce94bf81132a104f67661f1ce7d95f9f11de76\": container with ID starting with ead3f714fb7c9afe482ab6daa9ce94bf81132a104f67661f1ce7d95f9f11de76 not found: ID does not exist"
Jul 20 16:04:52.566779 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:52.566752    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4a2d96-56d96c454-l7wfp"]
Jul 20 16:04:52.576583 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:52.576559    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4a2d96-56d96c454-l7wfp"]
Jul 20 16:04:53.514364 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:53.514335    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ceb8e0fd-a233-492e-90af-52fbc4a7a9b4" path="/var/lib/kubelet/pods/ceb8e0fd-a233-492e-90af-52fbc4a7a9b4/volumes"
Jul 20 16:04:56.349053 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:56.349019    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:04:56.845590 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:04:56.845559    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:03.602402 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:03.602371    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:03.720845 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:03.720816    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:03.917142 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:03.917068    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"]
Jul 20 16:05:03.917337 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:03.917316    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8" podUID="dfb761b9-e224-49a8-81e6-058a358422a2" containerName="istio-proxy" containerID="cri-o://d1c2405df79c8e91ce1822c675e4d008ac7faa74b9167d82a8b54743a8eddb64" gracePeriod=30
Jul 20 16:05:03.940320 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:03.940275    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:04.172902 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:04.172832    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:04.494010 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:04.493945    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:04.857827 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:04.857794    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:05.124134 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:05.124054    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:07.528325 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:07.528273    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:07.665922 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:07.665889    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8791e0-79f5f6b586-nw7np"]
Jul 20 16:05:07.666246 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:07.666233    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="203ccc6c-38d8-463d-9302-57865132e8d4" containerName="maas-api"
Jul 20 16:05:07.666246 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:07.666247    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="203ccc6c-38d8-463d-9302-57865132e8d4" containerName="maas-api"
Jul 20 16:05:07.666357 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:07.666260    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ceb8e0fd-a233-492e-90af-52fbc4a7a9b4" containerName="maas-api"
Jul 20 16:05:07.666357 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:07.666266    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ceb8e0fd-a233-492e-90af-52fbc4a7a9b4" containerName="maas-api"
Jul 20 16:05:07.666357 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:07.666331    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="203ccc6c-38d8-463d-9302-57865132e8d4" containerName="maas-api"
Jul 20 16:05:07.666357 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:07.666339    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ceb8e0fd-a233-492e-90af-52fbc4a7a9b4" containerName="maas-api"
Jul 20 16:05:07.669187 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:07.669167    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8791e0-79f5f6b586-nw7np"
Jul 20 16:05:07.672171 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:07.672150    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-8791e0-serving-cert\""
Jul 20 16:05:07.672249 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:07.672162    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-96kqc\""
Jul 20 16:05:07.693081 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:07.693055    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:07.695838 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:07.695818    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8791e0-79f5f6b586-nw7np"]
Jul 20 16:05:07.770732 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:07.770706    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97e35eb7-883a-4695-8a50-4325aa14f9b1-maas-api-tls\") pod \"maas-api-e2e-shared-a-8791e0-79f5f6b586-nw7np\" (UID: \"97e35eb7-883a-4695-8a50-4325aa14f9b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8791e0-79f5f6b586-nw7np"
Jul 20 16:05:07.770879 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:07.770745    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8qcr\" (UniqueName: \"kubernetes.io/projected/97e35eb7-883a-4695-8a50-4325aa14f9b1-kube-api-access-m8qcr\") pod \"maas-api-e2e-shared-a-8791e0-79f5f6b586-nw7np\" (UID: \"97e35eb7-883a-4695-8a50-4325aa14f9b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8791e0-79f5f6b586-nw7np"
Jul 20 16:05:07.770879 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:07.770773    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97e35eb7-883a-4695-8a50-4325aa14f9b1-openshift-service-ca\") pod \"maas-api-e2e-shared-a-8791e0-79f5f6b586-nw7np\" (UID: \"97e35eb7-883a-4695-8a50-4325aa14f9b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8791e0-79f5f6b586-nw7np"
Jul 20 16:05:07.872149 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:07.872072    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8qcr\" (UniqueName: \"kubernetes.io/projected/97e35eb7-883a-4695-8a50-4325aa14f9b1-kube-api-access-m8qcr\") pod \"maas-api-e2e-shared-a-8791e0-79f5f6b586-nw7np\" (UID: \"97e35eb7-883a-4695-8a50-4325aa14f9b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8791e0-79f5f6b586-nw7np"
Jul 20 16:05:07.872149 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:07.872114    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97e35eb7-883a-4695-8a50-4325aa14f9b1-openshift-service-ca\") pod \"maas-api-e2e-shared-a-8791e0-79f5f6b586-nw7np\" (UID: \"97e35eb7-883a-4695-8a50-4325aa14f9b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8791e0-79f5f6b586-nw7np"
Jul 20 16:05:07.872437 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:07.872216    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97e35eb7-883a-4695-8a50-4325aa14f9b1-maas-api-tls\") pod \"maas-api-e2e-shared-a-8791e0-79f5f6b586-nw7np\" (UID: \"97e35eb7-883a-4695-8a50-4325aa14f9b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8791e0-79f5f6b586-nw7np"
Jul 20 16:05:07.872437 ip-10-0-130-19 kubenswrapper[2559]: E0720 16:05:07.872361    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-8791e0-serving-cert: secret "maas-api-e2e-shared-a-8791e0-serving-cert" not found
Jul 20 16:05:07.872437 ip-10-0-130-19 kubenswrapper[2559]: E0720 16:05:07.872427    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97e35eb7-883a-4695-8a50-4325aa14f9b1-maas-api-tls podName:97e35eb7-883a-4695-8a50-4325aa14f9b1 nodeName:}" failed. No retries permitted until 2026-07-20 16:05:08.372408186 +0000 UTC m=+2291.382029830 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/97e35eb7-883a-4695-8a50-4325aa14f9b1-maas-api-tls") pod "maas-api-e2e-shared-a-8791e0-79f5f6b586-nw7np" (UID: "97e35eb7-883a-4695-8a50-4325aa14f9b1") : secret "maas-api-e2e-shared-a-8791e0-serving-cert" not found
Jul 20 16:05:07.872785 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:07.872765    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97e35eb7-883a-4695-8a50-4325aa14f9b1-openshift-service-ca\") pod \"maas-api-e2e-shared-a-8791e0-79f5f6b586-nw7np\" (UID: \"97e35eb7-883a-4695-8a50-4325aa14f9b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8791e0-79f5f6b586-nw7np"
Jul 20 16:05:07.914854 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:07.914822    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8qcr\" (UniqueName: \"kubernetes.io/projected/97e35eb7-883a-4695-8a50-4325aa14f9b1-kube-api-access-m8qcr\") pod \"maas-api-e2e-shared-a-8791e0-79f5f6b586-nw7np\" (UID: \"97e35eb7-883a-4695-8a50-4325aa14f9b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8791e0-79f5f6b586-nw7np"
Jul 20 16:05:07.929002 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:07.928976    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:08.154496 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:08.154407    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:08.377178 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:08.377150    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97e35eb7-883a-4695-8a50-4325aa14f9b1-maas-api-tls\") pod \"maas-api-e2e-shared-a-8791e0-79f5f6b586-nw7np\" (UID: \"97e35eb7-883a-4695-8a50-4325aa14f9b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8791e0-79f5f6b586-nw7np"
Jul 20 16:05:08.379510 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:08.379491    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97e35eb7-883a-4695-8a50-4325aa14f9b1-maas-api-tls\") pod \"maas-api-e2e-shared-a-8791e0-79f5f6b586-nw7np\" (UID: \"97e35eb7-883a-4695-8a50-4325aa14f9b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8791e0-79f5f6b586-nw7np"
Jul 20 16:05:08.578668 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:08.578639    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8791e0-79f5f6b586-nw7np"
Jul 20 16:05:08.773274 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:08.773237    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:08.885845 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:08.885778    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:08.957847 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:08.957814    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8791e0-79f5f6b586-nw7np"]
Jul 20 16:05:08.993190 ip-10-0-130-19 kubenswrapper[2559]: W0720 16:05:08.993166    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97e35eb7_883a_4695_8a50_4325aa14f9b1.slice/crio-8d3f8469691d813b761d2162995e1c367378c4553dd2c96c8e0cbf1770234bb9 WatchSource:0}: Error finding container 8d3f8469691d813b761d2162995e1c367378c4553dd2c96c8e0cbf1770234bb9: Status 404 returned error can't find the container with id 8d3f8469691d813b761d2162995e1c367378c4553dd2c96c8e0cbf1770234bb9
Jul 20 16:05:09.159655 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.159633    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"
Jul 20 16:05:09.286138 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.286107    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfb761b9-e224-49a8-81e6-058a358422a2-workload-socket\") pod \"dfb761b9-e224-49a8-81e6-058a358422a2\" (UID: \"dfb761b9-e224-49a8-81e6-058a358422a2\") "
Jul 20 16:05:09.286138 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.286141    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dfb761b9-e224-49a8-81e6-058a358422a2-istio-data\") pod \"dfb761b9-e224-49a8-81e6-058a358422a2\" (UID: \"dfb761b9-e224-49a8-81e6-058a358422a2\") "
Jul 20 16:05:09.286392 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.286165    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dfb761b9-e224-49a8-81e6-058a358422a2-istiod-ca-cert\") pod \"dfb761b9-e224-49a8-81e6-058a358422a2\" (UID: \"dfb761b9-e224-49a8-81e6-058a358422a2\") "
Jul 20 16:05:09.286392 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.286183    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dfb761b9-e224-49a8-81e6-058a358422a2-istio-token\") pod \"dfb761b9-e224-49a8-81e6-058a358422a2\" (UID: \"dfb761b9-e224-49a8-81e6-058a358422a2\") "
Jul 20 16:05:09.286392 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.286207    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dfb761b9-e224-49a8-81e6-058a358422a2-istio-podinfo\") pod \"dfb761b9-e224-49a8-81e6-058a358422a2\" (UID: \"dfb761b9-e224-49a8-81e6-058a358422a2\") "
Jul 20 16:05:09.286392 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.286226    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dfb761b9-e224-49a8-81e6-058a358422a2-workload-certs\") pod \"dfb761b9-e224-49a8-81e6-058a358422a2\" (UID: \"dfb761b9-e224-49a8-81e6-058a358422a2\") "
Jul 20 16:05:09.286392 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.286271    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfb761b9-e224-49a8-81e6-058a358422a2-credential-socket\") pod \"dfb761b9-e224-49a8-81e6-058a358422a2\" (UID: \"dfb761b9-e224-49a8-81e6-058a358422a2\") "
Jul 20 16:05:09.286392 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.286361    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-np2pl\" (UniqueName: \"kubernetes.io/projected/dfb761b9-e224-49a8-81e6-058a358422a2-kube-api-access-np2pl\") pod \"dfb761b9-e224-49a8-81e6-058a358422a2\" (UID: \"dfb761b9-e224-49a8-81e6-058a358422a2\") "
Jul 20 16:05:09.286392 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.286370    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfb761b9-e224-49a8-81e6-058a358422a2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dfb761b9-e224-49a8-81e6-058a358422a2" (UID: "dfb761b9-e224-49a8-81e6-058a358422a2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:05:09.286392 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.286384    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dfb761b9-e224-49a8-81e6-058a358422a2-istio-envoy\") pod \"dfb761b9-e224-49a8-81e6-058a358422a2\" (UID: \"dfb761b9-e224-49a8-81e6-058a358422a2\") "
Jul 20 16:05:09.286806 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.286561    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfb761b9-e224-49a8-81e6-058a358422a2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dfb761b9-e224-49a8-81e6-058a358422a2" (UID: "dfb761b9-e224-49a8-81e6-058a358422a2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:05:09.286806 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.286578    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfb761b9-e224-49a8-81e6-058a358422a2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dfb761b9-e224-49a8-81e6-058a358422a2" (UID: "dfb761b9-e224-49a8-81e6-058a358422a2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:05:09.286806 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.286663    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dfb761b9-e224-49a8-81e6-058a358422a2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dfb761b9-e224-49a8-81e6-058a358422a2" (UID: "dfb761b9-e224-49a8-81e6-058a358422a2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:05:09.286806 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.286679    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfb761b9-e224-49a8-81e6-058a358422a2-workload-socket\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:05:09.286806 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.286696    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dfb761b9-e224-49a8-81e6-058a358422a2-workload-certs\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:05:09.286806 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.286711    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfb761b9-e224-49a8-81e6-058a358422a2-credential-socket\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:05:09.287263 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.287058    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfb761b9-e224-49a8-81e6-058a358422a2-istio-data" (OuterVolumeSpecName: "istio-data") pod "dfb761b9-e224-49a8-81e6-058a358422a2" (UID: "dfb761b9-e224-49a8-81e6-058a358422a2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:05:09.288584 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.288564    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dfb761b9-e224-49a8-81e6-058a358422a2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dfb761b9-e224-49a8-81e6-058a358422a2" (UID: "dfb761b9-e224-49a8-81e6-058a358422a2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 20 16:05:09.288584 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.288570    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfb761b9-e224-49a8-81e6-058a358422a2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dfb761b9-e224-49a8-81e6-058a358422a2" (UID: "dfb761b9-e224-49a8-81e6-058a358422a2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:05:09.288797 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.288754    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfb761b9-e224-49a8-81e6-058a358422a2-istio-token" (OuterVolumeSpecName: "istio-token") pod "dfb761b9-e224-49a8-81e6-058a358422a2" (UID: "dfb761b9-e224-49a8-81e6-058a358422a2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:05:09.288955 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.288933    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfb761b9-e224-49a8-81e6-058a358422a2-kube-api-access-np2pl" (OuterVolumeSpecName: "kube-api-access-np2pl") pod "dfb761b9-e224-49a8-81e6-058a358422a2" (UID: "dfb761b9-e224-49a8-81e6-058a358422a2"). InnerVolumeSpecName "kube-api-access-np2pl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:05:09.387225 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.387198    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dfb761b9-e224-49a8-81e6-058a358422a2-istio-data\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:05:09.387225 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.387225    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dfb761b9-e224-49a8-81e6-058a358422a2-istiod-ca-cert\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:05:09.387384 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.387235    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dfb761b9-e224-49a8-81e6-058a358422a2-istio-token\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:05:09.387384 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.387243    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dfb761b9-e224-49a8-81e6-058a358422a2-istio-podinfo\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:05:09.387384 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.387251    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-np2pl\" (UniqueName: \"kubernetes.io/projected/dfb761b9-e224-49a8-81e6-058a358422a2-kube-api-access-np2pl\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:05:09.387384 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.387259    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dfb761b9-e224-49a8-81e6-058a358422a2-istio-envoy\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:05:09.612965 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.612941    2559 generic.go:356] "Generic (PLEG): container finished" podID="dfb761b9-e224-49a8-81e6-058a358422a2" containerID="d1c2405df79c8e91ce1822c675e4d008ac7faa74b9167d82a8b54743a8eddb64" exitCode=0
Jul 20 16:05:09.613276 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.612995    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"
Jul 20 16:05:09.613276 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.613023    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8" event={"ID":"dfb761b9-e224-49a8-81e6-058a358422a2","Type":"ContainerDied","Data":"d1c2405df79c8e91ce1822c675e4d008ac7faa74b9167d82a8b54743a8eddb64"}
Jul 20 16:05:09.613276 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.613086    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8" event={"ID":"dfb761b9-e224-49a8-81e6-058a358422a2","Type":"ContainerDied","Data":"16159a1c6a00540d0f3759fc36ef9a3dc281acf3811de7252eb44159b88e7af4"}
Jul 20 16:05:09.613276 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.613121    2559 scope.go:117] "RemoveContainer" containerID="d1c2405df79c8e91ce1822c675e4d008ac7faa74b9167d82a8b54743a8eddb64"
Jul 20 16:05:09.614691 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.614670    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8791e0-79f5f6b586-nw7np" event={"ID":"97e35eb7-883a-4695-8a50-4325aa14f9b1","Type":"ContainerStarted","Data":"48b1f0a7ff8126ac3851ecad99ad0eb7afdebd6685654e38519290d79a8c3a9b"}
Jul 20 16:05:09.614788 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.614715    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8791e0-79f5f6b586-nw7np" event={"ID":"97e35eb7-883a-4695-8a50-4325aa14f9b1","Type":"ContainerStarted","Data":"8d3f8469691d813b761d2162995e1c367378c4553dd2c96c8e0cbf1770234bb9"}
Jul 20 16:05:09.614788 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.614744    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8791e0-79f5f6b586-nw7np"
Jul 20 16:05:09.620838 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.620822    2559 scope.go:117] "RemoveContainer" containerID="86d82c4d58374a1e21940f18449dcddac74a34791bb3cab810b3a84caf551582"
Jul 20 16:05:09.628587 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.628565    2559 scope.go:117] "RemoveContainer" containerID="d1c2405df79c8e91ce1822c675e4d008ac7faa74b9167d82a8b54743a8eddb64"
Jul 20 16:05:09.628840 ip-10-0-130-19 kubenswrapper[2559]: E0720 16:05:09.628819    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1c2405df79c8e91ce1822c675e4d008ac7faa74b9167d82a8b54743a8eddb64\": container with ID starting with d1c2405df79c8e91ce1822c675e4d008ac7faa74b9167d82a8b54743a8eddb64 not found: ID does not exist" containerID="d1c2405df79c8e91ce1822c675e4d008ac7faa74b9167d82a8b54743a8eddb64"
Jul 20 16:05:09.628924 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.628851    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1c2405df79c8e91ce1822c675e4d008ac7faa74b9167d82a8b54743a8eddb64"} err="failed to get container status \"d1c2405df79c8e91ce1822c675e4d008ac7faa74b9167d82a8b54743a8eddb64\": rpc error: code = NotFound desc = could not find container \"d1c2405df79c8e91ce1822c675e4d008ac7faa74b9167d82a8b54743a8eddb64\": container with ID starting with d1c2405df79c8e91ce1822c675e4d008ac7faa74b9167d82a8b54743a8eddb64 not found: ID does not exist"
Jul 20 16:05:09.628924 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.628874    2559 scope.go:117] "RemoveContainer" containerID="86d82c4d58374a1e21940f18449dcddac74a34791bb3cab810b3a84caf551582"
Jul 20 16:05:09.629198 ip-10-0-130-19 kubenswrapper[2559]: E0720 16:05:09.629168    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86d82c4d58374a1e21940f18449dcddac74a34791bb3cab810b3a84caf551582\": container with ID starting with 86d82c4d58374a1e21940f18449dcddac74a34791bb3cab810b3a84caf551582 not found: ID does not exist" containerID="86d82c4d58374a1e21940f18449dcddac74a34791bb3cab810b3a84caf551582"
Jul 20 16:05:09.629249 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.629214    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86d82c4d58374a1e21940f18449dcddac74a34791bb3cab810b3a84caf551582"} err="failed to get container status \"86d82c4d58374a1e21940f18449dcddac74a34791bb3cab810b3a84caf551582\": rpc error: code = NotFound desc = could not find container \"86d82c4d58374a1e21940f18449dcddac74a34791bb3cab810b3a84caf551582\": container with ID starting with 86d82c4d58374a1e21940f18449dcddac74a34791bb3cab810b3a84caf551582 not found: ID does not exist"
Jul 20 16:05:09.647600 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.647552    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8791e0-79f5f6b586-nw7np" podStartSLOduration=2.148969915 podStartE2EDuration="2.647533048s" totalImagesPullingTime="498.563133ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:05:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:05:09.643744196 +0000 UTC m=+2292.653365859" watchObservedRunningTime="2026-07-20 16:05:09.647533048 +0000 UTC m=+2292.657154711"
Jul 20 16:05:09.672272 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.672248    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"]
Jul 20 16:05:09.680887 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:09.680865    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-4a2d96-openshift-default-5b7c56f55c-rwvf8"]
Jul 20 16:05:11.514648 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:11.514616    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfb761b9-e224-49a8-81e6-058a358422a2" path="/var/lib/kubelet/pods/dfb761b9-e224-49a8-81e6-058a358422a2/volumes"
Jul 20 16:05:12.005428 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.005351    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:12.345375 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.345344    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:12.656455 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.656357    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"]
Jul 20 16:05:12.656892 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.656783    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfb761b9-e224-49a8-81e6-058a358422a2" containerName="istio-proxy"
Jul 20 16:05:12.656892 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.656800    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfb761b9-e224-49a8-81e6-058a358422a2" containerName="istio-proxy"
Jul 20 16:05:12.656979 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.656897    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfb761b9-e224-49a8-81e6-058a358422a2" containerName="istio-proxy"
Jul 20 16:05:12.656979 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.656910    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfb761b9-e224-49a8-81e6-058a358422a2" containerName="istio-proxy"
Jul 20 16:05:12.660213 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.660195    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"
Jul 20 16:05:12.663772 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.663758    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-a667cd-openshift-default-dockercfg-q7z2x\""
Jul 20 16:05:12.663861 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.663758    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 20 16:05:12.683822 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.683800    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"]
Jul 20 16:05:12.697379 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.697356    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:12.817071 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.817028    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-workload-certs\") pod \"e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d\" (UID: \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\") " pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"
Jul 20 16:05:12.817071 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.817075    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-istio-envoy\") pod \"e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d\" (UID: \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\") " pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"
Jul 20 16:05:12.817071 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.817095    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-istio-token\") pod \"e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d\" (UID: \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\") " pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"
Jul 20 16:05:12.817350 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.817178    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-istiod-ca-cert\") pod \"e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d\" (UID: \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\") " pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"
Jul 20 16:05:12.817350 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.817210    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-credential-socket\") pod \"e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d\" (UID: \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\") " pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"
Jul 20 16:05:12.817350 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.817235    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-workload-socket\") pod \"e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d\" (UID: \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\") " pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"
Jul 20 16:05:12.817350 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.817267    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-istio-podinfo\") pod \"e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d\" (UID: \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\") " pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"
Jul 20 16:05:12.817483 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.817369    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ft56\" (UniqueName: \"kubernetes.io/projected/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-kube-api-access-9ft56\") pod \"e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d\" (UID: \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\") " pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"
Jul 20 16:05:12.817483 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.817393    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-istio-data\") pod \"e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d\" (UID: \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\") " pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"
Jul 20 16:05:12.891110 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.891071    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:12.918333 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.918235    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ft56\" (UniqueName: \"kubernetes.io/projected/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-kube-api-access-9ft56\") pod \"e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d\" (UID: \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\") " pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"
Jul 20 16:05:12.918333 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.918279    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-istio-data\") pod \"e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d\" (UID: \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\") " pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"
Jul 20 16:05:12.918553 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.918363    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-workload-certs\") pod \"e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d\" (UID: \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\") " pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"
Jul 20 16:05:12.918553 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.918391    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-istio-envoy\") pod \"e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d\" (UID: \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\") " pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"
Jul 20 16:05:12.918553 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.918413    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-istio-token\") pod \"e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d\" (UID: \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\") " pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"
Jul 20 16:05:12.918553 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.918461    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-istiod-ca-cert\") pod \"e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d\" (UID: \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\") " pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"
Jul 20 16:05:12.918553 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.918498    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-credential-socket\") pod \"e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d\" (UID: \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\") " pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"
Jul 20 16:05:12.918553 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.918528    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-workload-socket\") pod \"e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d\" (UID: \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\") " pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"
Jul 20 16:05:12.918851 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.918557    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-istio-podinfo\") pod \"e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d\" (UID: \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\") " pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"
Jul 20 16:05:12.918851 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.918774    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-workload-certs\") pod \"e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d\" (UID: \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\") " pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"
Jul 20 16:05:12.918851 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.918829    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-istio-data\") pod \"e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d\" (UID: \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\") " pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"
Jul 20 16:05:12.919014 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.918912    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-credential-socket\") pod \"e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d\" (UID: \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\") " pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"
Jul 20 16:05:12.919014 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.918991    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-workload-socket\") pod \"e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d\" (UID: \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\") " pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"
Jul 20 16:05:12.919238 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.919215    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-istiod-ca-cert\") pod \"e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d\" (UID: \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\") " pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"
Jul 20 16:05:12.920721 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.920701    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-istio-envoy\") pod \"e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d\" (UID: \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\") " pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"
Jul 20 16:05:12.921035 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.921016    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-istio-podinfo\") pod \"e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d\" (UID: \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\") " pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"
Jul 20 16:05:12.927732 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.927703    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ft56\" (UniqueName: \"kubernetes.io/projected/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-kube-api-access-9ft56\") pod \"e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d\" (UID: \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\") " pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"
Jul 20 16:05:12.928541 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.928507    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-istio-token\") pod \"e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d\" (UID: \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\") " pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"
Jul 20 16:05:12.970118 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:12.970091    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"
Jul 20 16:05:13.108307 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:13.108262    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"]
Jul 20 16:05:13.109902 ip-10-0-130-19 kubenswrapper[2559]: W0720 16:05:13.109873    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07d5a0a6_90f0_4301_86d2_4bc2326a35f6.slice/crio-734062c1a71bac85ee97ed753d199f3bbde786b53015fa8d7af65501b9b58a38 WatchSource:0}: Error finding container 734062c1a71bac85ee97ed753d199f3bbde786b53015fa8d7af65501b9b58a38: Status 404 returned error can't find the container with id 734062c1a71bac85ee97ed753d199f3bbde786b53015fa8d7af65501b9b58a38
Jul 20 16:05:13.112000 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:13.111973    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 16:05:13.112075 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:13.112035    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 16:05:13.112075 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:13.112065    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 16:05:13.635778 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:13.635746    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d" event={"ID":"07d5a0a6-90f0-4301-86d2-4bc2326a35f6","Type":"ContainerStarted","Data":"1a07221b8432adac8496971cb11392f56d2d95a1b8d3ab43cb312e923c328ecc"}
Jul 20 16:05:13.635949 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:13.635798    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d" event={"ID":"07d5a0a6-90f0-4301-86d2-4bc2326a35f6","Type":"ContainerStarted","Data":"734062c1a71bac85ee97ed753d199f3bbde786b53015fa8d7af65501b9b58a38"}
Jul 20 16:05:13.662814 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:13.662768    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d" podStartSLOduration=1.662751643 podStartE2EDuration="1.662751643s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:05:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:05:13.660246518 +0000 UTC m=+2296.669868181" watchObservedRunningTime="2026-07-20 16:05:13.662751643 +0000 UTC m=+2296.672373306"
Jul 20 16:05:13.970713 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:13.970623    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"
Jul 20 16:05:14.976269 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:14.976238    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"
Jul 20 16:05:15.625925 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:15.625897    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8791e0-79f5f6b586-nw7np"
Jul 20 16:05:15.643947 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:15.643910    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"
Jul 20 16:05:15.644959 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:15.644941    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"
Jul 20 16:05:15.697051 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:15.697012    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:15.804432 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:15.804401    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:15.814776 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:15.814750    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-a667cd-7fd8b4b696-t2zrd"]
Jul 20 16:05:15.815124 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:15.815112    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfb761b9-e224-49a8-81e6-058a358422a2" containerName="istio-proxy"
Jul 20 16:05:15.815181 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:15.815125    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfb761b9-e224-49a8-81e6-058a358422a2" containerName="istio-proxy"
Jul 20 16:05:15.818078 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:15.818061    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a667cd-7fd8b4b696-t2zrd"
Jul 20 16:05:15.821440 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:15.821412    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-a667cd-serving-cert\""
Jul 20 16:05:15.830513 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:15.830363    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-a667cd-7fd8b4b696-t2zrd"]
Jul 20 16:05:15.898383 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:15.898255    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:15.942705 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:15.942681    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zq7hr\" (UniqueName: \"kubernetes.io/projected/b084a8cb-2f9b-4e2d-8e48-a5e84981c169-kube-api-access-zq7hr\") pod \"maas-api-e2e-shared-b-a667cd-7fd8b4b696-t2zrd\" (UID: \"b084a8cb-2f9b-4e2d-8e48-a5e84981c169\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a667cd-7fd8b4b696-t2zrd"
Jul 20 16:05:15.942839 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:15.942726    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b084a8cb-2f9b-4e2d-8e48-a5e84981c169-openshift-service-ca\") pod \"maas-api-e2e-shared-b-a667cd-7fd8b4b696-t2zrd\" (UID: \"b084a8cb-2f9b-4e2d-8e48-a5e84981c169\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a667cd-7fd8b4b696-t2zrd"
Jul 20 16:05:15.942839 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:15.942810    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b084a8cb-2f9b-4e2d-8e48-a5e84981c169-maas-api-tls\") pod \"maas-api-e2e-shared-b-a667cd-7fd8b4b696-t2zrd\" (UID: \"b084a8cb-2f9b-4e2d-8e48-a5e84981c169\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a667cd-7fd8b4b696-t2zrd"
Jul 20 16:05:16.043589 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:16.043560    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b084a8cb-2f9b-4e2d-8e48-a5e84981c169-openshift-service-ca\") pod \"maas-api-e2e-shared-b-a667cd-7fd8b4b696-t2zrd\" (UID: \"b084a8cb-2f9b-4e2d-8e48-a5e84981c169\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a667cd-7fd8b4b696-t2zrd"
Jul 20 16:05:16.043941 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:16.043607    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b084a8cb-2f9b-4e2d-8e48-a5e84981c169-maas-api-tls\") pod \"maas-api-e2e-shared-b-a667cd-7fd8b4b696-t2zrd\" (UID: \"b084a8cb-2f9b-4e2d-8e48-a5e84981c169\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a667cd-7fd8b4b696-t2zrd"
Jul 20 16:05:16.043941 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:16.043672    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq7hr\" (UniqueName: \"kubernetes.io/projected/b084a8cb-2f9b-4e2d-8e48-a5e84981c169-kube-api-access-zq7hr\") pod \"maas-api-e2e-shared-b-a667cd-7fd8b4b696-t2zrd\" (UID: \"b084a8cb-2f9b-4e2d-8e48-a5e84981c169\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a667cd-7fd8b4b696-t2zrd"
Jul 20 16:05:16.044303 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:16.044270    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b084a8cb-2f9b-4e2d-8e48-a5e84981c169-openshift-service-ca\") pod \"maas-api-e2e-shared-b-a667cd-7fd8b4b696-t2zrd\" (UID: \"b084a8cb-2f9b-4e2d-8e48-a5e84981c169\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a667cd-7fd8b4b696-t2zrd"
Jul 20 16:05:16.045956 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:16.045930    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b084a8cb-2f9b-4e2d-8e48-a5e84981c169-maas-api-tls\") pod \"maas-api-e2e-shared-b-a667cd-7fd8b4b696-t2zrd\" (UID: \"b084a8cb-2f9b-4e2d-8e48-a5e84981c169\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a667cd-7fd8b4b696-t2zrd"
Jul 20 16:05:16.077220 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:16.077191    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq7hr\" (UniqueName: \"kubernetes.io/projected/b084a8cb-2f9b-4e2d-8e48-a5e84981c169-kube-api-access-zq7hr\") pod \"maas-api-e2e-shared-b-a667cd-7fd8b4b696-t2zrd\" (UID: \"b084a8cb-2f9b-4e2d-8e48-a5e84981c169\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a667cd-7fd8b4b696-t2zrd"
Jul 20 16:05:16.084375 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:16.084353    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:16.131956 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:16.131932    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a667cd-7fd8b4b696-t2zrd"
Jul 20 16:05:16.280943 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:16.280918    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-a667cd-7fd8b4b696-t2zrd"]
Jul 20 16:05:16.282797 ip-10-0-130-19 kubenswrapper[2559]: W0720 16:05:16.282770    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb084a8cb_2f9b_4e2d_8e48_a5e84981c169.slice/crio-f613ff9cce95b9438d7087841678f3c7ffb6ee16fa6bb2ac35628c2fe973620c WatchSource:0}: Error finding container f613ff9cce95b9438d7087841678f3c7ffb6ee16fa6bb2ac35628c2fe973620c: Status 404 returned error can't find the container with id f613ff9cce95b9438d7087841678f3c7ffb6ee16fa6bb2ac35628c2fe973620c
Jul 20 16:05:16.586557 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:16.586525    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:16.650640 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:16.650610    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a667cd-7fd8b4b696-t2zrd" event={"ID":"b084a8cb-2f9b-4e2d-8e48-a5e84981c169","Type":"ContainerStarted","Data":"f613ff9cce95b9438d7087841678f3c7ffb6ee16fa6bb2ac35628c2fe973620c"}
Jul 20 16:05:16.692345 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:16.692316    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:17.656056 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:17.656018    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a667cd-7fd8b4b696-t2zrd" event={"ID":"b084a8cb-2f9b-4e2d-8e48-a5e84981c169","Type":"ContainerStarted","Data":"3db1d746bc99fc029e2b288f9093ef7d1f86b800b746b1ab4c63212280588bbb"}
Jul 20 16:05:17.656479 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:17.656210    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a667cd-7fd8b4b696-t2zrd"
Jul 20 16:05:20.198730 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:20.198694    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:23.665724 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:23.665692    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a667cd-7fd8b4b696-t2zrd"
Jul 20 16:05:23.686726 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:23.686676    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a667cd-7fd8b4b696-t2zrd" podStartSLOduration=8.186946021 podStartE2EDuration="8.686661009s" totalImagesPullingTime="499.714988ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:05:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:05:17.702259368 +0000 UTC m=+2300.711881029" watchObservedRunningTime="2026-07-20 16:05:23.686661009 +0000 UTC m=+2306.696282672"
Jul 20 16:05:29.897822 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:29.897784    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:30.001936 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:30.001901    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:30.703227 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:30.702602    2559 patch_prober.go:28] interesting pod/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": read tcp 10.134.0.2:40938->10.134.0.57:15021: read: connection reset by peer" start-of-body=
Jul 20 16:05:30.703227 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:30.702658    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d" podUID="07d5a0a6-90f0-4301-86d2-4bc2326a35f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": read tcp 10.134.0.2:40938->10.134.0.57:15021: read: connection reset by peer"
Jul 20 16:05:31.727377 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:31.727343    2559 generic.go:356] "Generic (PLEG): container finished" podID="07d5a0a6-90f0-4301-86d2-4bc2326a35f6" containerID="1a07221b8432adac8496971cb11392f56d2d95a1b8d3ab43cb312e923c328ecc" exitCode=0
Jul 20 16:05:31.727377 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:31.727380    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d" event={"ID":"07d5a0a6-90f0-4301-86d2-4bc2326a35f6","Type":"ContainerDied","Data":"1a07221b8432adac8496971cb11392f56d2d95a1b8d3ab43cb312e923c328ecc"}
Jul 20 16:05:31.727810 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:31.727746    2559 scope.go:117] "RemoveContainer" containerID="1a07221b8432adac8496971cb11392f56d2d95a1b8d3ab43cb312e923c328ecc"
Jul 20 16:05:31.728696 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:31.728667    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 16:05:31.728765 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:31.728734    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 16:05:31.728811 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:31.728764    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 16:05:32.736600 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:32.736562    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d" event={"ID":"07d5a0a6-90f0-4301-86d2-4bc2326a35f6","Type":"ContainerStarted","Data":"3c626a0effa88f501faeda82431b87223751d4341fbed454874828eb7900d893"}
Jul 20 16:05:32.971164 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:32.971120    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"
Jul 20 16:05:32.971428 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:32.971404    2559 patch_prober.go:28] interesting pod/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 20 16:05:32.971483 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:32.971456    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d" podUID="07d5a0a6-90f0-4301-86d2-4bc2326a35f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 20 16:05:33.970569 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:33.970536    2559 patch_prober.go:28] interesting pod/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 20 16:05:33.970965 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:33.970585    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d" podUID="07d5a0a6-90f0-4301-86d2-4bc2326a35f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 20 16:05:34.970804 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:34.970770    2559 patch_prober.go:28] interesting pod/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 20 16:05:34.971193 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:34.970825    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d" podUID="07d5a0a6-90f0-4301-86d2-4bc2326a35f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 20 16:05:35.970499 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:35.970464    2559 patch_prober.go:28] interesting pod/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 20 16:05:35.970659 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:35.970528    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d" podUID="07d5a0a6-90f0-4301-86d2-4bc2326a35f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 20 16:05:36.323544 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:36.323516    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-8791e0/auth-test-model-8791e0-kserve-754dd8cf95-l8pw2"]
Jul 20 16:05:36.326169 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:36.326152    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8791e0/auth-test-model-8791e0-kserve-754dd8cf95-l8pw2"
Jul 20 16:05:36.334316 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:36.332415    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-8791e0\"/\"kube-root-ca.crt\""
Jul 20 16:05:36.334316 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:36.332660    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-8791e0\"/\"auth-test-model-8791e0-kserve-self-signed-certs\""
Jul 20 16:05:36.334316 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:36.332853    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-8791e0\"/\"openshift-service-ca.crt\""
Jul 20 16:05:36.334316 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:36.333038    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-8791e0\"/\"default-dockercfg-hpcc6\""
Jul 20 16:05:36.337156 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:36.337123    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8791e0/auth-test-model-8791e0-kserve-754dd8cf95-l8pw2"]
Jul 20 16:05:36.350568 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:36.350143    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:36.449042 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:36.448666    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87534ae7-a3eb-47d2-9029-751af6ad4aa9-tmp-dir\") pod \"auth-test-model-8791e0-kserve-754dd8cf95-l8pw2\" (UID: \"87534ae7-a3eb-47d2-9029-751af6ad4aa9\") " pod="ai-tenant-e2e-shared-a-8791e0/auth-test-model-8791e0-kserve-754dd8cf95-l8pw2"
Jul 20 16:05:36.449042 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:36.448740    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87534ae7-a3eb-47d2-9029-751af6ad4aa9-dshm\") pod \"auth-test-model-8791e0-kserve-754dd8cf95-l8pw2\" (UID: \"87534ae7-a3eb-47d2-9029-751af6ad4aa9\") " pod="ai-tenant-e2e-shared-a-8791e0/auth-test-model-8791e0-kserve-754dd8cf95-l8pw2"
Jul 20 16:05:36.449042 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:36.448772    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v969m\" (UniqueName: \"kubernetes.io/projected/87534ae7-a3eb-47d2-9029-751af6ad4aa9-kube-api-access-v969m\") pod \"auth-test-model-8791e0-kserve-754dd8cf95-l8pw2\" (UID: \"87534ae7-a3eb-47d2-9029-751af6ad4aa9\") " pod="ai-tenant-e2e-shared-a-8791e0/auth-test-model-8791e0-kserve-754dd8cf95-l8pw2"
Jul 20 16:05:36.449042 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:36.448839    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87534ae7-a3eb-47d2-9029-751af6ad4aa9-home\") pod \"auth-test-model-8791e0-kserve-754dd8cf95-l8pw2\" (UID: \"87534ae7-a3eb-47d2-9029-751af6ad4aa9\") " pod="ai-tenant-e2e-shared-a-8791e0/auth-test-model-8791e0-kserve-754dd8cf95-l8pw2"
Jul 20 16:05:36.449042 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:36.448884    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87534ae7-a3eb-47d2-9029-751af6ad4aa9-model-cache\") pod \"auth-test-model-8791e0-kserve-754dd8cf95-l8pw2\" (UID: \"87534ae7-a3eb-47d2-9029-751af6ad4aa9\") " pod="ai-tenant-e2e-shared-a-8791e0/auth-test-model-8791e0-kserve-754dd8cf95-l8pw2"
Jul 20 16:05:36.449042 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:36.448909    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87534ae7-a3eb-47d2-9029-751af6ad4aa9-tls-certs\") pod \"auth-test-model-8791e0-kserve-754dd8cf95-l8pw2\" (UID: \"87534ae7-a3eb-47d2-9029-751af6ad4aa9\") " pod="ai-tenant-e2e-shared-a-8791e0/auth-test-model-8791e0-kserve-754dd8cf95-l8pw2"
Jul 20 16:05:36.549593 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:36.549548    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87534ae7-a3eb-47d2-9029-751af6ad4aa9-dshm\") pod \"auth-test-model-8791e0-kserve-754dd8cf95-l8pw2\" (UID: \"87534ae7-a3eb-47d2-9029-751af6ad4aa9\") " pod="ai-tenant-e2e-shared-a-8791e0/auth-test-model-8791e0-kserve-754dd8cf95-l8pw2"
Jul 20 16:05:36.549778 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:36.549598    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v969m\" (UniqueName: \"kubernetes.io/projected/87534ae7-a3eb-47d2-9029-751af6ad4aa9-kube-api-access-v969m\") pod \"auth-test-model-8791e0-kserve-754dd8cf95-l8pw2\" (UID: \"87534ae7-a3eb-47d2-9029-751af6ad4aa9\") " pod="ai-tenant-e2e-shared-a-8791e0/auth-test-model-8791e0-kserve-754dd8cf95-l8pw2"
Jul 20 16:05:36.549778 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:36.549651    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87534ae7-a3eb-47d2-9029-751af6ad4aa9-home\") pod \"auth-test-model-8791e0-kserve-754dd8cf95-l8pw2\" (UID: \"87534ae7-a3eb-47d2-9029-751af6ad4aa9\") " pod="ai-tenant-e2e-shared-a-8791e0/auth-test-model-8791e0-kserve-754dd8cf95-l8pw2"
Jul 20 16:05:36.549778 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:36.549693    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87534ae7-a3eb-47d2-9029-751af6ad4aa9-model-cache\") pod \"auth-test-model-8791e0-kserve-754dd8cf95-l8pw2\" (UID: \"87534ae7-a3eb-47d2-9029-751af6ad4aa9\") " pod="ai-tenant-e2e-shared-a-8791e0/auth-test-model-8791e0-kserve-754dd8cf95-l8pw2"
Jul 20 16:05:36.549778 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:36.549727    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87534ae7-a3eb-47d2-9029-751af6ad4aa9-tls-certs\") pod \"auth-test-model-8791e0-kserve-754dd8cf95-l8pw2\" (UID: \"87534ae7-a3eb-47d2-9029-751af6ad4aa9\") " pod="ai-tenant-e2e-shared-a-8791e0/auth-test-model-8791e0-kserve-754dd8cf95-l8pw2"
Jul 20 16:05:36.550024 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:36.549934    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87534ae7-a3eb-47d2-9029-751af6ad4aa9-tmp-dir\") pod \"auth-test-model-8791e0-kserve-754dd8cf95-l8pw2\" (UID: \"87534ae7-a3eb-47d2-9029-751af6ad4aa9\") " pod="ai-tenant-e2e-shared-a-8791e0/auth-test-model-8791e0-kserve-754dd8cf95-l8pw2"
Jul 20 16:05:36.550081 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:36.550050    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87534ae7-a3eb-47d2-9029-751af6ad4aa9-model-cache\") pod \"auth-test-model-8791e0-kserve-754dd8cf95-l8pw2\" (UID: \"87534ae7-a3eb-47d2-9029-751af6ad4aa9\") " pod="ai-tenant-e2e-shared-a-8791e0/auth-test-model-8791e0-kserve-754dd8cf95-l8pw2"
Jul 20 16:05:36.550081 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:36.550063    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87534ae7-a3eb-47d2-9029-751af6ad4aa9-home\") pod \"auth-test-model-8791e0-kserve-754dd8cf95-l8pw2\" (UID: \"87534ae7-a3eb-47d2-9029-751af6ad4aa9\") " pod="ai-tenant-e2e-shared-a-8791e0/auth-test-model-8791e0-kserve-754dd8cf95-l8pw2"
Jul 20 16:05:36.550403 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:36.550379    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87534ae7-a3eb-47d2-9029-751af6ad4aa9-tmp-dir\") pod \"auth-test-model-8791e0-kserve-754dd8cf95-l8pw2\" (UID: \"87534ae7-a3eb-47d2-9029-751af6ad4aa9\") " pod="ai-tenant-e2e-shared-a-8791e0/auth-test-model-8791e0-kserve-754dd8cf95-l8pw2"
Jul 20 16:05:36.551850 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:36.551821    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87534ae7-a3eb-47d2-9029-751af6ad4aa9-dshm\") pod \"auth-test-model-8791e0-kserve-754dd8cf95-l8pw2\" (UID: \"87534ae7-a3eb-47d2-9029-751af6ad4aa9\") " pod="ai-tenant-e2e-shared-a-8791e0/auth-test-model-8791e0-kserve-754dd8cf95-l8pw2"
Jul 20 16:05:36.552183 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:36.552163    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87534ae7-a3eb-47d2-9029-751af6ad4aa9-tls-certs\") pod \"auth-test-model-8791e0-kserve-754dd8cf95-l8pw2\" (UID: \"87534ae7-a3eb-47d2-9029-751af6ad4aa9\") " pod="ai-tenant-e2e-shared-a-8791e0/auth-test-model-8791e0-kserve-754dd8cf95-l8pw2"
Jul 20 16:05:36.558214 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:36.558187    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v969m\" (UniqueName: \"kubernetes.io/projected/87534ae7-a3eb-47d2-9029-751af6ad4aa9-kube-api-access-v969m\") pod \"auth-test-model-8791e0-kserve-754dd8cf95-l8pw2\" (UID: \"87534ae7-a3eb-47d2-9029-751af6ad4aa9\") " pod="ai-tenant-e2e-shared-a-8791e0/auth-test-model-8791e0-kserve-754dd8cf95-l8pw2"
Jul 20 16:05:36.651229 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:36.651138    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8791e0/auth-test-model-8791e0-kserve-754dd8cf95-l8pw2"
Jul 20 16:05:36.783917 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:36.783895    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8791e0/auth-test-model-8791e0-kserve-754dd8cf95-l8pw2"]
Jul 20 16:05:36.785344 ip-10-0-130-19 kubenswrapper[2559]: W0720 16:05:36.785320    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod87534ae7_a3eb_47d2_9029_751af6ad4aa9.slice/crio-db2b8e5ad909a0ae5bcaf7aa86ce96ddcf3845f797470e5dd6d6614021c543dc WatchSource:0}: Error finding container db2b8e5ad909a0ae5bcaf7aa86ce96ddcf3845f797470e5dd6d6614021c543dc: Status 404 returned error can't find the container with id db2b8e5ad909a0ae5bcaf7aa86ce96ddcf3845f797470e5dd6d6614021c543dc
Jul 20 16:05:36.974668 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:36.974596    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"
Jul 20 16:05:36.981001 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:36.975628    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"
Jul 20 16:05:36.981714 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:36.981693    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"
Jul 20 16:05:37.607672 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:37.607641    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:37.707546 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:37.707511    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:37.767338 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:37.767305    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8791e0/auth-test-model-8791e0-kserve-754dd8cf95-l8pw2" event={"ID":"87534ae7-a3eb-47d2-9029-751af6ad4aa9","Type":"ContainerStarted","Data":"ce01a86d62d826916decf832e491de6a4643bc396c503d7b61bd4f18646b2e91"}
Jul 20 16:05:37.767502 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:37.767349    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8791e0/auth-test-model-8791e0-kserve-754dd8cf95-l8pw2" event={"ID":"87534ae7-a3eb-47d2-9029-751af6ad4aa9","Type":"ContainerStarted","Data":"db2b8e5ad909a0ae5bcaf7aa86ce96ddcf3845f797470e5dd6d6614021c543dc"}
Jul 20 16:05:37.787221 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:37.787174    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-8791e0/auth-test-model-8791e0-kserve-754dd8cf95-l8pw2" podStartSLOduration=1.787151112 podStartE2EDuration="1.787151112s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:05:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:05:37.784598349 +0000 UTC m=+2320.794220013" watchObservedRunningTime="2026-07-20 16:05:37.787151112 +0000 UTC m=+2320.796772833"
Jul 20 16:05:38.570621 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:38.570586    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:38.671609 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:38.671532    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:39.375630 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:39.375596    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:39.477993 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:39.477963    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:05:46.651712 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:46.651679    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-8791e0/auth-test-model-8791e0-kserve-754dd8cf95-l8pw2"
Jul 20 16:05:46.652102 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:46.652010    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-8791e0/auth-test-model-8791e0-kserve-754dd8cf95-l8pw2"
Jul 20 16:05:46.664351 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:46.664330    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-8791e0/auth-test-model-8791e0-kserve-754dd8cf95-l8pw2"
Jul 20 16:05:46.820639 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:05:46.820603    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-8791e0/auth-test-model-8791e0-kserve-754dd8cf95-l8pw2"
Jul 20 16:06:09.495208 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:06:09.495175    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-a667cd/auth-test-model-a667cd-kserve-6bb5454987-lc5ks"]
Jul 20 16:06:09.497942 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:06:09.497914    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a667cd/auth-test-model-a667cd-kserve-6bb5454987-lc5ks"
Jul 20 16:06:09.501080 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:06:09.501053    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a667cd\"/\"kube-root-ca.crt\""
Jul 20 16:06:09.502070 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:06:09.502047    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a667cd\"/\"openshift-service-ca.crt\""
Jul 20 16:06:09.502228 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:06:09.502209    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a667cd\"/\"auth-test-model-a667cd-kserve-self-signed-certs\""
Jul 20 16:06:09.502313 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:06:09.502222    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a667cd\"/\"default-dockercfg-pdcht\""
Jul 20 16:06:09.514305 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:06:09.514264    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a667cd/auth-test-model-a667cd-kserve-6bb5454987-lc5ks"]
Jul 20 16:06:09.533302 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:06:09.533238    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:06:09.637615 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:06:09.637583    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4-tmp-dir\") pod \"auth-test-model-a667cd-kserve-6bb5454987-lc5ks\" (UID: \"dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4\") " pod="ai-tenant-e2e-shared-b-a667cd/auth-test-model-a667cd-kserve-6bb5454987-lc5ks"
Jul 20 16:06:09.637799 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:06:09.637626    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4-tls-certs\") pod \"auth-test-model-a667cd-kserve-6bb5454987-lc5ks\" (UID: \"dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4\") " pod="ai-tenant-e2e-shared-b-a667cd/auth-test-model-a667cd-kserve-6bb5454987-lc5ks"
Jul 20 16:06:09.637799 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:06:09.637726    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4-dshm\") pod \"auth-test-model-a667cd-kserve-6bb5454987-lc5ks\" (UID: \"dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4\") " pod="ai-tenant-e2e-shared-b-a667cd/auth-test-model-a667cd-kserve-6bb5454987-lc5ks"
Jul 20 16:06:09.637799 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:06:09.637786    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4-home\") pod \"auth-test-model-a667cd-kserve-6bb5454987-lc5ks\" (UID: \"dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4\") " pod="ai-tenant-e2e-shared-b-a667cd/auth-test-model-a667cd-kserve-6bb5454987-lc5ks"
Jul 20 16:06:09.637974 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:06:09.637818    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ls6mz\" (UniqueName: \"kubernetes.io/projected/dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4-kube-api-access-ls6mz\") pod \"auth-test-model-a667cd-kserve-6bb5454987-lc5ks\" (UID: \"dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4\") " pod="ai-tenant-e2e-shared-b-a667cd/auth-test-model-a667cd-kserve-6bb5454987-lc5ks"
Jul 20 16:06:09.637974 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:06:09.637863    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4-model-cache\") pod \"auth-test-model-a667cd-kserve-6bb5454987-lc5ks\" (UID: \"dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4\") " pod="ai-tenant-e2e-shared-b-a667cd/auth-test-model-a667cd-kserve-6bb5454987-lc5ks"
Jul 20 16:06:09.738851 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:06:09.738813    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4-tmp-dir\") pod \"auth-test-model-a667cd-kserve-6bb5454987-lc5ks\" (UID: \"dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4\") " pod="ai-tenant-e2e-shared-b-a667cd/auth-test-model-a667cd-kserve-6bb5454987-lc5ks"
Jul 20 16:06:09.738851 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:06:09.738853    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4-tls-certs\") pod \"auth-test-model-a667cd-kserve-6bb5454987-lc5ks\" (UID: \"dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4\") " pod="ai-tenant-e2e-shared-b-a667cd/auth-test-model-a667cd-kserve-6bb5454987-lc5ks"
Jul 20 16:06:09.739076 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:06:09.738885    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4-dshm\") pod \"auth-test-model-a667cd-kserve-6bb5454987-lc5ks\" (UID: \"dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4\") " pod="ai-tenant-e2e-shared-b-a667cd/auth-test-model-a667cd-kserve-6bb5454987-lc5ks"
Jul 20 16:06:09.739076 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:06:09.738929    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4-home\") pod \"auth-test-model-a667cd-kserve-6bb5454987-lc5ks\" (UID: \"dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4\") " pod="ai-tenant-e2e-shared-b-a667cd/auth-test-model-a667cd-kserve-6bb5454987-lc5ks"
Jul 20 16:06:09.739076 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:06:09.738951    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ls6mz\" (UniqueName: \"kubernetes.io/projected/dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4-kube-api-access-ls6mz\") pod \"auth-test-model-a667cd-kserve-6bb5454987-lc5ks\" (UID: \"dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4\") " pod="ai-tenant-e2e-shared-b-a667cd/auth-test-model-a667cd-kserve-6bb5454987-lc5ks"
Jul 20 16:06:09.739076 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:06:09.738994    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4-model-cache\") pod \"auth-test-model-a667cd-kserve-6bb5454987-lc5ks\" (UID: \"dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4\") " pod="ai-tenant-e2e-shared-b-a667cd/auth-test-model-a667cd-kserve-6bb5454987-lc5ks"
Jul 20 16:06:09.739332 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:06:09.739308    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4-tmp-dir\") pod \"auth-test-model-a667cd-kserve-6bb5454987-lc5ks\" (UID: \"dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4\") " pod="ai-tenant-e2e-shared-b-a667cd/auth-test-model-a667cd-kserve-6bb5454987-lc5ks"
Jul 20 16:06:09.739403 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:06:09.739378    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4-home\") pod \"auth-test-model-a667cd-kserve-6bb5454987-lc5ks\" (UID: \"dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4\") " pod="ai-tenant-e2e-shared-b-a667cd/auth-test-model-a667cd-kserve-6bb5454987-lc5ks"
Jul 20 16:06:09.739450 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:06:09.739433    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4-model-cache\") pod \"auth-test-model-a667cd-kserve-6bb5454987-lc5ks\" (UID: \"dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4\") " pod="ai-tenant-e2e-shared-b-a667cd/auth-test-model-a667cd-kserve-6bb5454987-lc5ks"
Jul 20 16:06:09.741177 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:06:09.741153    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4-dshm\") pod \"auth-test-model-a667cd-kserve-6bb5454987-lc5ks\" (UID: \"dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4\") " pod="ai-tenant-e2e-shared-b-a667cd/auth-test-model-a667cd-kserve-6bb5454987-lc5ks"
Jul 20 16:06:09.741600 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:06:09.741579    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4-tls-certs\") pod \"auth-test-model-a667cd-kserve-6bb5454987-lc5ks\" (UID: \"dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4\") " pod="ai-tenant-e2e-shared-b-a667cd/auth-test-model-a667cd-kserve-6bb5454987-lc5ks"
Jul 20 16:06:09.747823 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:06:09.747765    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ls6mz\" (UniqueName: \"kubernetes.io/projected/dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4-kube-api-access-ls6mz\") pod \"auth-test-model-a667cd-kserve-6bb5454987-lc5ks\" (UID: \"dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4\") " pod="ai-tenant-e2e-shared-b-a667cd/auth-test-model-a667cd-kserve-6bb5454987-lc5ks"
Jul 20 16:06:09.808102 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:06:09.808070    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a667cd/auth-test-model-a667cd-kserve-6bb5454987-lc5ks"
Jul 20 16:06:09.937782 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:06:09.937755    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a667cd/auth-test-model-a667cd-kserve-6bb5454987-lc5ks"]
Jul 20 16:06:09.939751 ip-10-0-130-19 kubenswrapper[2559]: W0720 16:06:09.939724    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddd7a62ac_deb5_48f2_a66d_ac3c374ab4b4.slice/crio-1b5202f57537fa6524939cb79428815ec1dda81ddc1eb12d52ebd525e864ea92 WatchSource:0}: Error finding container 1b5202f57537fa6524939cb79428815ec1dda81ddc1eb12d52ebd525e864ea92: Status 404 returned error can't find the container with id 1b5202f57537fa6524939cb79428815ec1dda81ddc1eb12d52ebd525e864ea92
Jul 20 16:06:10.919553 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:06:10.919520    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a667cd/auth-test-model-a667cd-kserve-6bb5454987-lc5ks" event={"ID":"dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4","Type":"ContainerStarted","Data":"d668127a116009f7d65e51b617b283975fc3471f28aed14e2997ab249caf06c8"}
Jul 20 16:06:10.919553 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:06:10.919557    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a667cd/auth-test-model-a667cd-kserve-6bb5454987-lc5ks" event={"ID":"dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4","Type":"ContainerStarted","Data":"1b5202f57537fa6524939cb79428815ec1dda81ddc1eb12d52ebd525e864ea92"}
Jul 20 16:06:10.936894 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:06:10.936814    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-a667cd/auth-test-model-a667cd-kserve-6bb5454987-lc5ks" podStartSLOduration=1.936798383 podStartE2EDuration="1.936798383s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:06:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:06:10.935547087 +0000 UTC m=+2353.945168752" watchObservedRunningTime="2026-07-20 16:06:10.936798383 +0000 UTC m=+2353.946420094"
Jul 20 16:06:19.808257 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:06:19.808221    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-a667cd/auth-test-model-a667cd-kserve-6bb5454987-lc5ks"
Jul 20 16:06:19.808715 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:06:19.808280    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-a667cd/auth-test-model-a667cd-kserve-6bb5454987-lc5ks"
Jul 20 16:06:19.820863 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:06:19.820837    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-a667cd/auth-test-model-a667cd-kserve-6bb5454987-lc5ks"
Jul 20 16:06:19.971678 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:06:19.971647    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-a667cd/auth-test-model-a667cd-kserve-6bb5454987-lc5ks"
Jul 20 16:06:27.203174 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:06:27.203131    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:08:04.478018 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:04.477985    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8791e0/auth-test-model-8791e0-kserve-754dd8cf95-l8pw2"]
Jul 20 16:08:04.547158 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:04.547124    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:08:05.467104 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:05.467067    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a667cd/auth-test-model-a667cd-kserve-6bb5454987-lc5ks"]
Jul 20 16:08:06.046374 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:06.046344    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:08:19.497777 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:19.497736    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-8791e0/auth-test-model-8791e0-kserve-754dd8cf95-l8pw2" podUID="87534ae7-a3eb-47d2-9029-751af6ad4aa9" containerName="main" containerID="cri-o://ce01a86d62d826916decf832e491de6a4643bc396c503d7b61bd4f18646b2e91" gracePeriod=45
Jul 20 16:08:19.739859 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:19.739834    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8791e0/auth-test-model-8791e0-kserve-754dd8cf95-l8pw2"
Jul 20 16:08:19.765552 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:19.765526    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87534ae7-a3eb-47d2-9029-751af6ad4aa9-model-cache\") pod \"87534ae7-a3eb-47d2-9029-751af6ad4aa9\" (UID: \"87534ae7-a3eb-47d2-9029-751af6ad4aa9\") "
Jul 20 16:08:19.765690 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:19.765594    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87534ae7-a3eb-47d2-9029-751af6ad4aa9-tmp-dir\") pod \"87534ae7-a3eb-47d2-9029-751af6ad4aa9\" (UID: \"87534ae7-a3eb-47d2-9029-751af6ad4aa9\") "
Jul 20 16:08:19.765690 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:19.765656    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87534ae7-a3eb-47d2-9029-751af6ad4aa9-tls-certs\") pod \"87534ae7-a3eb-47d2-9029-751af6ad4aa9\" (UID: \"87534ae7-a3eb-47d2-9029-751af6ad4aa9\") "
Jul 20 16:08:19.765690 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:19.765685    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v969m\" (UniqueName: \"kubernetes.io/projected/87534ae7-a3eb-47d2-9029-751af6ad4aa9-kube-api-access-v969m\") pod \"87534ae7-a3eb-47d2-9029-751af6ad4aa9\" (UID: \"87534ae7-a3eb-47d2-9029-751af6ad4aa9\") "
Jul 20 16:08:19.765860 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:19.765754    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87534ae7-a3eb-47d2-9029-751af6ad4aa9-model-cache" (OuterVolumeSpecName: "model-cache") pod "87534ae7-a3eb-47d2-9029-751af6ad4aa9" (UID: "87534ae7-a3eb-47d2-9029-751af6ad4aa9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:08:19.765860 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:19.765762    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87534ae7-a3eb-47d2-9029-751af6ad4aa9-dshm\") pod \"87534ae7-a3eb-47d2-9029-751af6ad4aa9\" (UID: \"87534ae7-a3eb-47d2-9029-751af6ad4aa9\") "
Jul 20 16:08:19.765860 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:19.765845    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87534ae7-a3eb-47d2-9029-751af6ad4aa9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "87534ae7-a3eb-47d2-9029-751af6ad4aa9" (UID: "87534ae7-a3eb-47d2-9029-751af6ad4aa9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:08:19.766000 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:19.765896    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87534ae7-a3eb-47d2-9029-751af6ad4aa9-home\") pod \"87534ae7-a3eb-47d2-9029-751af6ad4aa9\" (UID: \"87534ae7-a3eb-47d2-9029-751af6ad4aa9\") "
Jul 20 16:08:19.766104 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:19.766082    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87534ae7-a3eb-47d2-9029-751af6ad4aa9-home" (OuterVolumeSpecName: "home") pod "87534ae7-a3eb-47d2-9029-751af6ad4aa9" (UID: "87534ae7-a3eb-47d2-9029-751af6ad4aa9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:08:19.766239 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:19.766220    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87534ae7-a3eb-47d2-9029-751af6ad4aa9-model-cache\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:08:19.766336 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:19.766244    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87534ae7-a3eb-47d2-9029-751af6ad4aa9-tmp-dir\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:08:19.766336 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:19.766253    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87534ae7-a3eb-47d2-9029-751af6ad4aa9-home\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:08:19.767950 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:19.767914    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87534ae7-a3eb-47d2-9029-751af6ad4aa9-kube-api-access-v969m" (OuterVolumeSpecName: "kube-api-access-v969m") pod "87534ae7-a3eb-47d2-9029-751af6ad4aa9" (UID: "87534ae7-a3eb-47d2-9029-751af6ad4aa9"). InnerVolumeSpecName "kube-api-access-v969m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:08:19.768090 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:19.768072    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87534ae7-a3eb-47d2-9029-751af6ad4aa9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "87534ae7-a3eb-47d2-9029-751af6ad4aa9" (UID: "87534ae7-a3eb-47d2-9029-751af6ad4aa9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:08:19.768146 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:19.768086    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87534ae7-a3eb-47d2-9029-751af6ad4aa9-dshm" (OuterVolumeSpecName: "dshm") pod "87534ae7-a3eb-47d2-9029-751af6ad4aa9" (UID: "87534ae7-a3eb-47d2-9029-751af6ad4aa9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:08:19.866667 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:19.866640    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87534ae7-a3eb-47d2-9029-751af6ad4aa9-tls-certs\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:08:19.866667 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:19.866664    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v969m\" (UniqueName: \"kubernetes.io/projected/87534ae7-a3eb-47d2-9029-751af6ad4aa9-kube-api-access-v969m\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:08:19.866667 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:19.866673    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87534ae7-a3eb-47d2-9029-751af6ad4aa9-dshm\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:08:20.487650 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:20.487614    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-a667cd/auth-test-model-a667cd-kserve-6bb5454987-lc5ks" podUID="dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4" containerName="main" containerID="cri-o://d668127a116009f7d65e51b617b283975fc3471f28aed14e2997ab249caf06c8" gracePeriod=45
Jul 20 16:08:20.540736 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:20.540702    2559 generic.go:356] "Generic (PLEG): container finished" podID="87534ae7-a3eb-47d2-9029-751af6ad4aa9" containerID="ce01a86d62d826916decf832e491de6a4643bc396c503d7b61bd4f18646b2e91" exitCode=0
Jul 20 16:08:20.541044 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:20.540735    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8791e0/auth-test-model-8791e0-kserve-754dd8cf95-l8pw2"
Jul 20 16:08:20.541044 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:20.540754    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8791e0/auth-test-model-8791e0-kserve-754dd8cf95-l8pw2" event={"ID":"87534ae7-a3eb-47d2-9029-751af6ad4aa9","Type":"ContainerDied","Data":"ce01a86d62d826916decf832e491de6a4643bc396c503d7b61bd4f18646b2e91"}
Jul 20 16:08:20.541044 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:20.540793    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8791e0/auth-test-model-8791e0-kserve-754dd8cf95-l8pw2" event={"ID":"87534ae7-a3eb-47d2-9029-751af6ad4aa9","Type":"ContainerDied","Data":"db2b8e5ad909a0ae5bcaf7aa86ce96ddcf3845f797470e5dd6d6614021c543dc"}
Jul 20 16:08:20.541044 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:20.540818    2559 scope.go:117] "RemoveContainer" containerID="ce01a86d62d826916decf832e491de6a4643bc396c503d7b61bd4f18646b2e91"
Jul 20 16:08:20.566682 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:20.566657    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8791e0/auth-test-model-8791e0-kserve-754dd8cf95-l8pw2"]
Jul 20 16:08:20.570627 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:20.570605    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-8791e0/auth-test-model-8791e0-kserve-754dd8cf95-l8pw2"]
Jul 20 16:08:20.598057 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:20.598036    2559 scope.go:117] "RemoveContainer" containerID="ce01a86d62d826916decf832e491de6a4643bc396c503d7b61bd4f18646b2e91"
Jul 20 16:08:20.598348 ip-10-0-130-19 kubenswrapper[2559]: E0720 16:08:20.598330    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce01a86d62d826916decf832e491de6a4643bc396c503d7b61bd4f18646b2e91\": container with ID starting with ce01a86d62d826916decf832e491de6a4643bc396c503d7b61bd4f18646b2e91 not found: ID does not exist" containerID="ce01a86d62d826916decf832e491de6a4643bc396c503d7b61bd4f18646b2e91"
Jul 20 16:08:20.598423 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:20.598355    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce01a86d62d826916decf832e491de6a4643bc396c503d7b61bd4f18646b2e91"} err="failed to get container status \"ce01a86d62d826916decf832e491de6a4643bc396c503d7b61bd4f18646b2e91\": rpc error: code = NotFound desc = could not find container \"ce01a86d62d826916decf832e491de6a4643bc396c503d7b61bd4f18646b2e91\": container with ID starting with ce01a86d62d826916decf832e491de6a4643bc396c503d7b61bd4f18646b2e91 not found: ID does not exist"
Jul 20 16:08:20.740009 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:20.739957    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a667cd/auth-test-model-a667cd-kserve-6bb5454987-lc5ks"
Jul 20 16:08:20.774375 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:20.774352    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4-tls-certs\") pod \"dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4\" (UID: \"dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4\") "
Jul 20 16:08:20.774491 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:20.774434    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4-model-cache\") pod \"dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4\" (UID: \"dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4\") "
Jul 20 16:08:20.774491 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:20.774466    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ls6mz\" (UniqueName: \"kubernetes.io/projected/dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4-kube-api-access-ls6mz\") pod \"dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4\" (UID: \"dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4\") "
Jul 20 16:08:20.774491 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:20.774485    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4-tmp-dir\") pod \"dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4\" (UID: \"dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4\") "
Jul 20 16:08:20.774658 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:20.774499    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4-home\") pod \"dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4\" (UID: \"dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4\") "
Jul 20 16:08:20.774658 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:20.774522    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4-dshm\") pod \"dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4\" (UID: \"dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4\") "
Jul 20 16:08:20.774770 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:20.774717    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4-model-cache" (OuterVolumeSpecName: "model-cache") pod "dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4" (UID: "dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:08:20.774770 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:20.774732    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4" (UID: "dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:08:20.774770 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:20.774749    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4-home" (OuterVolumeSpecName: "home") pod "dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4" (UID: "dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:08:20.774943 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:20.774848    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4-model-cache\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:08:20.774943 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:20.774867    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4-tmp-dir\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:08:20.774943 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:20.774875    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4-home\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:08:20.776726 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:20.776705    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4-dshm" (OuterVolumeSpecName: "dshm") pod "dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4" (UID: "dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:08:20.776818 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:20.776722    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4-kube-api-access-ls6mz" (OuterVolumeSpecName: "kube-api-access-ls6mz") pod "dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4" (UID: "dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4"). InnerVolumeSpecName "kube-api-access-ls6mz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:08:20.776861 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:20.776839    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4" (UID: "dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:08:20.876117 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:20.876092    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ls6mz\" (UniqueName: \"kubernetes.io/projected/dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4-kube-api-access-ls6mz\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:08:20.876117 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:20.876114    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4-dshm\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:08:20.876261 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:20.876124    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4-tls-certs\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:08:21.514865 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:21.514834    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87534ae7-a3eb-47d2-9029-751af6ad4aa9" path="/var/lib/kubelet/pods/87534ae7-a3eb-47d2-9029-751af6ad4aa9/volumes"
Jul 20 16:08:21.546446 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:21.546421    2559 generic.go:356] "Generic (PLEG): container finished" podID="dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4" containerID="d668127a116009f7d65e51b617b283975fc3471f28aed14e2997ab249caf06c8" exitCode=0
Jul 20 16:08:21.546804 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:21.546464    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a667cd/auth-test-model-a667cd-kserve-6bb5454987-lc5ks"
Jul 20 16:08:21.546804 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:21.546496    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a667cd/auth-test-model-a667cd-kserve-6bb5454987-lc5ks" event={"ID":"dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4","Type":"ContainerDied","Data":"d668127a116009f7d65e51b617b283975fc3471f28aed14e2997ab249caf06c8"}
Jul 20 16:08:21.546804 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:21.546524    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a667cd/auth-test-model-a667cd-kserve-6bb5454987-lc5ks" event={"ID":"dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4","Type":"ContainerDied","Data":"1b5202f57537fa6524939cb79428815ec1dda81ddc1eb12d52ebd525e864ea92"}
Jul 20 16:08:21.546804 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:21.546541    2559 scope.go:117] "RemoveContainer" containerID="d668127a116009f7d65e51b617b283975fc3471f28aed14e2997ab249caf06c8"
Jul 20 16:08:21.554843 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:21.554827    2559 scope.go:117] "RemoveContainer" containerID="d668127a116009f7d65e51b617b283975fc3471f28aed14e2997ab249caf06c8"
Jul 20 16:08:21.555087 ip-10-0-130-19 kubenswrapper[2559]: E0720 16:08:21.555071    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d668127a116009f7d65e51b617b283975fc3471f28aed14e2997ab249caf06c8\": container with ID starting with d668127a116009f7d65e51b617b283975fc3471f28aed14e2997ab249caf06c8 not found: ID does not exist" containerID="d668127a116009f7d65e51b617b283975fc3471f28aed14e2997ab249caf06c8"
Jul 20 16:08:21.555165 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:21.555093    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d668127a116009f7d65e51b617b283975fc3471f28aed14e2997ab249caf06c8"} err="failed to get container status \"d668127a116009f7d65e51b617b283975fc3471f28aed14e2997ab249caf06c8\": rpc error: code = NotFound desc = could not find container \"d668127a116009f7d65e51b617b283975fc3471f28aed14e2997ab249caf06c8\": container with ID starting with d668127a116009f7d65e51b617b283975fc3471f28aed14e2997ab249caf06c8 not found: ID does not exist"
Jul 20 16:08:21.566629 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:21.566607    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a667cd/auth-test-model-a667cd-kserve-6bb5454987-lc5ks"]
Jul 20 16:08:21.568635 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:21.568619    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-a667cd/auth-test-model-a667cd-kserve-6bb5454987-lc5ks"]
Jul 20 16:08:23.514525 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:23.514494    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4" path="/var/lib/kubelet/pods/dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4/volumes"
Jul 20 16:08:36.092907 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:36.092875    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-8791e0/sub-test-model-8791e0-kserve-56b67c844c-fsjvd"]
Jul 20 16:08:36.093355 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:36.093264    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4" containerName="main"
Jul 20 16:08:36.093355 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:36.093276    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4" containerName="main"
Jul 20 16:08:36.093355 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:36.093304    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87534ae7-a3eb-47d2-9029-751af6ad4aa9" containerName="main"
Jul 20 16:08:36.093355 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:36.093310    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="87534ae7-a3eb-47d2-9029-751af6ad4aa9" containerName="main"
Jul 20 16:08:36.093551 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:36.093384    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="87534ae7-a3eb-47d2-9029-751af6ad4aa9" containerName="main"
Jul 20 16:08:36.093551 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:36.093403    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd7a62ac-deb5-48f2-a66d-ac3c374ab4b4" containerName="main"
Jul 20 16:08:36.096307 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:36.096277    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8791e0/sub-test-model-8791e0-kserve-56b67c844c-fsjvd"
Jul 20 16:08:36.099656 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:36.099632    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-8791e0\"/\"sub-test-model-8791e0-kserve-self-signed-certs\""
Jul 20 16:08:36.099781 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:36.099640    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-8791e0\"/\"openshift-service-ca.crt\""
Jul 20 16:08:36.099781 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:36.099681    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-8791e0\"/\"kube-root-ca.crt\""
Jul 20 16:08:36.099781 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:36.099732    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-8791e0\"/\"default-dockercfg-hpcc6\""
Jul 20 16:08:36.107082 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:36.107063    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8791e0/sub-test-model-8791e0-kserve-56b67c844c-fsjvd"]
Jul 20 16:08:36.118112 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:36.118089    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:08:36.199988 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:36.199945    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe-home\") pod \"sub-test-model-8791e0-kserve-56b67c844c-fsjvd\" (UID: \"04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe\") " pod="ai-tenant-e2e-shared-a-8791e0/sub-test-model-8791e0-kserve-56b67c844c-fsjvd"
Jul 20 16:08:36.200153 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:36.200010    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe-tls-certs\") pod \"sub-test-model-8791e0-kserve-56b67c844c-fsjvd\" (UID: \"04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe\") " pod="ai-tenant-e2e-shared-a-8791e0/sub-test-model-8791e0-kserve-56b67c844c-fsjvd"
Jul 20 16:08:36.200153 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:36.200097    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe-tmp-dir\") pod \"sub-test-model-8791e0-kserve-56b67c844c-fsjvd\" (UID: \"04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe\") " pod="ai-tenant-e2e-shared-a-8791e0/sub-test-model-8791e0-kserve-56b67c844c-fsjvd"
Jul 20 16:08:36.200153 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:36.200136    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe-model-cache\") pod \"sub-test-model-8791e0-kserve-56b67c844c-fsjvd\" (UID: \"04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe\") " pod="ai-tenant-e2e-shared-a-8791e0/sub-test-model-8791e0-kserve-56b67c844c-fsjvd"
Jul 20 16:08:36.200261 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:36.200171    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lqjg\" (UniqueName: \"kubernetes.io/projected/04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe-kube-api-access-5lqjg\") pod \"sub-test-model-8791e0-kserve-56b67c844c-fsjvd\" (UID: \"04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe\") " pod="ai-tenant-e2e-shared-a-8791e0/sub-test-model-8791e0-kserve-56b67c844c-fsjvd"
Jul 20 16:08:36.200261 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:36.200241    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe-dshm\") pod \"sub-test-model-8791e0-kserve-56b67c844c-fsjvd\" (UID: \"04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe\") " pod="ai-tenant-e2e-shared-a-8791e0/sub-test-model-8791e0-kserve-56b67c844c-fsjvd"
Jul 20 16:08:36.301399 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:36.301366    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe-tmp-dir\") pod \"sub-test-model-8791e0-kserve-56b67c844c-fsjvd\" (UID: \"04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe\") " pod="ai-tenant-e2e-shared-a-8791e0/sub-test-model-8791e0-kserve-56b67c844c-fsjvd"
Jul 20 16:08:36.301399 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:36.301404    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe-model-cache\") pod \"sub-test-model-8791e0-kserve-56b67c844c-fsjvd\" (UID: \"04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe\") " pod="ai-tenant-e2e-shared-a-8791e0/sub-test-model-8791e0-kserve-56b67c844c-fsjvd"
Jul 20 16:08:36.301595 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:36.301520    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lqjg\" (UniqueName: \"kubernetes.io/projected/04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe-kube-api-access-5lqjg\") pod \"sub-test-model-8791e0-kserve-56b67c844c-fsjvd\" (UID: \"04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe\") " pod="ai-tenant-e2e-shared-a-8791e0/sub-test-model-8791e0-kserve-56b67c844c-fsjvd"
Jul 20 16:08:36.301595 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:36.301583    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe-dshm\") pod \"sub-test-model-8791e0-kserve-56b67c844c-fsjvd\" (UID: \"04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe\") " pod="ai-tenant-e2e-shared-a-8791e0/sub-test-model-8791e0-kserve-56b67c844c-fsjvd"
Jul 20 16:08:36.301666 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:36.301617    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe-home\") pod \"sub-test-model-8791e0-kserve-56b67c844c-fsjvd\" (UID: \"04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe\") " pod="ai-tenant-e2e-shared-a-8791e0/sub-test-model-8791e0-kserve-56b67c844c-fsjvd"
Jul 20 16:08:36.301707 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:36.301663    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe-tls-certs\") pod \"sub-test-model-8791e0-kserve-56b67c844c-fsjvd\" (UID: \"04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe\") " pod="ai-tenant-e2e-shared-a-8791e0/sub-test-model-8791e0-kserve-56b67c844c-fsjvd"
Jul 20 16:08:36.301788 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:36.301771    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe-model-cache\") pod \"sub-test-model-8791e0-kserve-56b67c844c-fsjvd\" (UID: \"04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe\") " pod="ai-tenant-e2e-shared-a-8791e0/sub-test-model-8791e0-kserve-56b67c844c-fsjvd"
Jul 20 16:08:36.301844 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:36.301825    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe-tmp-dir\") pod \"sub-test-model-8791e0-kserve-56b67c844c-fsjvd\" (UID: \"04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe\") " pod="ai-tenant-e2e-shared-a-8791e0/sub-test-model-8791e0-kserve-56b67c844c-fsjvd"
Jul 20 16:08:36.301924 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:36.301911    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe-home\") pod \"sub-test-model-8791e0-kserve-56b67c844c-fsjvd\" (UID: \"04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe\") " pod="ai-tenant-e2e-shared-a-8791e0/sub-test-model-8791e0-kserve-56b67c844c-fsjvd"
Jul 20 16:08:36.303855 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:36.303830    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe-dshm\") pod \"sub-test-model-8791e0-kserve-56b67c844c-fsjvd\" (UID: \"04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe\") " pod="ai-tenant-e2e-shared-a-8791e0/sub-test-model-8791e0-kserve-56b67c844c-fsjvd"
Jul 20 16:08:36.304128 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:36.304107    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe-tls-certs\") pod \"sub-test-model-8791e0-kserve-56b67c844c-fsjvd\" (UID: \"04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe\") " pod="ai-tenant-e2e-shared-a-8791e0/sub-test-model-8791e0-kserve-56b67c844c-fsjvd"
Jul 20 16:08:36.310102 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:36.310071    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lqjg\" (UniqueName: \"kubernetes.io/projected/04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe-kube-api-access-5lqjg\") pod \"sub-test-model-8791e0-kserve-56b67c844c-fsjvd\" (UID: \"04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe\") " pod="ai-tenant-e2e-shared-a-8791e0/sub-test-model-8791e0-kserve-56b67c844c-fsjvd"
Jul 20 16:08:36.404605 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:36.404517    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8791e0/sub-test-model-8791e0-kserve-56b67c844c-fsjvd"
Jul 20 16:08:36.534048 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:36.534025    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8791e0/sub-test-model-8791e0-kserve-56b67c844c-fsjvd"]
Jul 20 16:08:36.535941 ip-10-0-130-19 kubenswrapper[2559]: W0720 16:08:36.535916    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod04ba3d0c_bf3c_4349_a1a1_5c7af907b9fe.slice/crio-84849fdc9c20e5d5ffd2b21e7d88e4f0bdbee098f2096d7343c87391d1e78d82 WatchSource:0}: Error finding container 84849fdc9c20e5d5ffd2b21e7d88e4f0bdbee098f2096d7343c87391d1e78d82: Status 404 returned error can't find the container with id 84849fdc9c20e5d5ffd2b21e7d88e4f0bdbee098f2096d7343c87391d1e78d82
Jul 20 16:08:36.612119 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:36.612081    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8791e0/sub-test-model-8791e0-kserve-56b67c844c-fsjvd" event={"ID":"04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe","Type":"ContainerStarted","Data":"90b18720da9d909a51961f896ce8f9d23f277afd54a157c925266c99b76c91bd"}
Jul 20 16:08:36.612271 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:36.612131    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8791e0/sub-test-model-8791e0-kserve-56b67c844c-fsjvd" event={"ID":"04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe","Type":"ContainerStarted","Data":"84849fdc9c20e5d5ffd2b21e7d88e4f0bdbee098f2096d7343c87391d1e78d82"}
Jul 20 16:08:36.629955 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:36.629912    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-8791e0/sub-test-model-8791e0-kserve-56b67c844c-fsjvd" podStartSLOduration=0.629896328 podStartE2EDuration="629.896328ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:08:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:08:36.628539138 +0000 UTC m=+2499.638160801" watchObservedRunningTime="2026-07-20 16:08:36.629896328 +0000 UTC m=+2499.639517992"
Jul 20 16:08:43.603772 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:43.603737    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:08:43.702021 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:43.701968    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:08:46.405051 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:46.405013    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-8791e0/sub-test-model-8791e0-kserve-56b67c844c-fsjvd"
Jul 20 16:08:46.405612 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:46.405061    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-8791e0/sub-test-model-8791e0-kserve-56b67c844c-fsjvd"
Jul 20 16:08:46.419983 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:46.419957    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-8791e0/sub-test-model-8791e0-kserve-56b67c844c-fsjvd"
Jul 20 16:08:46.668604 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:08:46.668526    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-8791e0/sub-test-model-8791e0-kserve-56b67c844c-fsjvd"
Jul 20 16:09:00.495543 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:00.495513    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-a667cd/sub-test-model-a667cd-kserve-846fcf65-c7vdb"]
Jul 20 16:09:00.500408 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:00.500388    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a667cd/sub-test-model-a667cd-kserve-846fcf65-c7vdb"
Jul 20 16:09:00.502897 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:00.502878    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a667cd\"/\"kube-root-ca.crt\""
Jul 20 16:09:00.503923 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:00.503907    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a667cd\"/\"sub-test-model-a667cd-kserve-self-signed-certs\""
Jul 20 16:09:00.504011 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:00.503939    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a667cd\"/\"openshift-service-ca.crt\""
Jul 20 16:09:00.504080 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:00.504033    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a667cd\"/\"default-dockercfg-pdcht\""
Jul 20 16:09:00.508786 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:00.508766    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a667cd/sub-test-model-a667cd-kserve-846fcf65-c7vdb"]
Jul 20 16:09:00.611305 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:00.611255    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/338f402e-abbb-46af-86d2-1d647b4b7676-tls-certs\") pod \"sub-test-model-a667cd-kserve-846fcf65-c7vdb\" (UID: \"338f402e-abbb-46af-86d2-1d647b4b7676\") " pod="ai-tenant-e2e-shared-b-a667cd/sub-test-model-a667cd-kserve-846fcf65-c7vdb"
Jul 20 16:09:00.611436 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:00.611308    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/338f402e-abbb-46af-86d2-1d647b4b7676-dshm\") pod \"sub-test-model-a667cd-kserve-846fcf65-c7vdb\" (UID: \"338f402e-abbb-46af-86d2-1d647b4b7676\") " pod="ai-tenant-e2e-shared-b-a667cd/sub-test-model-a667cd-kserve-846fcf65-c7vdb"
Jul 20 16:09:00.611436 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:00.611330    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zn4n\" (UniqueName: \"kubernetes.io/projected/338f402e-abbb-46af-86d2-1d647b4b7676-kube-api-access-4zn4n\") pod \"sub-test-model-a667cd-kserve-846fcf65-c7vdb\" (UID: \"338f402e-abbb-46af-86d2-1d647b4b7676\") " pod="ai-tenant-e2e-shared-b-a667cd/sub-test-model-a667cd-kserve-846fcf65-c7vdb"
Jul 20 16:09:00.611436 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:00.611378    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/338f402e-abbb-46af-86d2-1d647b4b7676-tmp-dir\") pod \"sub-test-model-a667cd-kserve-846fcf65-c7vdb\" (UID: \"338f402e-abbb-46af-86d2-1d647b4b7676\") " pod="ai-tenant-e2e-shared-b-a667cd/sub-test-model-a667cd-kserve-846fcf65-c7vdb"
Jul 20 16:09:00.611546 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:00.611463    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/338f402e-abbb-46af-86d2-1d647b4b7676-model-cache\") pod \"sub-test-model-a667cd-kserve-846fcf65-c7vdb\" (UID: \"338f402e-abbb-46af-86d2-1d647b4b7676\") " pod="ai-tenant-e2e-shared-b-a667cd/sub-test-model-a667cd-kserve-846fcf65-c7vdb"
Jul 20 16:09:00.611546 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:00.611532    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/338f402e-abbb-46af-86d2-1d647b4b7676-home\") pod \"sub-test-model-a667cd-kserve-846fcf65-c7vdb\" (UID: \"338f402e-abbb-46af-86d2-1d647b4b7676\") " pod="ai-tenant-e2e-shared-b-a667cd/sub-test-model-a667cd-kserve-846fcf65-c7vdb"
Jul 20 16:09:00.634245 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:00.634221    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:09:00.712092 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:00.712067    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/338f402e-abbb-46af-86d2-1d647b4b7676-model-cache\") pod \"sub-test-model-a667cd-kserve-846fcf65-c7vdb\" (UID: \"338f402e-abbb-46af-86d2-1d647b4b7676\") " pod="ai-tenant-e2e-shared-b-a667cd/sub-test-model-a667cd-kserve-846fcf65-c7vdb"
Jul 20 16:09:00.712202 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:00.712112    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/338f402e-abbb-46af-86d2-1d647b4b7676-home\") pod \"sub-test-model-a667cd-kserve-846fcf65-c7vdb\" (UID: \"338f402e-abbb-46af-86d2-1d647b4b7676\") " pod="ai-tenant-e2e-shared-b-a667cd/sub-test-model-a667cd-kserve-846fcf65-c7vdb"
Jul 20 16:09:00.712202 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:00.712138    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/338f402e-abbb-46af-86d2-1d647b4b7676-tls-certs\") pod \"sub-test-model-a667cd-kserve-846fcf65-c7vdb\" (UID: \"338f402e-abbb-46af-86d2-1d647b4b7676\") " pod="ai-tenant-e2e-shared-b-a667cd/sub-test-model-a667cd-kserve-846fcf65-c7vdb"
Jul 20 16:09:00.712202 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:00.712155    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/338f402e-abbb-46af-86d2-1d647b4b7676-dshm\") pod \"sub-test-model-a667cd-kserve-846fcf65-c7vdb\" (UID: \"338f402e-abbb-46af-86d2-1d647b4b7676\") " pod="ai-tenant-e2e-shared-b-a667cd/sub-test-model-a667cd-kserve-846fcf65-c7vdb"
Jul 20 16:09:00.712202 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:00.712172    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zn4n\" (UniqueName: \"kubernetes.io/projected/338f402e-abbb-46af-86d2-1d647b4b7676-kube-api-access-4zn4n\") pod \"sub-test-model-a667cd-kserve-846fcf65-c7vdb\" (UID: \"338f402e-abbb-46af-86d2-1d647b4b7676\") " pod="ai-tenant-e2e-shared-b-a667cd/sub-test-model-a667cd-kserve-846fcf65-c7vdb"
Jul 20 16:09:00.712202 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:00.712189    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/338f402e-abbb-46af-86d2-1d647b4b7676-tmp-dir\") pod \"sub-test-model-a667cd-kserve-846fcf65-c7vdb\" (UID: \"338f402e-abbb-46af-86d2-1d647b4b7676\") " pod="ai-tenant-e2e-shared-b-a667cd/sub-test-model-a667cd-kserve-846fcf65-c7vdb"
Jul 20 16:09:00.712523 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:00.712498    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/338f402e-abbb-46af-86d2-1d647b4b7676-model-cache\") pod \"sub-test-model-a667cd-kserve-846fcf65-c7vdb\" (UID: \"338f402e-abbb-46af-86d2-1d647b4b7676\") " pod="ai-tenant-e2e-shared-b-a667cd/sub-test-model-a667cd-kserve-846fcf65-c7vdb"
Jul 20 16:09:00.712603 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:00.712584    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/338f402e-abbb-46af-86d2-1d647b4b7676-home\") pod \"sub-test-model-a667cd-kserve-846fcf65-c7vdb\" (UID: \"338f402e-abbb-46af-86d2-1d647b4b7676\") " pod="ai-tenant-e2e-shared-b-a667cd/sub-test-model-a667cd-kserve-846fcf65-c7vdb"
Jul 20 16:09:00.712647 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:00.712611    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/338f402e-abbb-46af-86d2-1d647b4b7676-tmp-dir\") pod \"sub-test-model-a667cd-kserve-846fcf65-c7vdb\" (UID: \"338f402e-abbb-46af-86d2-1d647b4b7676\") " pod="ai-tenant-e2e-shared-b-a667cd/sub-test-model-a667cd-kserve-846fcf65-c7vdb"
Jul 20 16:09:00.714386 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:00.714365    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/338f402e-abbb-46af-86d2-1d647b4b7676-dshm\") pod \"sub-test-model-a667cd-kserve-846fcf65-c7vdb\" (UID: \"338f402e-abbb-46af-86d2-1d647b4b7676\") " pod="ai-tenant-e2e-shared-b-a667cd/sub-test-model-a667cd-kserve-846fcf65-c7vdb"
Jul 20 16:09:00.714531 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:00.714514    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/338f402e-abbb-46af-86d2-1d647b4b7676-tls-certs\") pod \"sub-test-model-a667cd-kserve-846fcf65-c7vdb\" (UID: \"338f402e-abbb-46af-86d2-1d647b4b7676\") " pod="ai-tenant-e2e-shared-b-a667cd/sub-test-model-a667cd-kserve-846fcf65-c7vdb"
Jul 20 16:09:00.721212 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:00.721187    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zn4n\" (UniqueName: \"kubernetes.io/projected/338f402e-abbb-46af-86d2-1d647b4b7676-kube-api-access-4zn4n\") pod \"sub-test-model-a667cd-kserve-846fcf65-c7vdb\" (UID: \"338f402e-abbb-46af-86d2-1d647b4b7676\") " pod="ai-tenant-e2e-shared-b-a667cd/sub-test-model-a667cd-kserve-846fcf65-c7vdb"
Jul 20 16:09:00.810250 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:00.810221    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a667cd/sub-test-model-a667cd-kserve-846fcf65-c7vdb"
Jul 20 16:09:00.933924 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:00.933897    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a667cd/sub-test-model-a667cd-kserve-846fcf65-c7vdb"]
Jul 20 16:09:00.935987 ip-10-0-130-19 kubenswrapper[2559]: W0720 16:09:00.935953    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod338f402e_abbb_46af_86d2_1d647b4b7676.slice/crio-6f68e133587a47340a23d1f3f9c7c02f1635e0846b1dd5ac6e205df0960df1fa WatchSource:0}: Error finding container 6f68e133587a47340a23d1f3f9c7c02f1635e0846b1dd5ac6e205df0960df1fa: Status 404 returned error can't find the container with id 6f68e133587a47340a23d1f3f9c7c02f1635e0846b1dd5ac6e205df0960df1fa
Jul 20 16:09:01.729643 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:01.729605    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a667cd/sub-test-model-a667cd-kserve-846fcf65-c7vdb" event={"ID":"338f402e-abbb-46af-86d2-1d647b4b7676","Type":"ContainerStarted","Data":"7d84e19f175a15bdf9c7c571cae0d32d98374aa761319c2891b3281f46d3153a"}
Jul 20 16:09:01.729996 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:01.729657    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a667cd/sub-test-model-a667cd-kserve-846fcf65-c7vdb" event={"ID":"338f402e-abbb-46af-86d2-1d647b4b7676","Type":"ContainerStarted","Data":"6f68e133587a47340a23d1f3f9c7c02f1635e0846b1dd5ac6e205df0960df1fa"}
Jul 20 16:09:01.749697 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:01.749650    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-a667cd/sub-test-model-a667cd-kserve-846fcf65-c7vdb" podStartSLOduration=1.749636631 podStartE2EDuration="1.749636631s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:09:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:09:01.748245548 +0000 UTC m=+2524.757867212" watchObservedRunningTime="2026-07-20 16:09:01.749636631 +0000 UTC m=+2524.759258292"
Jul 20 16:09:10.810946 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:10.810914    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-a667cd/sub-test-model-a667cd-kserve-846fcf65-c7vdb"
Jul 20 16:09:10.811369 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:10.810955    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-a667cd/sub-test-model-a667cd-kserve-846fcf65-c7vdb"
Jul 20 16:09:10.823583 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:10.823557    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-a667cd/sub-test-model-a667cd-kserve-846fcf65-c7vdb"
Jul 20 16:09:11.787782 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:11.787753    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-a667cd/sub-test-model-a667cd-kserve-846fcf65-c7vdb"
Jul 20 16:09:41.979472 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:41.979437    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8791e0/sub-test-model-8791e0-kserve-56b67c844c-fsjvd"]
Jul 20 16:09:42.005862 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:42.005835    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:09:42.918821 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:42.918790    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a667cd/sub-test-model-a667cd-kserve-846fcf65-c7vdb"]
Jul 20 16:09:42.930612 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:42.930577    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:09:48.909568 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:48.909530    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-8791e0/rate-test-model-8791e0-kserve-67b445dcf5-zdppp"]
Jul 20 16:09:48.913511 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:48.913490    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8791e0/rate-test-model-8791e0-kserve-67b445dcf5-zdppp"
Jul 20 16:09:48.916455 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:48.916435    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-8791e0\"/\"rate-test-model-8791e0-kserve-self-signed-certs\""
Jul 20 16:09:48.933636 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:48.933611    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:09:48.934150 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:48.934130    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8791e0/rate-test-model-8791e0-kserve-67b445dcf5-zdppp"]
Jul 20 16:09:49.041878 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:49.041845    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-559rx\" (UniqueName: \"kubernetes.io/projected/67daef6d-d1ac-410b-a315-a2d8fdb91200-kube-api-access-559rx\") pod \"rate-test-model-8791e0-kserve-67b445dcf5-zdppp\" (UID: \"67daef6d-d1ac-410b-a315-a2d8fdb91200\") " pod="ai-tenant-e2e-shared-a-8791e0/rate-test-model-8791e0-kserve-67b445dcf5-zdppp"
Jul 20 16:09:49.042034 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:49.041899    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67daef6d-d1ac-410b-a315-a2d8fdb91200-tls-certs\") pod \"rate-test-model-8791e0-kserve-67b445dcf5-zdppp\" (UID: \"67daef6d-d1ac-410b-a315-a2d8fdb91200\") " pod="ai-tenant-e2e-shared-a-8791e0/rate-test-model-8791e0-kserve-67b445dcf5-zdppp"
Jul 20 16:09:49.042034 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:49.041997    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67daef6d-d1ac-410b-a315-a2d8fdb91200-tmp-dir\") pod \"rate-test-model-8791e0-kserve-67b445dcf5-zdppp\" (UID: \"67daef6d-d1ac-410b-a315-a2d8fdb91200\") " pod="ai-tenant-e2e-shared-a-8791e0/rate-test-model-8791e0-kserve-67b445dcf5-zdppp"
Jul 20 16:09:49.042132 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:49.042100    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67daef6d-d1ac-410b-a315-a2d8fdb91200-model-cache\") pod \"rate-test-model-8791e0-kserve-67b445dcf5-zdppp\" (UID: \"67daef6d-d1ac-410b-a315-a2d8fdb91200\") " pod="ai-tenant-e2e-shared-a-8791e0/rate-test-model-8791e0-kserve-67b445dcf5-zdppp"
Jul 20 16:09:49.042179 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:49.042148    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67daef6d-d1ac-410b-a315-a2d8fdb91200-dshm\") pod \"rate-test-model-8791e0-kserve-67b445dcf5-zdppp\" (UID: \"67daef6d-d1ac-410b-a315-a2d8fdb91200\") " pod="ai-tenant-e2e-shared-a-8791e0/rate-test-model-8791e0-kserve-67b445dcf5-zdppp"
Jul 20 16:09:49.042307 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:49.042265    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67daef6d-d1ac-410b-a315-a2d8fdb91200-home\") pod \"rate-test-model-8791e0-kserve-67b445dcf5-zdppp\" (UID: \"67daef6d-d1ac-410b-a315-a2d8fdb91200\") " pod="ai-tenant-e2e-shared-a-8791e0/rate-test-model-8791e0-kserve-67b445dcf5-zdppp"
Jul 20 16:09:49.143190 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:49.143156    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67daef6d-d1ac-410b-a315-a2d8fdb91200-tmp-dir\") pod \"rate-test-model-8791e0-kserve-67b445dcf5-zdppp\" (UID: \"67daef6d-d1ac-410b-a315-a2d8fdb91200\") " pod="ai-tenant-e2e-shared-a-8791e0/rate-test-model-8791e0-kserve-67b445dcf5-zdppp"
Jul 20 16:09:49.143382 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:49.143203    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67daef6d-d1ac-410b-a315-a2d8fdb91200-model-cache\") pod \"rate-test-model-8791e0-kserve-67b445dcf5-zdppp\" (UID: \"67daef6d-d1ac-410b-a315-a2d8fdb91200\") " pod="ai-tenant-e2e-shared-a-8791e0/rate-test-model-8791e0-kserve-67b445dcf5-zdppp"
Jul 20 16:09:49.143382 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:49.143244    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67daef6d-d1ac-410b-a315-a2d8fdb91200-dshm\") pod \"rate-test-model-8791e0-kserve-67b445dcf5-zdppp\" (UID: \"67daef6d-d1ac-410b-a315-a2d8fdb91200\") " pod="ai-tenant-e2e-shared-a-8791e0/rate-test-model-8791e0-kserve-67b445dcf5-zdppp"
Jul 20 16:09:49.143382 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:49.143309    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67daef6d-d1ac-410b-a315-a2d8fdb91200-home\") pod \"rate-test-model-8791e0-kserve-67b445dcf5-zdppp\" (UID: \"67daef6d-d1ac-410b-a315-a2d8fdb91200\") " pod="ai-tenant-e2e-shared-a-8791e0/rate-test-model-8791e0-kserve-67b445dcf5-zdppp"
Jul 20 16:09:49.143382 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:49.143361    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-559rx\" (UniqueName: \"kubernetes.io/projected/67daef6d-d1ac-410b-a315-a2d8fdb91200-kube-api-access-559rx\") pod \"rate-test-model-8791e0-kserve-67b445dcf5-zdppp\" (UID: \"67daef6d-d1ac-410b-a315-a2d8fdb91200\") " pod="ai-tenant-e2e-shared-a-8791e0/rate-test-model-8791e0-kserve-67b445dcf5-zdppp"
Jul 20 16:09:49.143607 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:49.143407    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67daef6d-d1ac-410b-a315-a2d8fdb91200-tls-certs\") pod \"rate-test-model-8791e0-kserve-67b445dcf5-zdppp\" (UID: \"67daef6d-d1ac-410b-a315-a2d8fdb91200\") " pod="ai-tenant-e2e-shared-a-8791e0/rate-test-model-8791e0-kserve-67b445dcf5-zdppp"
Jul 20 16:09:49.143607 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:49.143587    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67daef6d-d1ac-410b-a315-a2d8fdb91200-tmp-dir\") pod \"rate-test-model-8791e0-kserve-67b445dcf5-zdppp\" (UID: \"67daef6d-d1ac-410b-a315-a2d8fdb91200\") " pod="ai-tenant-e2e-shared-a-8791e0/rate-test-model-8791e0-kserve-67b445dcf5-zdppp"
Jul 20 16:09:49.143718 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:49.143631    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67daef6d-d1ac-410b-a315-a2d8fdb91200-model-cache\") pod \"rate-test-model-8791e0-kserve-67b445dcf5-zdppp\" (UID: \"67daef6d-d1ac-410b-a315-a2d8fdb91200\") " pod="ai-tenant-e2e-shared-a-8791e0/rate-test-model-8791e0-kserve-67b445dcf5-zdppp"
Jul 20 16:09:49.143718 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:49.143701    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67daef6d-d1ac-410b-a315-a2d8fdb91200-home\") pod \"rate-test-model-8791e0-kserve-67b445dcf5-zdppp\" (UID: \"67daef6d-d1ac-410b-a315-a2d8fdb91200\") " pod="ai-tenant-e2e-shared-a-8791e0/rate-test-model-8791e0-kserve-67b445dcf5-zdppp"
Jul 20 16:09:49.145436 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:49.145415    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67daef6d-d1ac-410b-a315-a2d8fdb91200-dshm\") pod \"rate-test-model-8791e0-kserve-67b445dcf5-zdppp\" (UID: \"67daef6d-d1ac-410b-a315-a2d8fdb91200\") " pod="ai-tenant-e2e-shared-a-8791e0/rate-test-model-8791e0-kserve-67b445dcf5-zdppp"
Jul 20 16:09:49.145711 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:49.145693    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67daef6d-d1ac-410b-a315-a2d8fdb91200-tls-certs\") pod \"rate-test-model-8791e0-kserve-67b445dcf5-zdppp\" (UID: \"67daef6d-d1ac-410b-a315-a2d8fdb91200\") " pod="ai-tenant-e2e-shared-a-8791e0/rate-test-model-8791e0-kserve-67b445dcf5-zdppp"
Jul 20 16:09:49.153985 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:49.153967    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-559rx\" (UniqueName: \"kubernetes.io/projected/67daef6d-d1ac-410b-a315-a2d8fdb91200-kube-api-access-559rx\") pod \"rate-test-model-8791e0-kserve-67b445dcf5-zdppp\" (UID: \"67daef6d-d1ac-410b-a315-a2d8fdb91200\") " pod="ai-tenant-e2e-shared-a-8791e0/rate-test-model-8791e0-kserve-67b445dcf5-zdppp"
Jul 20 16:09:49.223876 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:49.223821    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8791e0/rate-test-model-8791e0-kserve-67b445dcf5-zdppp"
Jul 20 16:09:49.361518 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:49.361485    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8791e0/rate-test-model-8791e0-kserve-67b445dcf5-zdppp"]
Jul 20 16:09:49.365606 ip-10-0-130-19 kubenswrapper[2559]: W0720 16:09:49.365579    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod67daef6d_d1ac_410b_a315_a2d8fdb91200.slice/crio-8d0b69ad914cea1718e08996342b24c2a899fcb4fa9c3be4bbbb7c1ed4ccfbe7 WatchSource:0}: Error finding container 8d0b69ad914cea1718e08996342b24c2a899fcb4fa9c3be4bbbb7c1ed4ccfbe7: Status 404 returned error can't find the container with id 8d0b69ad914cea1718e08996342b24c2a899fcb4fa9c3be4bbbb7c1ed4ccfbe7
Jul 20 16:09:49.957826 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:49.957787    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8791e0/rate-test-model-8791e0-kserve-67b445dcf5-zdppp" event={"ID":"67daef6d-d1ac-410b-a315-a2d8fdb91200","Type":"ContainerStarted","Data":"00973ada2975c17e621aede9522b019143ce75fdeb25f88b4f0132ce4698f48d"}
Jul 20 16:09:49.957826 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:49.957826    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8791e0/rate-test-model-8791e0-kserve-67b445dcf5-zdppp" event={"ID":"67daef6d-d1ac-410b-a315-a2d8fdb91200","Type":"ContainerStarted","Data":"8d0b69ad914cea1718e08996342b24c2a899fcb4fa9c3be4bbbb7c1ed4ccfbe7"}
Jul 20 16:09:49.977127 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:49.977078    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-8791e0/rate-test-model-8791e0-kserve-67b445dcf5-zdppp" podStartSLOduration=1.977064492 podStartE2EDuration="1.977064492s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:09:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:09:49.975829104 +0000 UTC m=+2572.985450767" watchObservedRunningTime="2026-07-20 16:09:49.977064492 +0000 UTC m=+2572.986686154"
Jul 20 16:09:56.999267 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:56.999204    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-8791e0/sub-test-model-8791e0-kserve-56b67c844c-fsjvd" podUID="04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe" containerName="main" containerID="cri-o://90b18720da9d909a51961f896ce8f9d23f277afd54a157c925266c99b76c91bd" gracePeriod=45
Jul 20 16:09:57.243042 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:57.243019    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8791e0/sub-test-model-8791e0-kserve-56b67c844c-fsjvd"
Jul 20 16:09:57.418426 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:57.418403    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe-tmp-dir\") pod \"04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe\" (UID: \"04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe\") "
Jul 20 16:09:57.418611 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:57.418437    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe-tls-certs\") pod \"04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe\" (UID: \"04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe\") "
Jul 20 16:09:57.418611 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:57.418459    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe-home\") pod \"04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe\" (UID: \"04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe\") "
Jul 20 16:09:57.418611 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:57.418483    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe-model-cache\") pod \"04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe\" (UID: \"04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe\") "
Jul 20 16:09:57.418611 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:57.418525    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe-dshm\") pod \"04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe\" (UID: \"04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe\") "
Jul 20 16:09:57.418611 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:57.418576    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5lqjg\" (UniqueName: \"kubernetes.io/projected/04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe-kube-api-access-5lqjg\") pod \"04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe\" (UID: \"04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe\") "
Jul 20 16:09:57.418897 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:57.418712    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe" (UID: "04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:09:57.418897 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:57.418742    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe-home" (OuterVolumeSpecName: "home") pod "04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe" (UID: "04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:09:57.418897 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:57.418833    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe-model-cache" (OuterVolumeSpecName: "model-cache") pod "04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe" (UID: "04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:09:57.418897 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:57.418883    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe-tmp-dir\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:09:57.419027 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:57.418901    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe-home\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:09:57.420510 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:57.420484    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe" (UID: "04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:09:57.420651 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:57.420632    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe-dshm" (OuterVolumeSpecName: "dshm") pod "04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe" (UID: "04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:09:57.420713 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:57.420635    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe-kube-api-access-5lqjg" (OuterVolumeSpecName: "kube-api-access-5lqjg") pod "04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe" (UID: "04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe"). InnerVolumeSpecName "kube-api-access-5lqjg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:09:57.519392 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:57.519363    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe-tls-certs\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:09:57.519392 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:57.519387    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe-model-cache\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:09:57.519392 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:57.519396    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe-dshm\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:09:57.519646 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:57.519404    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lqjg\" (UniqueName: \"kubernetes.io/projected/04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe-kube-api-access-5lqjg\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:09:57.941549 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:57.941498    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-a667cd/sub-test-model-a667cd-kserve-846fcf65-c7vdb" podUID="338f402e-abbb-46af-86d2-1d647b4b7676" containerName="main" containerID="cri-o://7d84e19f175a15bdf9c7c571cae0d32d98374aa761319c2891b3281f46d3153a" gracePeriod=45
Jul 20 16:09:58.002701 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:58.002674    2559 generic.go:356] "Generic (PLEG): container finished" podID="04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe" containerID="90b18720da9d909a51961f896ce8f9d23f277afd54a157c925266c99b76c91bd" exitCode=0
Jul 20 16:09:58.003000 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:58.002710    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8791e0/sub-test-model-8791e0-kserve-56b67c844c-fsjvd"
Jul 20 16:09:58.003000 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:58.002757    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8791e0/sub-test-model-8791e0-kserve-56b67c844c-fsjvd" event={"ID":"04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe","Type":"ContainerDied","Data":"90b18720da9d909a51961f896ce8f9d23f277afd54a157c925266c99b76c91bd"}
Jul 20 16:09:58.003000 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:58.002795    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8791e0/sub-test-model-8791e0-kserve-56b67c844c-fsjvd" event={"ID":"04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe","Type":"ContainerDied","Data":"84849fdc9c20e5d5ffd2b21e7d88e4f0bdbee098f2096d7343c87391d1e78d82"}
Jul 20 16:09:58.003000 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:58.002815    2559 scope.go:117] "RemoveContainer" containerID="90b18720da9d909a51961f896ce8f9d23f277afd54a157c925266c99b76c91bd"
Jul 20 16:09:58.023101 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:58.023077    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8791e0/sub-test-model-8791e0-kserve-56b67c844c-fsjvd"]
Jul 20 16:09:58.026669 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:58.026647    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-8791e0/sub-test-model-8791e0-kserve-56b67c844c-fsjvd"]
Jul 20 16:09:58.052476 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:58.052414    2559 scope.go:117] "RemoveContainer" containerID="90b18720da9d909a51961f896ce8f9d23f277afd54a157c925266c99b76c91bd"
Jul 20 16:09:58.052698 ip-10-0-130-19 kubenswrapper[2559]: E0720 16:09:58.052672    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90b18720da9d909a51961f896ce8f9d23f277afd54a157c925266c99b76c91bd\": container with ID starting with 90b18720da9d909a51961f896ce8f9d23f277afd54a157c925266c99b76c91bd not found: ID does not exist" containerID="90b18720da9d909a51961f896ce8f9d23f277afd54a157c925266c99b76c91bd"
Jul 20 16:09:58.052781 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:58.052712    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90b18720da9d909a51961f896ce8f9d23f277afd54a157c925266c99b76c91bd"} err="failed to get container status \"90b18720da9d909a51961f896ce8f9d23f277afd54a157c925266c99b76c91bd\": rpc error: code = NotFound desc = could not find container \"90b18720da9d909a51961f896ce8f9d23f277afd54a157c925266c99b76c91bd\": container with ID starting with 90b18720da9d909a51961f896ce8f9d23f277afd54a157c925266c99b76c91bd not found: ID does not exist"
Jul 20 16:09:58.188207 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:58.188186    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a667cd/sub-test-model-a667cd-kserve-846fcf65-c7vdb"
Jul 20 16:09:58.326061 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:58.326037    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/338f402e-abbb-46af-86d2-1d647b4b7676-model-cache\") pod \"338f402e-abbb-46af-86d2-1d647b4b7676\" (UID: \"338f402e-abbb-46af-86d2-1d647b4b7676\") "
Jul 20 16:09:58.326222 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:58.326095    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/338f402e-abbb-46af-86d2-1d647b4b7676-tls-certs\") pod \"338f402e-abbb-46af-86d2-1d647b4b7676\" (UID: \"338f402e-abbb-46af-86d2-1d647b4b7676\") "
Jul 20 16:09:58.326222 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:58.326151    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/338f402e-abbb-46af-86d2-1d647b4b7676-dshm\") pod \"338f402e-abbb-46af-86d2-1d647b4b7676\" (UID: \"338f402e-abbb-46af-86d2-1d647b4b7676\") "
Jul 20 16:09:58.326222 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:58.326165    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/338f402e-abbb-46af-86d2-1d647b4b7676-home\") pod \"338f402e-abbb-46af-86d2-1d647b4b7676\" (UID: \"338f402e-abbb-46af-86d2-1d647b4b7676\") "
Jul 20 16:09:58.326222 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:58.326192    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/338f402e-abbb-46af-86d2-1d647b4b7676-tmp-dir\") pod \"338f402e-abbb-46af-86d2-1d647b4b7676\" (UID: \"338f402e-abbb-46af-86d2-1d647b4b7676\") "
Jul 20 16:09:58.326451 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:58.326228    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4zn4n\" (UniqueName: \"kubernetes.io/projected/338f402e-abbb-46af-86d2-1d647b4b7676-kube-api-access-4zn4n\") pod \"338f402e-abbb-46af-86d2-1d647b4b7676\" (UID: \"338f402e-abbb-46af-86d2-1d647b4b7676\") "
Jul 20 16:09:58.326451 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:58.326347    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/338f402e-abbb-46af-86d2-1d647b4b7676-model-cache" (OuterVolumeSpecName: "model-cache") pod "338f402e-abbb-46af-86d2-1d647b4b7676" (UID: "338f402e-abbb-46af-86d2-1d647b4b7676"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:09:58.326560 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:58.326441    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/338f402e-abbb-46af-86d2-1d647b4b7676-home" (OuterVolumeSpecName: "home") pod "338f402e-abbb-46af-86d2-1d647b4b7676" (UID: "338f402e-abbb-46af-86d2-1d647b4b7676"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:09:58.326560 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:58.326492    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/338f402e-abbb-46af-86d2-1d647b4b7676-model-cache\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:09:58.326560 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:58.326527    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/338f402e-abbb-46af-86d2-1d647b4b7676-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "338f402e-abbb-46af-86d2-1d647b4b7676" (UID: "338f402e-abbb-46af-86d2-1d647b4b7676"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:09:58.328307 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:58.328260    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/338f402e-abbb-46af-86d2-1d647b4b7676-dshm" (OuterVolumeSpecName: "dshm") pod "338f402e-abbb-46af-86d2-1d647b4b7676" (UID: "338f402e-abbb-46af-86d2-1d647b4b7676"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:09:58.328307 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:58.328270    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/338f402e-abbb-46af-86d2-1d647b4b7676-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "338f402e-abbb-46af-86d2-1d647b4b7676" (UID: "338f402e-abbb-46af-86d2-1d647b4b7676"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:09:58.328484 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:58.328459    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/338f402e-abbb-46af-86d2-1d647b4b7676-kube-api-access-4zn4n" (OuterVolumeSpecName: "kube-api-access-4zn4n") pod "338f402e-abbb-46af-86d2-1d647b4b7676" (UID: "338f402e-abbb-46af-86d2-1d647b4b7676"). InnerVolumeSpecName "kube-api-access-4zn4n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:09:58.427332 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:58.427271    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4zn4n\" (UniqueName: \"kubernetes.io/projected/338f402e-abbb-46af-86d2-1d647b4b7676-kube-api-access-4zn4n\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:09:58.427332 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:58.427326    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/338f402e-abbb-46af-86d2-1d647b4b7676-tls-certs\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:09:58.427523 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:58.427344    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/338f402e-abbb-46af-86d2-1d647b4b7676-dshm\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:09:58.427523 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:58.427357    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/338f402e-abbb-46af-86d2-1d647b4b7676-home\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:09:58.427523 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:58.427369    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/338f402e-abbb-46af-86d2-1d647b4b7676-tmp-dir\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:09:59.008906 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:59.008873    2559 generic.go:356] "Generic (PLEG): container finished" podID="338f402e-abbb-46af-86d2-1d647b4b7676" containerID="7d84e19f175a15bdf9c7c571cae0d32d98374aa761319c2891b3281f46d3153a" exitCode=0
Jul 20 16:09:59.009378 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:59.008925    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a667cd/sub-test-model-a667cd-kserve-846fcf65-c7vdb"
Jul 20 16:09:59.009378 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:59.008961    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a667cd/sub-test-model-a667cd-kserve-846fcf65-c7vdb" event={"ID":"338f402e-abbb-46af-86d2-1d647b4b7676","Type":"ContainerDied","Data":"7d84e19f175a15bdf9c7c571cae0d32d98374aa761319c2891b3281f46d3153a"}
Jul 20 16:09:59.009378 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:59.009000    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a667cd/sub-test-model-a667cd-kserve-846fcf65-c7vdb" event={"ID":"338f402e-abbb-46af-86d2-1d647b4b7676","Type":"ContainerDied","Data":"6f68e133587a47340a23d1f3f9c7c02f1635e0846b1dd5ac6e205df0960df1fa"}
Jul 20 16:09:59.009378 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:59.009024    2559 scope.go:117] "RemoveContainer" containerID="7d84e19f175a15bdf9c7c571cae0d32d98374aa761319c2891b3281f46d3153a"
Jul 20 16:09:59.019713 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:59.019697    2559 scope.go:117] "RemoveContainer" containerID="7d84e19f175a15bdf9c7c571cae0d32d98374aa761319c2891b3281f46d3153a"
Jul 20 16:09:59.019956 ip-10-0-130-19 kubenswrapper[2559]: E0720 16:09:59.019937    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d84e19f175a15bdf9c7c571cae0d32d98374aa761319c2891b3281f46d3153a\": container with ID starting with 7d84e19f175a15bdf9c7c571cae0d32d98374aa761319c2891b3281f46d3153a not found: ID does not exist" containerID="7d84e19f175a15bdf9c7c571cae0d32d98374aa761319c2891b3281f46d3153a"
Jul 20 16:09:59.019999 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:59.019964    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d84e19f175a15bdf9c7c571cae0d32d98374aa761319c2891b3281f46d3153a"} err="failed to get container status \"7d84e19f175a15bdf9c7c571cae0d32d98374aa761319c2891b3281f46d3153a\": rpc error: code = NotFound desc = could not find container \"7d84e19f175a15bdf9c7c571cae0d32d98374aa761319c2891b3281f46d3153a\": container with ID starting with 7d84e19f175a15bdf9c7c571cae0d32d98374aa761319c2891b3281f46d3153a not found: ID does not exist"
Jul 20 16:09:59.032281 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:59.032258    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a667cd/sub-test-model-a667cd-kserve-846fcf65-c7vdb"]
Jul 20 16:09:59.038317 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:59.038275    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-a667cd/sub-test-model-a667cd-kserve-846fcf65-c7vdb"]
Jul 20 16:09:59.224768 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:59.224739    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-8791e0/rate-test-model-8791e0-kserve-67b445dcf5-zdppp"
Jul 20 16:09:59.224874 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:59.224799    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-8791e0/rate-test-model-8791e0-kserve-67b445dcf5-zdppp"
Jul 20 16:09:59.237661 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:59.237640    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-8791e0/rate-test-model-8791e0-kserve-67b445dcf5-zdppp"
Jul 20 16:09:59.514909 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:59.514877    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe" path="/var/lib/kubelet/pods/04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe/volumes"
Jul 20 16:09:59.515298 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:09:59.515271    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="338f402e-abbb-46af-86d2-1d647b4b7676" path="/var/lib/kubelet/pods/338f402e-abbb-46af-86d2-1d647b4b7676/volumes"
Jul 20 16:10:00.028086 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:00.028061    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-8791e0/rate-test-model-8791e0-kserve-67b445dcf5-zdppp"
Jul 20 16:10:07.201159 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:07.201118    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-a667cd/rate-test-model-a667cd-kserve-6f87557c59-clcc4"]
Jul 20 16:10:07.201671 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:07.201531    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="338f402e-abbb-46af-86d2-1d647b4b7676" containerName="main"
Jul 20 16:10:07.201671 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:07.201544    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="338f402e-abbb-46af-86d2-1d647b4b7676" containerName="main"
Jul 20 16:10:07.201671 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:07.201566    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe" containerName="main"
Jul 20 16:10:07.201671 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:07.201574    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe" containerName="main"
Jul 20 16:10:07.201671 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:07.201646    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="338f402e-abbb-46af-86d2-1d647b4b7676" containerName="main"
Jul 20 16:10:07.201671 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:07.201658    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="04ba3d0c-bf3c-4349-a1a1-5c7af907b9fe" containerName="main"
Jul 20 16:10:07.204558 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:07.204536    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a667cd/rate-test-model-a667cd-kserve-6f87557c59-clcc4"
Jul 20 16:10:07.211599 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:07.211563    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a667cd\"/\"kube-root-ca.crt\""
Jul 20 16:10:07.211599 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:07.211591    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a667cd\"/\"default-dockercfg-pdcht\""
Jul 20 16:10:07.211756 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:07.211612    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a667cd\"/\"rate-test-model-a667cd-kserve-self-signed-certs\""
Jul 20 16:10:07.211756 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:07.211742    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a667cd\"/\"openshift-service-ca.crt\""
Jul 20 16:10:07.226200 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:07.226180    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a667cd/rate-test-model-a667cd-kserve-6f87557c59-clcc4"]
Jul 20 16:10:07.266068 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:07.266047    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:10:07.297031 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:07.297012    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b23c78a1-86a6-4375-aff9-beb6259bc9b6-dshm\") pod \"rate-test-model-a667cd-kserve-6f87557c59-clcc4\" (UID: \"b23c78a1-86a6-4375-aff9-beb6259bc9b6\") " pod="ai-tenant-e2e-shared-b-a667cd/rate-test-model-a667cd-kserve-6f87557c59-clcc4"
Jul 20 16:10:07.297121 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:07.297048    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b23c78a1-86a6-4375-aff9-beb6259bc9b6-home\") pod \"rate-test-model-a667cd-kserve-6f87557c59-clcc4\" (UID: \"b23c78a1-86a6-4375-aff9-beb6259bc9b6\") " pod="ai-tenant-e2e-shared-b-a667cd/rate-test-model-a667cd-kserve-6f87557c59-clcc4"
Jul 20 16:10:07.297121 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:07.297067    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b23c78a1-86a6-4375-aff9-beb6259bc9b6-model-cache\") pod \"rate-test-model-a667cd-kserve-6f87557c59-clcc4\" (UID: \"b23c78a1-86a6-4375-aff9-beb6259bc9b6\") " pod="ai-tenant-e2e-shared-b-a667cd/rate-test-model-a667cd-kserve-6f87557c59-clcc4"
Jul 20 16:10:07.297121 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:07.297083    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b23c78a1-86a6-4375-aff9-beb6259bc9b6-tmp-dir\") pod \"rate-test-model-a667cd-kserve-6f87557c59-clcc4\" (UID: \"b23c78a1-86a6-4375-aff9-beb6259bc9b6\") " pod="ai-tenant-e2e-shared-b-a667cd/rate-test-model-a667cd-kserve-6f87557c59-clcc4"
Jul 20 16:10:07.297226 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:07.297148    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkqn6\" (UniqueName: \"kubernetes.io/projected/b23c78a1-86a6-4375-aff9-beb6259bc9b6-kube-api-access-wkqn6\") pod \"rate-test-model-a667cd-kserve-6f87557c59-clcc4\" (UID: \"b23c78a1-86a6-4375-aff9-beb6259bc9b6\") " pod="ai-tenant-e2e-shared-b-a667cd/rate-test-model-a667cd-kserve-6f87557c59-clcc4"
Jul 20 16:10:07.297226 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:07.297198    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b23c78a1-86a6-4375-aff9-beb6259bc9b6-tls-certs\") pod \"rate-test-model-a667cd-kserve-6f87557c59-clcc4\" (UID: \"b23c78a1-86a6-4375-aff9-beb6259bc9b6\") " pod="ai-tenant-e2e-shared-b-a667cd/rate-test-model-a667cd-kserve-6f87557c59-clcc4"
Jul 20 16:10:07.398488 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:07.398463    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b23c78a1-86a6-4375-aff9-beb6259bc9b6-dshm\") pod \"rate-test-model-a667cd-kserve-6f87557c59-clcc4\" (UID: \"b23c78a1-86a6-4375-aff9-beb6259bc9b6\") " pod="ai-tenant-e2e-shared-b-a667cd/rate-test-model-a667cd-kserve-6f87557c59-clcc4"
Jul 20 16:10:07.398594 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:07.398504    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b23c78a1-86a6-4375-aff9-beb6259bc9b6-home\") pod \"rate-test-model-a667cd-kserve-6f87557c59-clcc4\" (UID: \"b23c78a1-86a6-4375-aff9-beb6259bc9b6\") " pod="ai-tenant-e2e-shared-b-a667cd/rate-test-model-a667cd-kserve-6f87557c59-clcc4"
Jul 20 16:10:07.398594 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:07.398520    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b23c78a1-86a6-4375-aff9-beb6259bc9b6-model-cache\") pod \"rate-test-model-a667cd-kserve-6f87557c59-clcc4\" (UID: \"b23c78a1-86a6-4375-aff9-beb6259bc9b6\") " pod="ai-tenant-e2e-shared-b-a667cd/rate-test-model-a667cd-kserve-6f87557c59-clcc4"
Jul 20 16:10:07.398594 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:07.398536    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b23c78a1-86a6-4375-aff9-beb6259bc9b6-tmp-dir\") pod \"rate-test-model-a667cd-kserve-6f87557c59-clcc4\" (UID: \"b23c78a1-86a6-4375-aff9-beb6259bc9b6\") " pod="ai-tenant-e2e-shared-b-a667cd/rate-test-model-a667cd-kserve-6f87557c59-clcc4"
Jul 20 16:10:07.398594 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:07.398563    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkqn6\" (UniqueName: \"kubernetes.io/projected/b23c78a1-86a6-4375-aff9-beb6259bc9b6-kube-api-access-wkqn6\") pod \"rate-test-model-a667cd-kserve-6f87557c59-clcc4\" (UID: \"b23c78a1-86a6-4375-aff9-beb6259bc9b6\") " pod="ai-tenant-e2e-shared-b-a667cd/rate-test-model-a667cd-kserve-6f87557c59-clcc4"
Jul 20 16:10:07.398779 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:07.398604    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b23c78a1-86a6-4375-aff9-beb6259bc9b6-tls-certs\") pod \"rate-test-model-a667cd-kserve-6f87557c59-clcc4\" (UID: \"b23c78a1-86a6-4375-aff9-beb6259bc9b6\") " pod="ai-tenant-e2e-shared-b-a667cd/rate-test-model-a667cd-kserve-6f87557c59-clcc4"
Jul 20 16:10:07.398966 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:07.398950    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b23c78a1-86a6-4375-aff9-beb6259bc9b6-home\") pod \"rate-test-model-a667cd-kserve-6f87557c59-clcc4\" (UID: \"b23c78a1-86a6-4375-aff9-beb6259bc9b6\") " pod="ai-tenant-e2e-shared-b-a667cd/rate-test-model-a667cd-kserve-6f87557c59-clcc4"
Jul 20 16:10:07.399023 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:07.398994    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b23c78a1-86a6-4375-aff9-beb6259bc9b6-model-cache\") pod \"rate-test-model-a667cd-kserve-6f87557c59-clcc4\" (UID: \"b23c78a1-86a6-4375-aff9-beb6259bc9b6\") " pod="ai-tenant-e2e-shared-b-a667cd/rate-test-model-a667cd-kserve-6f87557c59-clcc4"
Jul 20 16:10:07.399058 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:07.399027    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b23c78a1-86a6-4375-aff9-beb6259bc9b6-tmp-dir\") pod \"rate-test-model-a667cd-kserve-6f87557c59-clcc4\" (UID: \"b23c78a1-86a6-4375-aff9-beb6259bc9b6\") " pod="ai-tenant-e2e-shared-b-a667cd/rate-test-model-a667cd-kserve-6f87557c59-clcc4"
Jul 20 16:10:07.400778 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:07.400760    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b23c78a1-86a6-4375-aff9-beb6259bc9b6-dshm\") pod \"rate-test-model-a667cd-kserve-6f87557c59-clcc4\" (UID: \"b23c78a1-86a6-4375-aff9-beb6259bc9b6\") " pod="ai-tenant-e2e-shared-b-a667cd/rate-test-model-a667cd-kserve-6f87557c59-clcc4"
Jul 20 16:10:07.401057 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:07.401035    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b23c78a1-86a6-4375-aff9-beb6259bc9b6-tls-certs\") pod \"rate-test-model-a667cd-kserve-6f87557c59-clcc4\" (UID: \"b23c78a1-86a6-4375-aff9-beb6259bc9b6\") " pod="ai-tenant-e2e-shared-b-a667cd/rate-test-model-a667cd-kserve-6f87557c59-clcc4"
Jul 20 16:10:07.408794 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:07.408772    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkqn6\" (UniqueName: \"kubernetes.io/projected/b23c78a1-86a6-4375-aff9-beb6259bc9b6-kube-api-access-wkqn6\") pod \"rate-test-model-a667cd-kserve-6f87557c59-clcc4\" (UID: \"b23c78a1-86a6-4375-aff9-beb6259bc9b6\") " pod="ai-tenant-e2e-shared-b-a667cd/rate-test-model-a667cd-kserve-6f87557c59-clcc4"
Jul 20 16:10:07.514023 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:07.514003    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a667cd/rate-test-model-a667cd-kserve-6f87557c59-clcc4"
Jul 20 16:10:07.641989 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:07.641968    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a667cd/rate-test-model-a667cd-kserve-6f87557c59-clcc4"]
Jul 20 16:10:07.644145 ip-10-0-130-19 kubenswrapper[2559]: W0720 16:10:07.644114    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb23c78a1_86a6_4375_aff9_beb6259bc9b6.slice/crio-2ccbc7f6d7b5f18a6b76d28c206ffe79baeb6550a5d4bffa1b7a4218688585f2 WatchSource:0}: Error finding container 2ccbc7f6d7b5f18a6b76d28c206ffe79baeb6550a5d4bffa1b7a4218688585f2: Status 404 returned error can't find the container with id 2ccbc7f6d7b5f18a6b76d28c206ffe79baeb6550a5d4bffa1b7a4218688585f2
Jul 20 16:10:08.056673 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:08.056633    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a667cd/rate-test-model-a667cd-kserve-6f87557c59-clcc4" event={"ID":"b23c78a1-86a6-4375-aff9-beb6259bc9b6","Type":"ContainerStarted","Data":"b7af1d9dfa350c9dec485410f67fca0895ff71eb67ed90d6825e0c8146be12fd"}
Jul 20 16:10:08.056865 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:08.056685    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a667cd/rate-test-model-a667cd-kserve-6f87557c59-clcc4" event={"ID":"b23c78a1-86a6-4375-aff9-beb6259bc9b6","Type":"ContainerStarted","Data":"2ccbc7f6d7b5f18a6b76d28c206ffe79baeb6550a5d4bffa1b7a4218688585f2"}
Jul 20 16:10:08.075100 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:08.075050    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-a667cd/rate-test-model-a667cd-kserve-6f87557c59-clcc4" podStartSLOduration=1.075033862 podStartE2EDuration="1.075033862s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:10:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:10:08.07362996 +0000 UTC m=+2591.083251633" watchObservedRunningTime="2026-07-20 16:10:08.075033862 +0000 UTC m=+2591.084655525"
Jul 20 16:10:09.581534 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:09.581494    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:10:09.681082 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:09.681037    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:10:17.515484 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:17.515449    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-a667cd/rate-test-model-a667cd-kserve-6f87557c59-clcc4"
Jul 20 16:10:17.515484 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:17.515491    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-a667cd/rate-test-model-a667cd-kserve-6f87557c59-clcc4"
Jul 20 16:10:17.527049 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:17.527022    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-a667cd/rate-test-model-a667cd-kserve-6f87557c59-clcc4"
Jul 20 16:10:18.114731 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:18.114704    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-a667cd/rate-test-model-a667cd-kserve-6f87557c59-clcc4"
Jul 20 16:10:30.935963 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:30.935916    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:10:31.226054 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:31.225947    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:10:38.491493 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:38.491446    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:10:38.586718 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:38.586672    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:10:39.469400 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:39.469370    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:10:39.570623 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:39.570587    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:10:46.973151 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:46.973119    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8791e0/rate-test-model-8791e0-kserve-67b445dcf5-zdppp"]
Jul 20 16:10:46.992581 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:46.992552    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:10:47.810601 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:47.810569    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a667cd/rate-test-model-a667cd-kserve-6f87557c59-clcc4"]
Jul 20 16:10:47.834691 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:10:47.834663    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:11:01.993047 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:01.992984    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-8791e0/rate-test-model-8791e0-kserve-67b445dcf5-zdppp" podUID="67daef6d-d1ac-410b-a315-a2d8fdb91200" containerName="main" containerID="cri-o://00973ada2975c17e621aede9522b019143ce75fdeb25f88b4f0132ce4698f48d" gracePeriod=45
Jul 20 16:11:02.240338 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:02.240314    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8791e0/rate-test-model-8791e0-kserve-67b445dcf5-zdppp"
Jul 20 16:11:02.276535 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:02.276515    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67daef6d-d1ac-410b-a315-a2d8fdb91200-tmp-dir\") pod \"67daef6d-d1ac-410b-a315-a2d8fdb91200\" (UID: \"67daef6d-d1ac-410b-a315-a2d8fdb91200\") "
Jul 20 16:11:02.276670 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:02.276544    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67daef6d-d1ac-410b-a315-a2d8fdb91200-home\") pod \"67daef6d-d1ac-410b-a315-a2d8fdb91200\" (UID: \"67daef6d-d1ac-410b-a315-a2d8fdb91200\") "
Jul 20 16:11:02.276670 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:02.276569    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-559rx\" (UniqueName: \"kubernetes.io/projected/67daef6d-d1ac-410b-a315-a2d8fdb91200-kube-api-access-559rx\") pod \"67daef6d-d1ac-410b-a315-a2d8fdb91200\" (UID: \"67daef6d-d1ac-410b-a315-a2d8fdb91200\") "
Jul 20 16:11:02.276670 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:02.276592    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67daef6d-d1ac-410b-a315-a2d8fdb91200-dshm\") pod \"67daef6d-d1ac-410b-a315-a2d8fdb91200\" (UID: \"67daef6d-d1ac-410b-a315-a2d8fdb91200\") "
Jul 20 16:11:02.276670 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:02.276632    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67daef6d-d1ac-410b-a315-a2d8fdb91200-tls-certs\") pod \"67daef6d-d1ac-410b-a315-a2d8fdb91200\" (UID: \"67daef6d-d1ac-410b-a315-a2d8fdb91200\") "
Jul 20 16:11:02.276879 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:02.276672    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67daef6d-d1ac-410b-a315-a2d8fdb91200-model-cache\") pod \"67daef6d-d1ac-410b-a315-a2d8fdb91200\" (UID: \"67daef6d-d1ac-410b-a315-a2d8fdb91200\") "
Jul 20 16:11:02.276879 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:02.276843    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67daef6d-d1ac-410b-a315-a2d8fdb91200-home" (OuterVolumeSpecName: "home") pod "67daef6d-d1ac-410b-a315-a2d8fdb91200" (UID: "67daef6d-d1ac-410b-a315-a2d8fdb91200"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:11:02.276879 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:02.276857    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67daef6d-d1ac-410b-a315-a2d8fdb91200-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "67daef6d-d1ac-410b-a315-a2d8fdb91200" (UID: "67daef6d-d1ac-410b-a315-a2d8fdb91200"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:11:02.277095 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:02.276970    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67daef6d-d1ac-410b-a315-a2d8fdb91200-tmp-dir\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:11:02.277095 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:02.276988    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67daef6d-d1ac-410b-a315-a2d8fdb91200-home\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:11:02.277095 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:02.277055    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67daef6d-d1ac-410b-a315-a2d8fdb91200-model-cache" (OuterVolumeSpecName: "model-cache") pod "67daef6d-d1ac-410b-a315-a2d8fdb91200" (UID: "67daef6d-d1ac-410b-a315-a2d8fdb91200"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:11:02.278693 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:02.278672    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67daef6d-d1ac-410b-a315-a2d8fdb91200-dshm" (OuterVolumeSpecName: "dshm") pod "67daef6d-d1ac-410b-a315-a2d8fdb91200" (UID: "67daef6d-d1ac-410b-a315-a2d8fdb91200"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:11:02.278792 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:02.278752    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67daef6d-d1ac-410b-a315-a2d8fdb91200-kube-api-access-559rx" (OuterVolumeSpecName: "kube-api-access-559rx") pod "67daef6d-d1ac-410b-a315-a2d8fdb91200" (UID: "67daef6d-d1ac-410b-a315-a2d8fdb91200"). InnerVolumeSpecName "kube-api-access-559rx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:11:02.278838 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:02.278788    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67daef6d-d1ac-410b-a315-a2d8fdb91200-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "67daef6d-d1ac-410b-a315-a2d8fdb91200" (UID: "67daef6d-d1ac-410b-a315-a2d8fdb91200"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:11:02.322255 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:02.322229    2559 generic.go:356] "Generic (PLEG): container finished" podID="67daef6d-d1ac-410b-a315-a2d8fdb91200" containerID="00973ada2975c17e621aede9522b019143ce75fdeb25f88b4f0132ce4698f48d" exitCode=0
Jul 20 16:11:02.322387 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:02.322271    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8791e0/rate-test-model-8791e0-kserve-67b445dcf5-zdppp"
Jul 20 16:11:02.322387 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:02.322328    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8791e0/rate-test-model-8791e0-kserve-67b445dcf5-zdppp" event={"ID":"67daef6d-d1ac-410b-a315-a2d8fdb91200","Type":"ContainerDied","Data":"00973ada2975c17e621aede9522b019143ce75fdeb25f88b4f0132ce4698f48d"}
Jul 20 16:11:02.322387 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:02.322378    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8791e0/rate-test-model-8791e0-kserve-67b445dcf5-zdppp" event={"ID":"67daef6d-d1ac-410b-a315-a2d8fdb91200","Type":"ContainerDied","Data":"8d0b69ad914cea1718e08996342b24c2a899fcb4fa9c3be4bbbb7c1ed4ccfbe7"}
Jul 20 16:11:02.322559 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:02.322400    2559 scope.go:117] "RemoveContainer" containerID="00973ada2975c17e621aede9522b019143ce75fdeb25f88b4f0132ce4698f48d"
Jul 20 16:11:02.330783 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:02.330769    2559 scope.go:117] "RemoveContainer" containerID="00973ada2975c17e621aede9522b019143ce75fdeb25f88b4f0132ce4698f48d"
Jul 20 16:11:02.331019 ip-10-0-130-19 kubenswrapper[2559]: E0720 16:11:02.330998    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00973ada2975c17e621aede9522b019143ce75fdeb25f88b4f0132ce4698f48d\": container with ID starting with 00973ada2975c17e621aede9522b019143ce75fdeb25f88b4f0132ce4698f48d not found: ID does not exist" containerID="00973ada2975c17e621aede9522b019143ce75fdeb25f88b4f0132ce4698f48d"
Jul 20 16:11:02.331080 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:02.331026    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00973ada2975c17e621aede9522b019143ce75fdeb25f88b4f0132ce4698f48d"} err="failed to get container status \"00973ada2975c17e621aede9522b019143ce75fdeb25f88b4f0132ce4698f48d\": rpc error: code = NotFound desc = could not find container \"00973ada2975c17e621aede9522b019143ce75fdeb25f88b4f0132ce4698f48d\": container with ID starting with 00973ada2975c17e621aede9522b019143ce75fdeb25f88b4f0132ce4698f48d not found: ID does not exist"
Jul 20 16:11:02.347618 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:02.347593    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8791e0/rate-test-model-8791e0-kserve-67b445dcf5-zdppp"]
Jul 20 16:11:02.352665 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:02.352646    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-8791e0/rate-test-model-8791e0-kserve-67b445dcf5-zdppp"]
Jul 20 16:11:02.377707 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:02.377684    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-559rx\" (UniqueName: \"kubernetes.io/projected/67daef6d-d1ac-410b-a315-a2d8fdb91200-kube-api-access-559rx\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:11:02.377707 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:02.377704    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67daef6d-d1ac-410b-a315-a2d8fdb91200-dshm\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:11:02.377825 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:02.377714    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67daef6d-d1ac-410b-a315-a2d8fdb91200-tls-certs\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:11:02.377825 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:02.377724    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67daef6d-d1ac-410b-a315-a2d8fdb91200-model-cache\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:11:02.829984 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:02.829945    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-a667cd/rate-test-model-a667cd-kserve-6f87557c59-clcc4" podUID="b23c78a1-86a6-4375-aff9-beb6259bc9b6" containerName="main" containerID="cri-o://b7af1d9dfa350c9dec485410f67fca0895ff71eb67ed90d6825e0c8146be12fd" gracePeriod=45
Jul 20 16:11:03.081524 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:03.081463    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a667cd/rate-test-model-a667cd-kserve-6f87557c59-clcc4"
Jul 20 16:11:03.182789 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:03.182759    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b23c78a1-86a6-4375-aff9-beb6259bc9b6-tmp-dir\") pod \"b23c78a1-86a6-4375-aff9-beb6259bc9b6\" (UID: \"b23c78a1-86a6-4375-aff9-beb6259bc9b6\") "
Jul 20 16:11:03.182943 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:03.182812    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b23c78a1-86a6-4375-aff9-beb6259bc9b6-tls-certs\") pod \"b23c78a1-86a6-4375-aff9-beb6259bc9b6\" (UID: \"b23c78a1-86a6-4375-aff9-beb6259bc9b6\") "
Jul 20 16:11:03.182943 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:03.182839    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b23c78a1-86a6-4375-aff9-beb6259bc9b6-home\") pod \"b23c78a1-86a6-4375-aff9-beb6259bc9b6\" (UID: \"b23c78a1-86a6-4375-aff9-beb6259bc9b6\") "
Jul 20 16:11:03.182943 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:03.182890    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b23c78a1-86a6-4375-aff9-beb6259bc9b6-model-cache\") pod \"b23c78a1-86a6-4375-aff9-beb6259bc9b6\" (UID: \"b23c78a1-86a6-4375-aff9-beb6259bc9b6\") "
Jul 20 16:11:03.182943 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:03.182914    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wkqn6\" (UniqueName: \"kubernetes.io/projected/b23c78a1-86a6-4375-aff9-beb6259bc9b6-kube-api-access-wkqn6\") pod \"b23c78a1-86a6-4375-aff9-beb6259bc9b6\" (UID: \"b23c78a1-86a6-4375-aff9-beb6259bc9b6\") "
Jul 20 16:11:03.183133 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:03.182952    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b23c78a1-86a6-4375-aff9-beb6259bc9b6-dshm\") pod \"b23c78a1-86a6-4375-aff9-beb6259bc9b6\" (UID: \"b23c78a1-86a6-4375-aff9-beb6259bc9b6\") "
Jul 20 16:11:03.183133 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:03.183051    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b23c78a1-86a6-4375-aff9-beb6259bc9b6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b23c78a1-86a6-4375-aff9-beb6259bc9b6" (UID: "b23c78a1-86a6-4375-aff9-beb6259bc9b6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:11:03.183223 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:03.183161    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b23c78a1-86a6-4375-aff9-beb6259bc9b6-home" (OuterVolumeSpecName: "home") pod "b23c78a1-86a6-4375-aff9-beb6259bc9b6" (UID: "b23c78a1-86a6-4375-aff9-beb6259bc9b6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:11:03.183223 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:03.183168    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b23c78a1-86a6-4375-aff9-beb6259bc9b6-model-cache" (OuterVolumeSpecName: "model-cache") pod "b23c78a1-86a6-4375-aff9-beb6259bc9b6" (UID: "b23c78a1-86a6-4375-aff9-beb6259bc9b6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:11:03.183337 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:03.183300    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b23c78a1-86a6-4375-aff9-beb6259bc9b6-model-cache\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:11:03.183337 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:03.183317    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b23c78a1-86a6-4375-aff9-beb6259bc9b6-tmp-dir\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:11:03.183337 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:03.183326    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b23c78a1-86a6-4375-aff9-beb6259bc9b6-home\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:11:03.185021 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:03.184988    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b23c78a1-86a6-4375-aff9-beb6259bc9b6-dshm" (OuterVolumeSpecName: "dshm") pod "b23c78a1-86a6-4375-aff9-beb6259bc9b6" (UID: "b23c78a1-86a6-4375-aff9-beb6259bc9b6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:11:03.185138 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:03.185049    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b23c78a1-86a6-4375-aff9-beb6259bc9b6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b23c78a1-86a6-4375-aff9-beb6259bc9b6" (UID: "b23c78a1-86a6-4375-aff9-beb6259bc9b6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:11:03.185138 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:03.185070    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b23c78a1-86a6-4375-aff9-beb6259bc9b6-kube-api-access-wkqn6" (OuterVolumeSpecName: "kube-api-access-wkqn6") pod "b23c78a1-86a6-4375-aff9-beb6259bc9b6" (UID: "b23c78a1-86a6-4375-aff9-beb6259bc9b6"). InnerVolumeSpecName "kube-api-access-wkqn6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:11:03.284670 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:03.284642    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b23c78a1-86a6-4375-aff9-beb6259bc9b6-tls-certs\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:11:03.284670 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:03.284665    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkqn6\" (UniqueName: \"kubernetes.io/projected/b23c78a1-86a6-4375-aff9-beb6259bc9b6-kube-api-access-wkqn6\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:11:03.284670 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:03.284673    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b23c78a1-86a6-4375-aff9-beb6259bc9b6-dshm\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:11:03.327994 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:03.327963    2559 generic.go:356] "Generic (PLEG): container finished" podID="b23c78a1-86a6-4375-aff9-beb6259bc9b6" containerID="b7af1d9dfa350c9dec485410f67fca0895ff71eb67ed90d6825e0c8146be12fd" exitCode=0
Jul 20 16:11:03.328116 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:03.328009    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a667cd/rate-test-model-a667cd-kserve-6f87557c59-clcc4"
Jul 20 16:11:03.328116 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:03.328048    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a667cd/rate-test-model-a667cd-kserve-6f87557c59-clcc4" event={"ID":"b23c78a1-86a6-4375-aff9-beb6259bc9b6","Type":"ContainerDied","Data":"b7af1d9dfa350c9dec485410f67fca0895ff71eb67ed90d6825e0c8146be12fd"}
Jul 20 16:11:03.328116 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:03.328092    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a667cd/rate-test-model-a667cd-kserve-6f87557c59-clcc4" event={"ID":"b23c78a1-86a6-4375-aff9-beb6259bc9b6","Type":"ContainerDied","Data":"2ccbc7f6d7b5f18a6b76d28c206ffe79baeb6550a5d4bffa1b7a4218688585f2"}
Jul 20 16:11:03.328116 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:03.328111    2559 scope.go:117] "RemoveContainer" containerID="b7af1d9dfa350c9dec485410f67fca0895ff71eb67ed90d6825e0c8146be12fd"
Jul 20 16:11:03.336662 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:03.336648    2559 scope.go:117] "RemoveContainer" containerID="b7af1d9dfa350c9dec485410f67fca0895ff71eb67ed90d6825e0c8146be12fd"
Jul 20 16:11:03.336890 ip-10-0-130-19 kubenswrapper[2559]: E0720 16:11:03.336872    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7af1d9dfa350c9dec485410f67fca0895ff71eb67ed90d6825e0c8146be12fd\": container with ID starting with b7af1d9dfa350c9dec485410f67fca0895ff71eb67ed90d6825e0c8146be12fd not found: ID does not exist" containerID="b7af1d9dfa350c9dec485410f67fca0895ff71eb67ed90d6825e0c8146be12fd"
Jul 20 16:11:03.336969 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:03.336899    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7af1d9dfa350c9dec485410f67fca0895ff71eb67ed90d6825e0c8146be12fd"} err="failed to get container status \"b7af1d9dfa350c9dec485410f67fca0895ff71eb67ed90d6825e0c8146be12fd\": rpc error: code = NotFound desc = could not find container \"b7af1d9dfa350c9dec485410f67fca0895ff71eb67ed90d6825e0c8146be12fd\": container with ID starting with b7af1d9dfa350c9dec485410f67fca0895ff71eb67ed90d6825e0c8146be12fd not found: ID does not exist"
Jul 20 16:11:03.350549 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:03.350527    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a667cd/rate-test-model-a667cd-kserve-6f87557c59-clcc4"]
Jul 20 16:11:03.352686 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:03.352666    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-a667cd/rate-test-model-a667cd-kserve-6f87557c59-clcc4"]
Jul 20 16:11:03.515100 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:03.515067    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67daef6d-d1ac-410b-a315-a2d8fdb91200" path="/var/lib/kubelet/pods/67daef6d-d1ac-410b-a315-a2d8fdb91200/volumes"
Jul 20 16:11:03.515523 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:03.515509    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b23c78a1-86a6-4375-aff9-beb6259bc9b6" path="/var/lib/kubelet/pods/b23c78a1-86a6-4375-aff9-beb6259bc9b6/volumes"
Jul 20 16:11:07.801410 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:07.801361    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:11:07.987937 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:07.987907    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:11:17.692437 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:17.692404    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8791e0-79f5f6b586-nw7np"]
Jul 20 16:11:17.692903 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:17.692637    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8791e0-79f5f6b586-nw7np" podUID="97e35eb7-883a-4695-8a50-4325aa14f9b1" containerName="maas-api" containerID="cri-o://48b1f0a7ff8126ac3851ecad99ad0eb7afdebd6685654e38519290d79a8c3a9b" gracePeriod=30
Jul 20 16:11:17.733408 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:17.733369    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:11:17.939770 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:17.939745    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8791e0-79f5f6b586-nw7np"
Jul 20 16:11:18.009551 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:18.009484    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m8qcr\" (UniqueName: \"kubernetes.io/projected/97e35eb7-883a-4695-8a50-4325aa14f9b1-kube-api-access-m8qcr\") pod \"97e35eb7-883a-4695-8a50-4325aa14f9b1\" (UID: \"97e35eb7-883a-4695-8a50-4325aa14f9b1\") "
Jul 20 16:11:18.009671 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:18.009570    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97e35eb7-883a-4695-8a50-4325aa14f9b1-openshift-service-ca\") pod \"97e35eb7-883a-4695-8a50-4325aa14f9b1\" (UID: \"97e35eb7-883a-4695-8a50-4325aa14f9b1\") "
Jul 20 16:11:18.009671 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:18.009633    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97e35eb7-883a-4695-8a50-4325aa14f9b1-maas-api-tls\") pod \"97e35eb7-883a-4695-8a50-4325aa14f9b1\" (UID: \"97e35eb7-883a-4695-8a50-4325aa14f9b1\") "
Jul 20 16:11:18.009912 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:18.009883    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97e35eb7-883a-4695-8a50-4325aa14f9b1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "97e35eb7-883a-4695-8a50-4325aa14f9b1" (UID: "97e35eb7-883a-4695-8a50-4325aa14f9b1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:11:18.011558 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:18.011538    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97e35eb7-883a-4695-8a50-4325aa14f9b1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "97e35eb7-883a-4695-8a50-4325aa14f9b1" (UID: "97e35eb7-883a-4695-8a50-4325aa14f9b1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:11:18.011644 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:18.011585    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97e35eb7-883a-4695-8a50-4325aa14f9b1-kube-api-access-m8qcr" (OuterVolumeSpecName: "kube-api-access-m8qcr") pod "97e35eb7-883a-4695-8a50-4325aa14f9b1" (UID: "97e35eb7-883a-4695-8a50-4325aa14f9b1"). InnerVolumeSpecName "kube-api-access-m8qcr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:11:18.110491 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:18.110464    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97e35eb7-883a-4695-8a50-4325aa14f9b1-openshift-service-ca\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:11:18.110491 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:18.110494    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97e35eb7-883a-4695-8a50-4325aa14f9b1-maas-api-tls\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:11:18.110699 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:18.110509    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m8qcr\" (UniqueName: \"kubernetes.io/projected/97e35eb7-883a-4695-8a50-4325aa14f9b1-kube-api-access-m8qcr\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:11:18.399480 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:18.399451    2559 generic.go:356] "Generic (PLEG): container finished" podID="97e35eb7-883a-4695-8a50-4325aa14f9b1" containerID="48b1f0a7ff8126ac3851ecad99ad0eb7afdebd6685654e38519290d79a8c3a9b" exitCode=0
Jul 20 16:11:18.399631 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:18.399485    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8791e0-79f5f6b586-nw7np" event={"ID":"97e35eb7-883a-4695-8a50-4325aa14f9b1","Type":"ContainerDied","Data":"48b1f0a7ff8126ac3851ecad99ad0eb7afdebd6685654e38519290d79a8c3a9b"}
Jul 20 16:11:18.399631 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:18.399492    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8791e0-79f5f6b586-nw7np"
Jul 20 16:11:18.399631 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:18.399527    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8791e0-79f5f6b586-nw7np" event={"ID":"97e35eb7-883a-4695-8a50-4325aa14f9b1","Type":"ContainerDied","Data":"8d3f8469691d813b761d2162995e1c367378c4553dd2c96c8e0cbf1770234bb9"}
Jul 20 16:11:18.399631 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:18.399547    2559 scope.go:117] "RemoveContainer" containerID="48b1f0a7ff8126ac3851ecad99ad0eb7afdebd6685654e38519290d79a8c3a9b"
Jul 20 16:11:18.408147 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:18.408132    2559 scope.go:117] "RemoveContainer" containerID="48b1f0a7ff8126ac3851ecad99ad0eb7afdebd6685654e38519290d79a8c3a9b"
Jul 20 16:11:18.408395 ip-10-0-130-19 kubenswrapper[2559]: E0720 16:11:18.408379    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48b1f0a7ff8126ac3851ecad99ad0eb7afdebd6685654e38519290d79a8c3a9b\": container with ID starting with 48b1f0a7ff8126ac3851ecad99ad0eb7afdebd6685654e38519290d79a8c3a9b not found: ID does not exist" containerID="48b1f0a7ff8126ac3851ecad99ad0eb7afdebd6685654e38519290d79a8c3a9b"
Jul 20 16:11:18.408451 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:18.408403    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48b1f0a7ff8126ac3851ecad99ad0eb7afdebd6685654e38519290d79a8c3a9b"} err="failed to get container status \"48b1f0a7ff8126ac3851ecad99ad0eb7afdebd6685654e38519290d79a8c3a9b\": rpc error: code = NotFound desc = could not find container \"48b1f0a7ff8126ac3851ecad99ad0eb7afdebd6685654e38519290d79a8c3a9b\": container with ID starting with 48b1f0a7ff8126ac3851ecad99ad0eb7afdebd6685654e38519290d79a8c3a9b not found: ID does not exist"
Jul 20 16:11:18.442463 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:18.442440    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8791e0-79f5f6b586-nw7np"]
Jul 20 16:11:18.445650 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:18.445630    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8791e0-79f5f6b586-nw7np"]
Jul 20 16:11:19.514689 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:19.514656    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97e35eb7-883a-4695-8a50-4325aa14f9b1" path="/var/lib/kubelet/pods/97e35eb7-883a-4695-8a50-4325aa14f9b1/volumes"
Jul 20 16:11:22.033975 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:22.033942    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:11:22.446024 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:22.445944    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:11:29.901560 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:29.901526    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:11:29.983830 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:29.983802    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:11:30.201242 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:30.201158    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:11:30.429087 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:30.429051    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:11:31.291412 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:31.291376    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:11:31.390815 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:31.390783    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:11:41.223159 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:41.223123    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-a667cd-7fd8b4b696-t2zrd"]
Jul 20 16:11:41.223627 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:41.223438    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a667cd-7fd8b4b696-t2zrd" podUID="b084a8cb-2f9b-4e2d-8e48-a5e84981c169" containerName="maas-api" containerID="cri-o://3db1d746bc99fc029e2b288f9093ef7d1f86b800b746b1ab4c63212280588bbb" gracePeriod=30
Jul 20 16:11:41.257850 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:41.257822    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:11:41.465388 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:41.465367    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a667cd-7fd8b4b696-t2zrd"
Jul 20 16:11:41.507266 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:41.507194    2559 generic.go:356] "Generic (PLEG): container finished" podID="b084a8cb-2f9b-4e2d-8e48-a5e84981c169" containerID="3db1d746bc99fc029e2b288f9093ef7d1f86b800b746b1ab4c63212280588bbb" exitCode=0
Jul 20 16:11:41.507266 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:41.507243    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a667cd-7fd8b4b696-t2zrd"
Jul 20 16:11:41.507421 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:41.507278    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a667cd-7fd8b4b696-t2zrd" event={"ID":"b084a8cb-2f9b-4e2d-8e48-a5e84981c169","Type":"ContainerDied","Data":"3db1d746bc99fc029e2b288f9093ef7d1f86b800b746b1ab4c63212280588bbb"}
Jul 20 16:11:41.507421 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:41.507344    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a667cd-7fd8b4b696-t2zrd" event={"ID":"b084a8cb-2f9b-4e2d-8e48-a5e84981c169","Type":"ContainerDied","Data":"f613ff9cce95b9438d7087841678f3c7ffb6ee16fa6bb2ac35628c2fe973620c"}
Jul 20 16:11:41.507421 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:41.507373    2559 scope.go:117] "RemoveContainer" containerID="3db1d746bc99fc029e2b288f9093ef7d1f86b800b746b1ab4c63212280588bbb"
Jul 20 16:11:41.518631 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:41.518612    2559 scope.go:117] "RemoveContainer" containerID="3db1d746bc99fc029e2b288f9093ef7d1f86b800b746b1ab4c63212280588bbb"
Jul 20 16:11:41.518881 ip-10-0-130-19 kubenswrapper[2559]: E0720 16:11:41.518859    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3db1d746bc99fc029e2b288f9093ef7d1f86b800b746b1ab4c63212280588bbb\": container with ID starting with 3db1d746bc99fc029e2b288f9093ef7d1f86b800b746b1ab4c63212280588bbb not found: ID does not exist" containerID="3db1d746bc99fc029e2b288f9093ef7d1f86b800b746b1ab4c63212280588bbb"
Jul 20 16:11:41.518942 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:41.518892    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3db1d746bc99fc029e2b288f9093ef7d1f86b800b746b1ab4c63212280588bbb"} err="failed to get container status \"3db1d746bc99fc029e2b288f9093ef7d1f86b800b746b1ab4c63212280588bbb\": rpc error: code = NotFound desc = could not find container \"3db1d746bc99fc029e2b288f9093ef7d1f86b800b746b1ab4c63212280588bbb\": container with ID starting with 3db1d746bc99fc029e2b288f9093ef7d1f86b800b746b1ab4c63212280588bbb not found: ID does not exist"
Jul 20 16:11:41.615417 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:41.615385    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zq7hr\" (UniqueName: \"kubernetes.io/projected/b084a8cb-2f9b-4e2d-8e48-a5e84981c169-kube-api-access-zq7hr\") pod \"b084a8cb-2f9b-4e2d-8e48-a5e84981c169\" (UID: \"b084a8cb-2f9b-4e2d-8e48-a5e84981c169\") "
Jul 20 16:11:41.615569 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:41.615437    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b084a8cb-2f9b-4e2d-8e48-a5e84981c169-openshift-service-ca\") pod \"b084a8cb-2f9b-4e2d-8e48-a5e84981c169\" (UID: \"b084a8cb-2f9b-4e2d-8e48-a5e84981c169\") "
Jul 20 16:11:41.615569 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:41.615468    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b084a8cb-2f9b-4e2d-8e48-a5e84981c169-maas-api-tls\") pod \"b084a8cb-2f9b-4e2d-8e48-a5e84981c169\" (UID: \"b084a8cb-2f9b-4e2d-8e48-a5e84981c169\") "
Jul 20 16:11:41.615796 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:41.615771    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b084a8cb-2f9b-4e2d-8e48-a5e84981c169-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b084a8cb-2f9b-4e2d-8e48-a5e84981c169" (UID: "b084a8cb-2f9b-4e2d-8e48-a5e84981c169"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:11:41.617457 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:41.617434    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b084a8cb-2f9b-4e2d-8e48-a5e84981c169-kube-api-access-zq7hr" (OuterVolumeSpecName: "kube-api-access-zq7hr") pod "b084a8cb-2f9b-4e2d-8e48-a5e84981c169" (UID: "b084a8cb-2f9b-4e2d-8e48-a5e84981c169"). InnerVolumeSpecName "kube-api-access-zq7hr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:11:41.617535 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:41.617486    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b084a8cb-2f9b-4e2d-8e48-a5e84981c169-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b084a8cb-2f9b-4e2d-8e48-a5e84981c169" (UID: "b084a8cb-2f9b-4e2d-8e48-a5e84981c169"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:11:41.717005 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:41.716981    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b084a8cb-2f9b-4e2d-8e48-a5e84981c169-openshift-service-ca\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:11:41.717005 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:41.717002    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b084a8cb-2f9b-4e2d-8e48-a5e84981c169-maas-api-tls\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:11:41.717138 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:41.717011    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zq7hr\" (UniqueName: \"kubernetes.io/projected/b084a8cb-2f9b-4e2d-8e48-a5e84981c169-kube-api-access-zq7hr\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:11:41.828729 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:41.828705    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-a667cd-7fd8b4b696-t2zrd"]
Jul 20 16:11:41.833903 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:41.833882    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-a667cd-7fd8b4b696-t2zrd"]
Jul 20 16:11:43.514734 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:43.514705    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b084a8cb-2f9b-4e2d-8e48-a5e84981c169" path="/var/lib/kubelet/pods/b084a8cb-2f9b-4e2d-8e48-a5e84981c169/volumes"
Jul 20 16:11:45.671414 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:45.671385    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:11:46.144810 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:46.144778    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:11:46.534263 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:46.534202    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:11:52.347536 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:52.347460    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:11:52.452686 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:52.452652    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:11:52.600413 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:52.600343    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"]
Jul 20 16:11:52.600575 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:52.600555    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d" podUID="07d5a0a6-90f0-4301-86d2-4bc2326a35f6" containerName="istio-proxy" containerID="cri-o://3c626a0effa88f501faeda82431b87223751d4341fbed454874828eb7900d893" gracePeriod=30
Jul 20 16:11:52.647690 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:52.647655    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:11:52.857488 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:52.857382    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:11:53.324199 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:53.324165    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-959fcf8ff-7fq7c_ad3725af-f98b-40ef-a66f-8949d3e21236/authorino/0.log"
Jul 20 16:11:57.769968 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:57.769942    2559 scope.go:117] "RemoveContainer" containerID="1a07221b8432adac8496971cb11392f56d2d95a1b8d3ab43cb312e923c328ecc"
Jul 20 16:11:57.854301 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:57.854259    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"
Jul 20 16:11:57.934102 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:57.934034    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-istiod-ca-cert\") pod \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\" (UID: \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\") "
Jul 20 16:11:57.934102 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:57.934069    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9ft56\" (UniqueName: \"kubernetes.io/projected/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-kube-api-access-9ft56\") pod \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\" (UID: \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\") "
Jul 20 16:11:57.934102 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:57.934091    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-istio-token\") pod \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\" (UID: \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\") "
Jul 20 16:11:57.934355 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:57.934108    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-istio-data\") pod \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\" (UID: \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\") "
Jul 20 16:11:57.934355 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:57.934141    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-workload-certs\") pod \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\" (UID: \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\") "
Jul 20 16:11:57.934355 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:57.934166    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-istio-podinfo\") pod \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\" (UID: \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\") "
Jul 20 16:11:57.934355 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:57.934227    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-workload-socket\") pod \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\" (UID: \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\") "
Jul 20 16:11:57.934355 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:57.934253    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-istio-envoy\") pod \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\" (UID: \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\") "
Jul 20 16:11:57.934355 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:57.934312    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-credential-socket\") pod \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\" (UID: \"07d5a0a6-90f0-4301-86d2-4bc2326a35f6\") "
Jul 20 16:11:57.934663 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:57.934439    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "07d5a0a6-90f0-4301-86d2-4bc2326a35f6" (UID: "07d5a0a6-90f0-4301-86d2-4bc2326a35f6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:11:57.934663 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:57.934546    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "07d5a0a6-90f0-4301-86d2-4bc2326a35f6" (UID: "07d5a0a6-90f0-4301-86d2-4bc2326a35f6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:11:57.934663 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:57.934598    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-istiod-ca-cert\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:11:57.934821 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:57.934665    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "07d5a0a6-90f0-4301-86d2-4bc2326a35f6" (UID: "07d5a0a6-90f0-4301-86d2-4bc2326a35f6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:11:57.934821 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:57.934767    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "07d5a0a6-90f0-4301-86d2-4bc2326a35f6" (UID: "07d5a0a6-90f0-4301-86d2-4bc2326a35f6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:11:57.935253 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:57.935228    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-istio-data" (OuterVolumeSpecName: "istio-data") pod "07d5a0a6-90f0-4301-86d2-4bc2326a35f6" (UID: "07d5a0a6-90f0-4301-86d2-4bc2326a35f6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:11:57.936479 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:57.936453    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "07d5a0a6-90f0-4301-86d2-4bc2326a35f6" (UID: "07d5a0a6-90f0-4301-86d2-4bc2326a35f6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:11:57.936572 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:57.936483    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "07d5a0a6-90f0-4301-86d2-4bc2326a35f6" (UID: "07d5a0a6-90f0-4301-86d2-4bc2326a35f6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 20 16:11:57.936672 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:57.936655    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-kube-api-access-9ft56" (OuterVolumeSpecName: "kube-api-access-9ft56") pod "07d5a0a6-90f0-4301-86d2-4bc2326a35f6" (UID: "07d5a0a6-90f0-4301-86d2-4bc2326a35f6"). InnerVolumeSpecName "kube-api-access-9ft56". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:11:57.936713 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:57.936673    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-istio-token" (OuterVolumeSpecName: "istio-token") pod "07d5a0a6-90f0-4301-86d2-4bc2326a35f6" (UID: "07d5a0a6-90f0-4301-86d2-4bc2326a35f6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:11:58.035228 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:58.035195    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-workload-certs\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:11:58.035228 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:58.035222    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-istio-podinfo\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:11:58.035228 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:58.035231    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-workload-socket\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:11:58.035434 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:58.035239    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-istio-envoy\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:11:58.035434 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:58.035247    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-credential-socket\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:11:58.035434 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:58.035257    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ft56\" (UniqueName: \"kubernetes.io/projected/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-kube-api-access-9ft56\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:11:58.035434 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:58.035271    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-istio-token\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:11:58.035434 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:58.035283    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07d5a0a6-90f0-4301-86d2-4bc2326a35f6-istio-data\") on node \"ip-10-0-130-19.ec2.internal\" DevicePath \"\""
Jul 20 16:11:58.585853 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:58.585822    2559 generic.go:356] "Generic (PLEG): container finished" podID="07d5a0a6-90f0-4301-86d2-4bc2326a35f6" containerID="3c626a0effa88f501faeda82431b87223751d4341fbed454874828eb7900d893" exitCode=0
Jul 20 16:11:58.585853 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:58.585855    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d" event={"ID":"07d5a0a6-90f0-4301-86d2-4bc2326a35f6","Type":"ContainerDied","Data":"3c626a0effa88f501faeda82431b87223751d4341fbed454874828eb7900d893"}
Jul 20 16:11:58.586062 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:58.585862    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"
Jul 20 16:11:58.586062 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:58.585889    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d" event={"ID":"07d5a0a6-90f0-4301-86d2-4bc2326a35f6","Type":"ContainerDied","Data":"734062c1a71bac85ee97ed753d199f3bbde786b53015fa8d7af65501b9b58a38"}
Jul 20 16:11:58.586062 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:58.585910    2559 scope.go:117] "RemoveContainer" containerID="3c626a0effa88f501faeda82431b87223751d4341fbed454874828eb7900d893"
Jul 20 16:11:58.594971 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:58.594957    2559 scope.go:117] "RemoveContainer" containerID="3c626a0effa88f501faeda82431b87223751d4341fbed454874828eb7900d893"
Jul 20 16:11:58.595190 ip-10-0-130-19 kubenswrapper[2559]: E0720 16:11:58.595171    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c626a0effa88f501faeda82431b87223751d4341fbed454874828eb7900d893\": container with ID starting with 3c626a0effa88f501faeda82431b87223751d4341fbed454874828eb7900d893 not found: ID does not exist" containerID="3c626a0effa88f501faeda82431b87223751d4341fbed454874828eb7900d893"
Jul 20 16:11:58.595270 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:58.595202    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c626a0effa88f501faeda82431b87223751d4341fbed454874828eb7900d893"} err="failed to get container status \"3c626a0effa88f501faeda82431b87223751d4341fbed454874828eb7900d893\": rpc error: code = NotFound desc = could not find container \"3c626a0effa88f501faeda82431b87223751d4341fbed454874828eb7900d893\": container with ID starting with 3c626a0effa88f501faeda82431b87223751d4341fbed454874828eb7900d893 not found: ID does not exist"
Jul 20 16:11:58.608330 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:58.608302    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"]
Jul 20 16:11:58.612176 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:58.612153    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-a667cd-openshift-default-79d78fffcf-m2g2d"]
Jul 20 16:11:59.117733 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:59.117702    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-jg2lz_82c3d216-9a4e-4896-b740-fe3448a8e2e1/manager/0.log"
Jul 20 16:11:59.242260 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:59.242239    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-854b67b788-f8jpt_760c77d8-0f3d-4d1b-8584-f7d087872a76/manager/0.log"
Jul 20 16:11:59.480356 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:59.480266    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-cd2b4_61b825d3-0c47-4a83-a3b9-04cf6a519bec/manager/0.log"
Jul 20 16:11:59.514066 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:59.514041    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07d5a0a6-90f0-4301-86d2-4bc2326a35f6" path="/var/lib/kubelet/pods/07d5a0a6-90f0-4301-86d2-4bc2326a35f6/volumes"
Jul 20 16:11:59.885381 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:59.885358    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65bbbccfd9-srbk6_4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f/copy-manifests/0.log"
Jul 20 16:11:59.893273 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:11:59.893250    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65bbbccfd9-srbk6_4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f/manager/1.log"
Jul 20 16:12:00.230841 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:00.230768    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-597f477cb8-sdsxn_7baf8881-13bf-441c-8417-4648a2efe781/maas-api/0.log"
Jul 20 16:12:00.580965 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:00.580936    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-9bq72_d3c36fb8-86ed-4c81-a11c-ca4ac80a2d23/postgres/0.log"
Jul 20 16:12:01.918585 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:01.918547    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-959fcf8ff-7fq7c_ad3725af-f98b-40ef-a66f-8949d3e21236/authorino/0.log"
Jul 20 16:12:03.357108 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:03.357079    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6598cc5cf9-hg6cj_9f41b33a-4491-4c6c-a238-636fe2b70a46/kube-auth-proxy/0.log"
Jul 20 16:12:03.717091 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:03.717013    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-zj7qr_f8fdc8b6-4243-4136-8c59-138c45519065/payload-pre-processing/0.log"
Jul 20 16:12:04.307748 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:04.307714    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-jcqwq_593d4ab2-3745-45d5-8f37-d9d1702e82f8/main/0.log"
Jul 20 16:12:04.531554 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:04.531528    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-56jp7_3e25304b-f3b6-4168-b67e-c1c153bb0f5a/main/0.log"
Jul 20 16:12:10.528935 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:10.528897    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xwbnm/must-gather-hxgd2"]
Jul 20 16:12:10.529326 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:10.529242    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97e35eb7-883a-4695-8a50-4325aa14f9b1" containerName="maas-api"
Jul 20 16:12:10.529326 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:10.529253    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="97e35eb7-883a-4695-8a50-4325aa14f9b1" containerName="maas-api"
Jul 20 16:12:10.529326 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:10.529270    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b23c78a1-86a6-4375-aff9-beb6259bc9b6" containerName="main"
Jul 20 16:12:10.529326 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:10.529275    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b23c78a1-86a6-4375-aff9-beb6259bc9b6" containerName="main"
Jul 20 16:12:10.529326 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:10.529298    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b084a8cb-2f9b-4e2d-8e48-a5e84981c169" containerName="maas-api"
Jul 20 16:12:10.529326 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:10.529306    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b084a8cb-2f9b-4e2d-8e48-a5e84981c169" containerName="maas-api"
Jul 20 16:12:10.529326 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:10.529317    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07d5a0a6-90f0-4301-86d2-4bc2326a35f6" containerName="istio-proxy"
Jul 20 16:12:10.529326 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:10.529325    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="07d5a0a6-90f0-4301-86d2-4bc2326a35f6" containerName="istio-proxy"
Jul 20 16:12:10.529588 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:10.529336    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67daef6d-d1ac-410b-a315-a2d8fdb91200" containerName="main"
Jul 20 16:12:10.529588 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:10.529341    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="67daef6d-d1ac-410b-a315-a2d8fdb91200" containerName="main"
Jul 20 16:12:10.529588 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:10.529349    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07d5a0a6-90f0-4301-86d2-4bc2326a35f6" containerName="istio-proxy"
Jul 20 16:12:10.529588 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:10.529356    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="07d5a0a6-90f0-4301-86d2-4bc2326a35f6" containerName="istio-proxy"
Jul 20 16:12:10.529588 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:10.529406    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="97e35eb7-883a-4695-8a50-4325aa14f9b1" containerName="maas-api"
Jul 20 16:12:10.529588 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:10.529413    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="07d5a0a6-90f0-4301-86d2-4bc2326a35f6" containerName="istio-proxy"
Jul 20 16:12:10.529588 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:10.529419    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b084a8cb-2f9b-4e2d-8e48-a5e84981c169" containerName="maas-api"
Jul 20 16:12:10.529588 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:10.529425    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="07d5a0a6-90f0-4301-86d2-4bc2326a35f6" containerName="istio-proxy"
Jul 20 16:12:10.529588 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:10.529431    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b23c78a1-86a6-4375-aff9-beb6259bc9b6" containerName="main"
Jul 20 16:12:10.529588 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:10.529437    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="67daef6d-d1ac-410b-a315-a2d8fdb91200" containerName="main"
Jul 20 16:12:10.532434 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:10.532417    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xwbnm/must-gather-hxgd2"
Jul 20 16:12:10.534865 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:10.534838    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xwbnm\"/\"openshift-service-ca.crt\""
Jul 20 16:12:10.535810 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:10.535791    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xwbnm\"/\"default-dockercfg-j8nw4\""
Jul 20 16:12:10.535910 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:10.535799    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xwbnm\"/\"kube-root-ca.crt\""
Jul 20 16:12:10.545627 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:10.545605    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xwbnm/must-gather-hxgd2"]
Jul 20 16:12:10.547495 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:10.547472    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:12:10.634615 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:10.634590    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5b1685e6-8c61-43f4-a478-084b6ed54ee0-must-gather-output\") pod \"must-gather-hxgd2\" (UID: \"5b1685e6-8c61-43f4-a478-084b6ed54ee0\") " pod="openshift-must-gather-xwbnm/must-gather-hxgd2"
Jul 20 16:12:10.634713 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:10.634621    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kz9dh\" (UniqueName: \"kubernetes.io/projected/5b1685e6-8c61-43f4-a478-084b6ed54ee0-kube-api-access-kz9dh\") pod \"must-gather-hxgd2\" (UID: \"5b1685e6-8c61-43f4-a478-084b6ed54ee0\") " pod="openshift-must-gather-xwbnm/must-gather-hxgd2"
Jul 20 16:12:10.735899 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:10.735874    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5b1685e6-8c61-43f4-a478-084b6ed54ee0-must-gather-output\") pod \"must-gather-hxgd2\" (UID: \"5b1685e6-8c61-43f4-a478-084b6ed54ee0\") " pod="openshift-must-gather-xwbnm/must-gather-hxgd2"
Jul 20 16:12:10.736009 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:10.735909    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kz9dh\" (UniqueName: \"kubernetes.io/projected/5b1685e6-8c61-43f4-a478-084b6ed54ee0-kube-api-access-kz9dh\") pod \"must-gather-hxgd2\" (UID: \"5b1685e6-8c61-43f4-a478-084b6ed54ee0\") " pod="openshift-must-gather-xwbnm/must-gather-hxgd2"
Jul 20 16:12:10.736202 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:10.736187    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5b1685e6-8c61-43f4-a478-084b6ed54ee0-must-gather-output\") pod \"must-gather-hxgd2\" (UID: \"5b1685e6-8c61-43f4-a478-084b6ed54ee0\") " pod="openshift-must-gather-xwbnm/must-gather-hxgd2"
Jul 20 16:12:10.743919 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:10.743902    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kz9dh\" (UniqueName: \"kubernetes.io/projected/5b1685e6-8c61-43f4-a478-084b6ed54ee0-kube-api-access-kz9dh\") pod \"must-gather-hxgd2\" (UID: \"5b1685e6-8c61-43f4-a478-084b6ed54ee0\") " pod="openshift-must-gather-xwbnm/must-gather-hxgd2"
Jul 20 16:12:10.770995 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:10.770970    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log" limit=1024
Jul 20 16:12:10.841429 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:10.841368    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xwbnm/must-gather-hxgd2"
Jul 20 16:12:10.966542 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:10.966503    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xwbnm/must-gather-hxgd2"]
Jul 20 16:12:10.967709 ip-10-0-130-19 kubenswrapper[2559]: W0720 16:12:10.967683    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5b1685e6_8c61_43f4_a478_084b6ed54ee0.slice/crio-9a5dfc56e25d3760d31db9c77c67ef0713afc8a0aea2617bf5eae4ceea1585a7 WatchSource:0}: Error finding container 9a5dfc56e25d3760d31db9c77c67ef0713afc8a0aea2617bf5eae4ceea1585a7: Status 404 returned error can't find the container with id 9a5dfc56e25d3760d31db9c77c67ef0713afc8a0aea2617bf5eae4ceea1585a7
Jul 20 16:12:10.969501 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:10.969479    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 16:12:11.647731 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:11.647694    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xwbnm/must-gather-hxgd2" event={"ID":"5b1685e6-8c61-43f4-a478-084b6ed54ee0","Type":"ContainerStarted","Data":"9a5dfc56e25d3760d31db9c77c67ef0713afc8a0aea2617bf5eae4ceea1585a7"}
Jul 20 16:12:12.655846 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:12.655796    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xwbnm/must-gather-hxgd2" event={"ID":"5b1685e6-8c61-43f4-a478-084b6ed54ee0","Type":"ContainerStarted","Data":"52c5c9c6f1262d0cdc96f9161f649c4932bcdc574ec9a72d3a5b4636582b30a4"}
Jul 20 16:12:12.655846 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:12.655851    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xwbnm/must-gather-hxgd2" event={"ID":"5b1685e6-8c61-43f4-a478-084b6ed54ee0","Type":"ContainerStarted","Data":"968febc826b9b4017d22f91699f248f41d024ef3271c859c6c1a0821ebf751fe"}
Jul 20 16:12:12.673595 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:12.673549    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xwbnm/must-gather-hxgd2" podStartSLOduration=1.753395284 podStartE2EDuration="2.67353372s" totalImagesPullingTime="920.138436ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:12:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:12:12.671054999 +0000 UTC m=+2715.680676662" watchObservedRunningTime="2026-07-20 16:12:12.67353372 +0000 UTC m=+2715.683155382"
Jul 20 16:12:13.607800 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:13.607767    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-t54tw_23b1e60d-e25e-48f8-9ab8-c361f433594c/global-pull-secret-syncer/0.log"
Jul 20 16:12:13.663109 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:13.663079    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-2m28f_f60e8181-bece-4bff-b8a3-4ad06b349d6c/konnectivity-agent/0.log"
Jul 20 16:12:13.764370 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:13.764338    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-19.ec2.internal_9120b8f40b5d9a133728742e2ee8b96d/haproxy/0.log"
Jul 20 16:12:18.608207 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:18.607436    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-959fcf8ff-7fq7c_ad3725af-f98b-40ef-a66f-8949d3e21236/authorino/0.log"
Jul 20 16:12:20.906444 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:20.906394    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-xmqng_aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4/kube-state-metrics/0.log"
Jul 20 16:12:20.936085 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:20.936049    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-xmqng_aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4/kube-rbac-proxy-main/0.log"
Jul 20 16:12:20.979529 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:20.979415    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-xmqng_aa525c7a-d6d4-4a82-b068-14c7d0f5dbe4/kube-rbac-proxy-self/0.log"
Jul 20 16:12:21.045326 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:21.045280    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-zmnbs_947ca3d1-14d5-4e72-b73c-b04324947ddb/monitoring-plugin/0.log"
Jul 20 16:12:21.089624 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:21.089582    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5j487_a93e0f62-0ea5-4f63-ac6a-ad5252b879c7/node-exporter/0.log"
Jul 20 16:12:21.122621 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:21.122559    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5j487_a93e0f62-0ea5-4f63-ac6a-ad5252b879c7/kube-rbac-proxy/0.log"
Jul 20 16:12:21.155522 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:21.155491    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5j487_a93e0f62-0ea5-4f63-ac6a-ad5252b879c7/init-textfile/0.log"
Jul 20 16:12:21.448938 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:21.448910    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bcbcfeb8-056b-42d8-be5a-7d0e6b845afe/prometheus/0.log"
Jul 20 16:12:21.479393 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:21.479364    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bcbcfeb8-056b-42d8-be5a-7d0e6b845afe/config-reloader/0.log"
Jul 20 16:12:21.505344 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:21.505316    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bcbcfeb8-056b-42d8-be5a-7d0e6b845afe/thanos-sidecar/0.log"
Jul 20 16:12:21.535402 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:21.535371    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bcbcfeb8-056b-42d8-be5a-7d0e6b845afe/kube-rbac-proxy-web/0.log"
Jul 20 16:12:21.562789 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:21.562764    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bcbcfeb8-056b-42d8-be5a-7d0e6b845afe/kube-rbac-proxy/0.log"
Jul 20 16:12:21.591058 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:21.591031    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bcbcfeb8-056b-42d8-be5a-7d0e6b845afe/kube-rbac-proxy-thanos/0.log"
Jul 20 16:12:21.622981 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:21.622950    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bcbcfeb8-056b-42d8-be5a-7d0e6b845afe/init-config-reloader/0.log"
Jul 20 16:12:21.840327 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:21.840265    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-778645887f-jd5cm_8d92cbb5-adb9-4c90-b192-4f2f14738ef7/thanos-query/0.log"
Jul 20 16:12:21.867125 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:21.867099    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-778645887f-jd5cm_8d92cbb5-adb9-4c90-b192-4f2f14738ef7/kube-rbac-proxy-web/0.log"
Jul 20 16:12:21.891276 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:21.891250    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-778645887f-jd5cm_8d92cbb5-adb9-4c90-b192-4f2f14738ef7/kube-rbac-proxy/0.log"
Jul 20 16:12:21.914922 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:21.914897    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-778645887f-jd5cm_8d92cbb5-adb9-4c90-b192-4f2f14738ef7/prom-label-proxy/0.log"
Jul 20 16:12:21.939204 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:21.939175    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-778645887f-jd5cm_8d92cbb5-adb9-4c90-b192-4f2f14738ef7/kube-rbac-proxy-rules/0.log"
Jul 20 16:12:21.962983 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:21.962958    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-778645887f-jd5cm_8d92cbb5-adb9-4c90-b192-4f2f14738ef7/kube-rbac-proxy-metrics/0.log"
Jul 20 16:12:23.333909 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:23.333875    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xwbnm/perf-node-gather-daemonset-qbl2s"]
Jul 20 16:12:23.338377 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:23.338355    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-qbl2s"
Jul 20 16:12:23.348408 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:23.348382    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xwbnm/perf-node-gather-daemonset-qbl2s"]
Jul 20 16:12:23.476103 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:23.476065    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/682b2f34-9743-4fc5-b33c-2e023ea3c108-lib-modules\") pod \"perf-node-gather-daemonset-qbl2s\" (UID: \"682b2f34-9743-4fc5-b33c-2e023ea3c108\") " pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-qbl2s"
Jul 20 16:12:23.476529 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:23.476498    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/682b2f34-9743-4fc5-b33c-2e023ea3c108-podres\") pod \"perf-node-gather-daemonset-qbl2s\" (UID: \"682b2f34-9743-4fc5-b33c-2e023ea3c108\") " pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-qbl2s"
Jul 20 16:12:23.476789 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:23.476774    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/682b2f34-9743-4fc5-b33c-2e023ea3c108-sys\") pod \"perf-node-gather-daemonset-qbl2s\" (UID: \"682b2f34-9743-4fc5-b33c-2e023ea3c108\") " pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-qbl2s"
Jul 20 16:12:23.476935 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:23.476918    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cd424\" (UniqueName: \"kubernetes.io/projected/682b2f34-9743-4fc5-b33c-2e023ea3c108-kube-api-access-cd424\") pod \"perf-node-gather-daemonset-qbl2s\" (UID: \"682b2f34-9743-4fc5-b33c-2e023ea3c108\") " pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-qbl2s"
Jul 20 16:12:23.477075 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:23.477061    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/682b2f34-9743-4fc5-b33c-2e023ea3c108-proc\") pod \"perf-node-gather-daemonset-qbl2s\" (UID: \"682b2f34-9743-4fc5-b33c-2e023ea3c108\") " pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-qbl2s"
Jul 20 16:12:23.578748 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:23.578707    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/682b2f34-9743-4fc5-b33c-2e023ea3c108-podres\") pod \"perf-node-gather-daemonset-qbl2s\" (UID: \"682b2f34-9743-4fc5-b33c-2e023ea3c108\") " pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-qbl2s"
Jul 20 16:12:23.578918 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:23.578805    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/682b2f34-9743-4fc5-b33c-2e023ea3c108-sys\") pod \"perf-node-gather-daemonset-qbl2s\" (UID: \"682b2f34-9743-4fc5-b33c-2e023ea3c108\") " pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-qbl2s"
Jul 20 16:12:23.578918 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:23.578837    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cd424\" (UniqueName: \"kubernetes.io/projected/682b2f34-9743-4fc5-b33c-2e023ea3c108-kube-api-access-cd424\") pod \"perf-node-gather-daemonset-qbl2s\" (UID: \"682b2f34-9743-4fc5-b33c-2e023ea3c108\") " pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-qbl2s"
Jul 20 16:12:23.578918 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:23.578873    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/682b2f34-9743-4fc5-b33c-2e023ea3c108-proc\") pod \"perf-node-gather-daemonset-qbl2s\" (UID: \"682b2f34-9743-4fc5-b33c-2e023ea3c108\") " pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-qbl2s"
Jul 20 16:12:23.578918 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:23.578891    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/682b2f34-9743-4fc5-b33c-2e023ea3c108-podres\") pod \"perf-node-gather-daemonset-qbl2s\" (UID: \"682b2f34-9743-4fc5-b33c-2e023ea3c108\") " pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-qbl2s"
Jul 20 16:12:23.579106 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:23.578931    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/682b2f34-9743-4fc5-b33c-2e023ea3c108-lib-modules\") pod \"perf-node-gather-daemonset-qbl2s\" (UID: \"682b2f34-9743-4fc5-b33c-2e023ea3c108\") " pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-qbl2s"
Jul 20 16:12:23.579106 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:23.578956    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/682b2f34-9743-4fc5-b33c-2e023ea3c108-sys\") pod \"perf-node-gather-daemonset-qbl2s\" (UID: \"682b2f34-9743-4fc5-b33c-2e023ea3c108\") " pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-qbl2s"
Jul 20 16:12:23.579106 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:23.579065    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/682b2f34-9743-4fc5-b33c-2e023ea3c108-proc\") pod \"perf-node-gather-daemonset-qbl2s\" (UID: \"682b2f34-9743-4fc5-b33c-2e023ea3c108\") " pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-qbl2s"
Jul 20 16:12:23.579236 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:23.579169    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/682b2f34-9743-4fc5-b33c-2e023ea3c108-lib-modules\") pod \"perf-node-gather-daemonset-qbl2s\" (UID: \"682b2f34-9743-4fc5-b33c-2e023ea3c108\") " pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-qbl2s"
Jul 20 16:12:23.587218 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:23.587156    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cd424\" (UniqueName: \"kubernetes.io/projected/682b2f34-9743-4fc5-b33c-2e023ea3c108-kube-api-access-cd424\") pod \"perf-node-gather-daemonset-qbl2s\" (UID: \"682b2f34-9743-4fc5-b33c-2e023ea3c108\") " pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-qbl2s"
Jul 20 16:12:23.651030 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:23.650997    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-qbl2s"
Jul 20 16:12:23.830571 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:23.830414    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xwbnm/perf-node-gather-daemonset-qbl2s"]
Jul 20 16:12:23.833191 ip-10-0-130-19 kubenswrapper[2559]: W0720 16:12:23.833157    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod682b2f34_9743_4fc5_b33c_2e023ea3c108.slice/crio-b550d234f37e2a324874c612ada1323ea236a5818a3fb87f12b9cff9e8023b80 WatchSource:0}: Error finding container b550d234f37e2a324874c612ada1323ea236a5818a3fb87f12b9cff9e8023b80: Status 404 returned error can't find the container with id b550d234f37e2a324874c612ada1323ea236a5818a3fb87f12b9cff9e8023b80
Jul 20 16:12:24.730808 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:24.730765    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-qbl2s" event={"ID":"682b2f34-9743-4fc5-b33c-2e023ea3c108","Type":"ContainerStarted","Data":"88cec171380026dc7ff1cbeb066d547a85b1377465f4ace439f4b83439941cf0"}
Jul 20 16:12:24.731272 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:24.730837    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-qbl2s" event={"ID":"682b2f34-9743-4fc5-b33c-2e023ea3c108","Type":"ContainerStarted","Data":"b550d234f37e2a324874c612ada1323ea236a5818a3fb87f12b9cff9e8023b80"}
Jul 20 16:12:24.731272 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:24.731142    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-qbl2s"
Jul 20 16:12:24.751257 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:24.751181    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-qbl2s" podStartSLOduration=1.751159221 podStartE2EDuration="1.751159221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:12:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:12:24.7457025 +0000 UTC m=+2727.755324163" watchObservedRunningTime="2026-07-20 16:12:24.751159221 +0000 UTC m=+2727.760780884"
Jul 20 16:12:25.923164 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:25.923108    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2bz87_ca3c4e74-af1d-4595-aea5-19ebd06b7263/dns/0.log"
Jul 20 16:12:25.946895 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:25.946871    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2bz87_ca3c4e74-af1d-4595-aea5-19ebd06b7263/kube-rbac-proxy/0.log"
Jul 20 16:12:26.107601 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:26.107573    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-j45gb_ada5dd42-e780-4cc0-a845-ea417f1eac27/dns-node-resolver/0.log"
Jul 20 16:12:26.736353 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:26.736325    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6cb967f748-tsjhw_87c5f3e8-bc71-446b-9098-047de625ee8d/registry/0.log"
Jul 20 16:12:26.885082 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:26.885055    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pqqrs_b87d2983-fef9-4867-89c6-5f3fe76bd062/node-ca/0.log"
Jul 20 16:12:27.990946 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:27.990921    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6598cc5cf9-hg6cj_9f41b33a-4491-4c6c-a238-636fe2b70a46/kube-auth-proxy/0.log"
Jul 20 16:12:28.167825 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:28.167791    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-zj7qr_f8fdc8b6-4243-4136-8c59-138c45519065/payload-pre-processing/0.log"
Jul 20 16:12:28.889449 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:28.889423    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fxdt4_144b485c-ce5b-4ac5-82c7-28f6fa5320af/serve-healthcheck-canary/0.log"
Jul 20 16:12:29.476090 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:29.476048    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gd8kb_1797129a-35ab-4364-98a7-7d6dc8f8e6b1/kube-rbac-proxy/0.log"
Jul 20 16:12:29.502349 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:29.502326    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gd8kb_1797129a-35ab-4364-98a7-7d6dc8f8e6b1/exporter/0.log"
Jul 20 16:12:29.532003 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:29.531971    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gd8kb_1797129a-35ab-4364-98a7-7d6dc8f8e6b1/extractor/0.log"
Jul 20 16:12:30.746584 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:30.746555    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-qbl2s"
Jul 20 16:12:32.081371 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:32.081322    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-jg2lz_82c3d216-9a4e-4896-b740-fe3448a8e2e1/manager/0.log"
Jul 20 16:12:32.246052 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:32.245835    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-854b67b788-f8jpt_760c77d8-0f3d-4d1b-8584-f7d087872a76/manager/0.log"
Jul 20 16:12:32.358427 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:32.358397    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-cd2b4_61b825d3-0c47-4a83-a3b9-04cf6a519bec/manager/0.log"
Jul 20 16:12:32.501710 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:32.501638    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65bbbccfd9-srbk6_4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f/manager/1.log"
Jul 20 16:12:32.568312 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:32.567970    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65bbbccfd9-srbk6_4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f/manager/0.log"
Jul 20 16:12:32.588459 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:32.588431    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65bbbccfd9-srbk6_4a1bc22f-d0b8-4034-8e37-db8cb8c7f00f/copy-manifests/0.log"
Jul 20 16:12:33.930480 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:33.930445    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5b8ffc8644-n5rcf_e65179e3-c7f6-4d9e-9a4f-99f773d3e1c9/manager/0.log"
Jul 20 16:12:34.167037 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:34.167010    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6598cc5cf9-hg6cj_9f41b33a-4491-4c6c-a238-636fe2b70a46/kube-auth-proxy/0.log"
Jul 20 16:12:34.310014 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:34.309977    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-zj7qr_f8fdc8b6-4243-4136-8c59-138c45519065/payload-pre-processing/0.log"
Jul 20 16:12:35.684882 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:35.684859    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6598cc5cf9-hg6cj_9f41b33a-4491-4c6c-a238-636fe2b70a46/kube-auth-proxy/0.log"
Jul 20 16:12:35.800562 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:35.800534    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-zj7qr_f8fdc8b6-4243-4136-8c59-138c45519065/payload-pre-processing/0.log"
Jul 20 16:12:40.635594 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:40.635563    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gpzpc_5edc4d01-8bc5-45b4-9ae3-ca95d913d778/kube-multus-additional-cni-plugins/0.log"
Jul 20 16:12:40.659666 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:40.659634    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gpzpc_5edc4d01-8bc5-45b4-9ae3-ca95d913d778/egress-router-binary-copy/0.log"
Jul 20 16:12:40.682438 ip-10-0-130-19 kubenswrapper[2559]: I0720 16:12:40.682416    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gpzpc_5edc4d01-8bc5-45b4-9ae3-ca95d913d778/cni-plugins/0.log"