Sep 05 07:48:17.116243 ip-10-0-133-14 systemd[1]: Starting Kubernetes Kubelet...
Sep 05 07:48:17.531146 ip-10-0-133-14 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 05 07:48:17.531146 ip-10-0-133-14 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 05 07:48:17.531146 ip-10-0-133-14 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 05 07:48:17.531146 ip-10-0-133-14 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 05 07:48:17.531146 ip-10-0-133-14 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 05 07:48:17.533208 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.532346    2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 05 07:48:17.536780 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536764    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 05 07:48:17.536830 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536794    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 05 07:48:17.536830 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536798    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 05 07:48:17.536830 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536801    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 05 07:48:17.536830 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536806    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 05 07:48:17.536830 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536810    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 05 07:48:17.536830 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536813    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 05 07:48:17.536830 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536816    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 05 07:48:17.536830 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536819    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 05 07:48:17.536830 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536822    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 05 07:48:17.536830 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536824    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 05 07:48:17.536830 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536827    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 05 07:48:17.536830 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536830    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 05 07:48:17.536830 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536833    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 05 07:48:17.536830 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536836    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 05 07:48:17.537185 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536854    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 05 07:48:17.537185 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536858    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 05 07:48:17.537185 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536860    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 05 07:48:17.537185 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536863    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 05 07:48:17.537185 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536865    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 05 07:48:17.537185 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536868    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 05 07:48:17.537185 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536870    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 05 07:48:17.537185 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536873    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 05 07:48:17.537185 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536876    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 05 07:48:17.537185 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536878    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 05 07:48:17.537185 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536881    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 05 07:48:17.537185 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536884    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 05 07:48:17.537185 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536886    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 05 07:48:17.537185 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536889    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 05 07:48:17.537185 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536891    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 05 07:48:17.537185 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536894    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 05 07:48:17.537185 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536896    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 05 07:48:17.537185 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536899    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 05 07:48:17.537185 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536902    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 05 07:48:17.537185 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536905    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 05 07:48:17.537691 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536909    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 05 07:48:17.537691 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536911    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 05 07:48:17.537691 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536913    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 05 07:48:17.537691 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536916    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 05 07:48:17.537691 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536918    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 05 07:48:17.537691 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536921    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 05 07:48:17.537691 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536923    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 05 07:48:17.537691 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536925    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 05 07:48:17.537691 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536928    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 05 07:48:17.537691 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536930    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 05 07:48:17.537691 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536932    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 05 07:48:17.537691 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536936    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 05 07:48:17.537691 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536939    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 05 07:48:17.537691 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536943    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 05 07:48:17.537691 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536946    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 05 07:48:17.537691 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536949    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 05 07:48:17.537691 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536952    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 05 07:48:17.537691 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536954    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 05 07:48:17.537691 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536957    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 05 07:48:17.538165 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536959    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 05 07:48:17.538165 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536962    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 05 07:48:17.538165 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536964    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 05 07:48:17.538165 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536967    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 05 07:48:17.538165 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536969    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 05 07:48:17.538165 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536971    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 05 07:48:17.538165 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536974    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 05 07:48:17.538165 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536976    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 05 07:48:17.538165 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536978    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 05 07:48:17.538165 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536981    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 05 07:48:17.538165 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536983    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 05 07:48:17.538165 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536986    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 05 07:48:17.538165 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536989    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 05 07:48:17.538165 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536992    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 05 07:48:17.538165 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536994    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 05 07:48:17.538165 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536996    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 05 07:48:17.538165 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.536999    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 05 07:48:17.538165 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.537001    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 05 07:48:17.538165 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.537004    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 05 07:48:17.538165 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.537006    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 05 07:48:17.539082 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.537009    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 05 07:48:17.539082 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.537011    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 05 07:48:17.539082 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.537015    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 05 07:48:17.539082 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.537018    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 05 07:48:17.539082 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.537020    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 05 07:48:17.539082 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.537023    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 05 07:48:17.539082 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.537026    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 05 07:48:17.539082 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.537029    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 05 07:48:17.539082 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.537032    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 05 07:48:17.539082 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.537034    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 05 07:48:17.539082 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.537037    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 05 07:48:17.539082 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.537039    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 05 07:48:17.539082 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.537043    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 05 07:48:17.539082 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.537046    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 05 07:48:17.539082 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.537049    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 05 07:48:17.539082 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.537051    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 05 07:48:17.539082 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.537054    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 05 07:48:17.539082 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.537056    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 05 07:48:17.539082 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.537058    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 05 07:48:17.539626 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.537061    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 05 07:48:17.539626 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.537064    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 05 07:48:17.539626 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.537066    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 05 07:48:17.539626 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.537068    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 05 07:48:17.539626 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.537071    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 05 07:48:17.539626 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.537073    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 05 07:48:17.539626 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538402    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 05 07:48:17.539626 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538415    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 05 07:48:17.539626 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538425    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 05 07:48:17.539626 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538429    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 05 07:48:17.539626 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538433    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 05 07:48:17.539626 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538438    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 05 07:48:17.539626 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538443    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 05 07:48:17.539626 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538447    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 05 07:48:17.539626 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538451    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 05 07:48:17.539626 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538455    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 05 07:48:17.539626 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538461    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 05 07:48:17.539626 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538469    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 05 07:48:17.539626 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538475    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 05 07:48:17.540218 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538480    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 05 07:48:17.540218 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538489    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 05 07:48:17.540218 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538493    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 05 07:48:17.540218 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538498    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 05 07:48:17.540218 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538502    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 05 07:48:17.540218 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538507    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 05 07:48:17.540218 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538511    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 05 07:48:17.540218 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538515    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 05 07:48:17.540218 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538519    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 05 07:48:17.540218 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538523    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 05 07:48:17.540218 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538527    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 05 07:48:17.540218 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538531    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 05 07:48:17.540218 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538536    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 05 07:48:17.540218 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538540    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 05 07:48:17.540218 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538549    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 05 07:48:17.540218 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538553    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 05 07:48:17.540218 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538557    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 05 07:48:17.540218 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538561    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 05 07:48:17.540218 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538565    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 05 07:48:17.540218 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538569    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 05 07:48:17.540713 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538574    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 05 07:48:17.540713 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538579    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 05 07:48:17.540713 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538584    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 05 07:48:17.540713 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538588    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 05 07:48:17.540713 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538600    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 05 07:48:17.540713 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538604    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 05 07:48:17.540713 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538613    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 05 07:48:17.540713 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538617    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 05 07:48:17.540713 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538621    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 05 07:48:17.540713 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538628    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 05 07:48:17.540713 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538634    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 05 07:48:17.540713 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538641    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 05 07:48:17.540713 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538645    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 05 07:48:17.540713 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538649    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 05 07:48:17.540713 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538653    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 05 07:48:17.540713 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538658    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 05 07:48:17.540713 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538662    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 05 07:48:17.540713 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538667    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 05 07:48:17.540713 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538676    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 05 07:48:17.541233 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538680    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 05 07:48:17.541233 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538684    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 05 07:48:17.541233 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538689    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 05 07:48:17.541233 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538693    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 05 07:48:17.541233 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538697    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 05 07:48:17.541233 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538702    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 05 07:48:17.541233 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538707    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 05 07:48:17.541233 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538712    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 05 07:48:17.541233 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538717    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 05 07:48:17.541233 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538721    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 05 07:48:17.541233 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538725    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 05 07:48:17.541233 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538730    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 05 07:48:17.541233 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538739    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 05 07:48:17.541233 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538744    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 05 07:48:17.541233 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538749    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 05 07:48:17.541233 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538754    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 05 07:48:17.541233 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538758    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 05 07:48:17.541233 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538762    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 05 07:48:17.541233 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538766    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 05 07:48:17.541233 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538770    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 05 07:48:17.541725 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538774    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 05 07:48:17.541725 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538779    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 05 07:48:17.541725 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538783    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 05 07:48:17.541725 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538787    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 05 07:48:17.541725 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538792    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 05 07:48:17.541725 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538800    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 05 07:48:17.541725 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538804    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 05 07:48:17.541725 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538811    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 05 07:48:17.541725 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538819    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 05 07:48:17.541725 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538823    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 05 07:48:17.541725 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538828    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 05 07:48:17.541725 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538832    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 05 07:48:17.541725 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538837    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 05 07:48:17.541725 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538858    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 05 07:48:17.541725 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538863    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 05 07:48:17.541725 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538867    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 05 07:48:17.541725 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538875    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 05 07:48:17.541725 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538883    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 05 07:48:17.541725 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538890    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 05 07:48:17.542202 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538900    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 05 07:48:17.542202 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538904    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 05 07:48:17.542202 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538908    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 05 07:48:17.542202 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538912    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 05 07:48:17.542202 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538918    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 05 07:48:17.542202 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538922    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 05 07:48:17.542202 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538927    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 05 07:48:17.542202 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.538931    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 05 07:48:17.542202 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539721    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 05 07:48:17.542202 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539729    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 05 07:48:17.542202 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539732    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 05 07:48:17.542202 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539735    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 05 07:48:17.542202 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539738    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 05 07:48:17.542202 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539741    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 05 07:48:17.542202 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539744    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 05 07:48:17.542202 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539747    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 05 07:48:17.542202 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539750    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 05 07:48:17.542202 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539753    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 05 07:48:17.542202 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539756    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 05 07:48:17.542202 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539758    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 05 07:48:17.542696 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539761    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 05 07:48:17.542696 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539763    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 05 07:48:17.542696 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539766    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 05 07:48:17.542696 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539768    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 05 07:48:17.542696 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539771    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 05 07:48:17.542696 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539773    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 05 07:48:17.542696 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539776    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 05 07:48:17.542696 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539779    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 05 07:48:17.542696 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539781    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 05 07:48:17.542696 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539784    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 05 07:48:17.542696 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539786    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 05 07:48:17.542696 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539788    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 05 07:48:17.542696 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539793    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 05 07:48:17.542696 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539798    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 05 07:48:17.542696 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539803    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 05 07:48:17.542696 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539806    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 05 07:48:17.542696 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539808    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 05 07:48:17.542696 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539811    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 05 07:48:17.542696 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539814    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 05 07:48:17.542696 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539817    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 05 07:48:17.543193 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539820    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 05 07:48:17.543193 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539823    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 05 07:48:17.543193 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539826    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 05 07:48:17.543193 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539829    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 05 07:48:17.543193 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539832    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 05 07:48:17.543193 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539835    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 05 07:48:17.543193 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539838    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 05 07:48:17.543193 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539864    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 05 07:48:17.543193 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539868    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 05 07:48:17.543193 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539872    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 05 07:48:17.543193 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539875    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 05 07:48:17.543193 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539878    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 05 07:48:17.543193 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539881    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 05 07:48:17.543193 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539883    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 05 07:48:17.543193 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539886    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 05 07:48:17.543193 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539888    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 05 07:48:17.543193 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539891    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 05 07:48:17.543193 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539893    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 05 07:48:17.543193 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539896    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 05 07:48:17.543193 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539898    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 05 07:48:17.543667 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539901    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 05 07:48:17.543667 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539903    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 05 07:48:17.543667 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539906    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 05 07:48:17.543667 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539908    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 05 07:48:17.543667 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539913    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 05 07:48:17.543667 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539915    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 05 07:48:17.543667 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539918    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 05 07:48:17.543667 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539920    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 05 07:48:17.543667 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539923    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 05 07:48:17.543667 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539926    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 05 07:48:17.543667 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539928    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 05 07:48:17.543667 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539931    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 05 07:48:17.543667 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539933    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 05 07:48:17.543667 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539936    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 05 07:48:17.543667 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539939    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 05 07:48:17.543667 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539942    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 05 07:48:17.543667 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539945    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 05 07:48:17.543667 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539948    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 05 07:48:17.543667 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539952    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 05 07:48:17.543667 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539955    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 05 07:48:17.544164 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539957    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 05 07:48:17.544164 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539960    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 05 07:48:17.544164 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539964    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 05 07:48:17.544164 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539967    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 05 07:48:17.544164 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539970    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 05 07:48:17.544164 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539973    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 05 07:48:17.544164 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539976    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 05 07:48:17.544164 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539979    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 05 07:48:17.544164 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539981    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 05 07:48:17.544164 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539984    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 05 07:48:17.544164 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539987    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 05 07:48:17.544164 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539990    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 05 07:48:17.544164 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539993    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 05 07:48:17.544164 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539995    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 05 07:48:17.544164 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.539998    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 05 07:48:17.544164 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540000    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 05 07:48:17.544164 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540004    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 05 07:48:17.544164 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540006    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 05 07:48:17.544164 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540009    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 05 07:48:17.544623 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540011    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 05 07:48:17.544623 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540014    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 05 07:48:17.544623 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540016    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 05 07:48:17.544623 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540019    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 05 07:48:17.544623 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540021    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 05 07:48:17.544623 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540023    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 05 07:48:17.544623 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540026    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 05 07:48:17.544623 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540028    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 05 07:48:17.544623 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540105    2563 flags.go:64] FLAG: --address="0.0.0.0"
Sep 05 07:48:17.544623 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540140    2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 05 07:48:17.544623 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540146    2563 flags.go:64] FLAG: --anonymous-auth="true"
Sep 05 07:48:17.544623 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540150    2563 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 05 07:48:17.544623 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540154    2563 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 05 07:48:17.544623 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540157    2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 05 07:48:17.544623 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540161    2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 05 07:48:17.544623 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540165    2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 05 07:48:17.544623 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540168    2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 05 07:48:17.544623 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540171    2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 05 07:48:17.544623 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540174    2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 05 07:48:17.544623 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540178    2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 05 07:48:17.544623 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540181    2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 05 07:48:17.545136 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540184    2563 flags.go:64] FLAG: --cgroup-root=""
Sep 05 07:48:17.545136 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540187    2563 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 05 07:48:17.545136 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540190    2563 flags.go:64] FLAG: --client-ca-file=""
Sep 05 07:48:17.545136 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540194    2563 flags.go:64] FLAG: --cloud-provider="external"
Sep 05 07:48:17.545136 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540197    2563 flags.go:64] FLAG: --cluster-dns="[]"
Sep 05 07:48:17.545136 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540201    2563 flags.go:64] FLAG: --cluster-domain=""
Sep 05 07:48:17.545136 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540204    2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 05 07:48:17.545136 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540207    2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 05 07:48:17.545136 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540211    2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 05 07:48:17.545136 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540214    2563 flags.go:64] FLAG: --container-log-max-files="5"
Sep 05 07:48:17.545136 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540218    2563 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 05 07:48:17.545136 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540221    2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 05 07:48:17.545136 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540224    2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 05 07:48:17.545136 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540228    2563 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 05 07:48:17.545136 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540231    2563 flags.go:64] FLAG: --contention-profiling="false"
Sep 05 07:48:17.545136 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540234    2563 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 05 07:48:17.545136 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540237    2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 05 07:48:17.545136 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540240    2563 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 05 07:48:17.545136 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540242    2563 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 05 07:48:17.545136 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540246    2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 05 07:48:17.545136 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540252    2563 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 05 07:48:17.545136 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540254    2563 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 05 07:48:17.545136 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540257    2563 flags.go:64] FLAG: --enable-load-reader="false"
Sep 05 07:48:17.545136 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540260    2563 flags.go:64] FLAG: --enable-server="true"
Sep 05 07:48:17.545136 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540263    2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 05 07:48:17.545718 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540267    2563 flags.go:64] FLAG: --event-burst="100"
Sep 05 07:48:17.545718 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540270    2563 flags.go:64] FLAG: --event-qps="50"
Sep 05 07:48:17.545718 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540273    2563 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 05 07:48:17.545718 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540276    2563 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 05 07:48:17.545718 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540279    2563 flags.go:64] FLAG: --eviction-hard=""
Sep 05 07:48:17.545718 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540283    2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 05 07:48:17.545718 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540285    2563 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 05 07:48:17.545718 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540288    2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 05 07:48:17.545718 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540291    2563 flags.go:64] FLAG: --eviction-soft=""
Sep 05 07:48:17.545718 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540294    2563 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 05 07:48:17.545718 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540297    2563 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 05 07:48:17.545718 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540300    2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 05 07:48:17.545718 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540303    2563 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 05 07:48:17.545718 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540305    2563 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 05 07:48:17.545718 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540308    2563 flags.go:64] FLAG: --fail-swap-on="true"
Sep 05 07:48:17.545718 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540312    2563 flags.go:64] FLAG: --feature-gates=""
Sep 05 07:48:17.545718 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540315    2563 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 05 07:48:17.545718 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540318    2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 05 07:48:17.545718 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540321    2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 05 07:48:17.545718 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540324    2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 05 07:48:17.545718 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540327    2563 flags.go:64] FLAG: --healthz-port="10248"
Sep 05 07:48:17.545718 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540330    2563 flags.go:64] FLAG: --help="false"
Sep 05 07:48:17.545718 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540333    2563 flags.go:64] FLAG: --hostname-override="ip-10-0-133-14.ec2.internal"
Sep 05 07:48:17.545718 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540335    2563 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 05 07:48:17.545718 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540338    2563 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 05 07:48:17.546350 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540341    2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 05 07:48:17.546350 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540345    2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 05 07:48:17.546350 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540349    2563 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 05 07:48:17.546350 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540352    2563 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 05 07:48:17.546350 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540354    2563 flags.go:64] FLAG: --image-service-endpoint=""
Sep 05 07:48:17.546350 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540358    2563 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 05 07:48:17.546350 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540360    2563 flags.go:64] FLAG: --kube-api-burst="100"
Sep 05 07:48:17.546350 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540363    2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 05 07:48:17.546350 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540366    2563 flags.go:64] FLAG: --kube-api-qps="50"
Sep 05 07:48:17.546350 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540369    2563 flags.go:64] FLAG: --kube-reserved=""
Sep 05 07:48:17.546350 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540372    2563 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 05 07:48:17.546350 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540375    2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 05 07:48:17.546350 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540378    2563 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 05 07:48:17.546350 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540380    2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 05 07:48:17.546350 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540383    2563 flags.go:64] FLAG: --lock-file=""
Sep 05 07:48:17.546350 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540385    2563 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 05 07:48:17.546350 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540388    2563 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 05 07:48:17.546350 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540391    2563 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 05 07:48:17.546350 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540396    2563 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 05 07:48:17.546350 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540400    2563 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 05 07:48:17.546350 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540403    2563 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 05 07:48:17.546350 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540406    2563 flags.go:64] FLAG: --logging-format="text"
Sep 05 07:48:17.546350 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540408    2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 05 07:48:17.546939 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540412    2563 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 05 07:48:17.546939 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540414    2563 flags.go:64] FLAG: --manifest-url=""
Sep 05 07:48:17.546939 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540417    2563 flags.go:64] FLAG: --manifest-url-header=""
Sep 05 07:48:17.546939 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540421    2563 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 05 07:48:17.546939 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540424    2563 flags.go:64] FLAG: --max-open-files="1000000"
Sep 05 07:48:17.546939 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540428    2563 flags.go:64] FLAG: --max-pods="110"
Sep 05 07:48:17.546939 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540431    2563 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 05 07:48:17.546939 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540433    2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 05 07:48:17.546939 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540436    2563 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 05 07:48:17.546939 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540439    2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 05 07:48:17.546939 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540442    2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 05 07:48:17.546939 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540446    2563 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 05 07:48:17.546939 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540449    2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 05 07:48:17.546939 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540456    2563 flags.go:64] FLAG: --node-status-max-images="50"
Sep 05 07:48:17.546939 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540458    2563 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 05 07:48:17.546939 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540461    2563 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 05 07:48:17.546939 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540464    2563 flags.go:64] FLAG: --pod-cidr=""
Sep 05 07:48:17.546939 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540467    2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 05 07:48:17.546939 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540472    2563 flags.go:64] FLAG: --pod-manifest-path=""
Sep 05 07:48:17.546939 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540475    2563 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 05 07:48:17.546939 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540478    2563 flags.go:64] FLAG: --pods-per-core="0"
Sep 05 07:48:17.546939 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540481    2563 flags.go:64] FLAG: --port="10250"
Sep 05 07:48:17.546939 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540483    2563 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 05 07:48:17.546939 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540486    2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e175897ef63d985a"
Sep 05 07:48:17.547520 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540489    2563 flags.go:64] FLAG: --qos-reserved=""
Sep 05 07:48:17.547520 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540511    2563 flags.go:64] FLAG: --read-only-port="10255"
Sep 05 07:48:17.547520 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540515    2563 flags.go:64] FLAG: --register-node="true"
Sep 05 07:48:17.547520 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540519    2563 flags.go:64] FLAG: --register-with-taints=""
Sep 05 07:48:17.547520 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540523    2563 flags.go:64] FLAG: --registry-burst="10"
Sep 05 07:48:17.547520 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540526    2563 flags.go:64] FLAG: --registry-qps="5"
Sep 05 07:48:17.547520 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540529    2563 flags.go:64] FLAG: --reserved-cpus=""
Sep 05 07:48:17.547520 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540532    2563 flags.go:64] FLAG: --reserved-memory=""
Sep 05 07:48:17.547520 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540535    2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 05 07:48:17.547520 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540538    2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 05 07:48:17.547520 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540541    2563 flags.go:64] FLAG: --rotate-certificates="false"
Sep 05 07:48:17.547520 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540544    2563 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 05 07:48:17.547520 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540547    2563 flags.go:64] FLAG: --runonce="false"
Sep 05 07:48:17.547520 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540550    2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 05 07:48:17.547520 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540552    2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 05 07:48:17.547520 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540555    2563 flags.go:64] FLAG: --seccomp-default="false"
Sep 05 07:48:17.547520 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540558    2563 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 05 07:48:17.547520 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540561    2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 05 07:48:17.547520 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540564    2563 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 05 07:48:17.547520 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540566    2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 05 07:48:17.547520 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540571    2563 flags.go:64] FLAG: --storage-driver-password="root"
Sep 05 07:48:17.547520 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540574    2563 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 05 07:48:17.547520 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540576    2563 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 05 07:48:17.547520 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540579    2563 flags.go:64] FLAG: --storage-driver-user="root"
Sep 05 07:48:17.547520 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540582    2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 05 07:48:17.547520 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540585    2563 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 05 07:48:17.548223 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540587    2563 flags.go:64] FLAG: --system-cgroups=""
Sep 05 07:48:17.548223 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540590    2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 05 07:48:17.548223 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540595    2563 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 05 07:48:17.548223 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540598    2563 flags.go:64] FLAG: --tls-cert-file=""
Sep 05 07:48:17.548223 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540601    2563 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 05 07:48:17.548223 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540605    2563 flags.go:64] FLAG: --tls-min-version=""
Sep 05 07:48:17.548223 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540607    2563 flags.go:64] FLAG: --tls-private-key-file=""
Sep 05 07:48:17.548223 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540610    2563 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 05 07:48:17.548223 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540613    2563 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 05 07:48:17.548223 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540615    2563 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 05 07:48:17.548223 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540618    2563 flags.go:64] FLAG: --v="2"
Sep 05 07:48:17.548223 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540622    2563 flags.go:64] FLAG: --version="false"
Sep 05 07:48:17.548223 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540629    2563 flags.go:64] FLAG: --vmodule=""
Sep 05 07:48:17.548223 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540633    2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 05 07:48:17.548223 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.540636    2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 05 07:48:17.548223 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540747    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 05 07:48:17.548223 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540751    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 05 07:48:17.548223 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540754    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 05 07:48:17.548223 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540757    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 05 07:48:17.548223 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540760    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 05 07:48:17.548223 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540762    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 05 07:48:17.548223 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540765    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 05 07:48:17.548223 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540767    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 05 07:48:17.548786 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540770    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 05 07:48:17.548786 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540773    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 05 07:48:17.548786 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540775    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 05 07:48:17.548786 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540779    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 05 07:48:17.548786 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540786    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 05 07:48:17.548786 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540788    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 05 07:48:17.548786 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540791    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 05 07:48:17.548786 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540794    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 05 07:48:17.548786 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540797    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 05 07:48:17.548786 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540799    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 05 07:48:17.548786 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540801    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 05 07:48:17.548786 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540804    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 05 07:48:17.548786 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540806    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 05 07:48:17.548786 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540809    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 05 07:48:17.548786 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540811    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 05 07:48:17.548786 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540814    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 05 07:48:17.548786 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540816    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 05 07:48:17.548786 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540818    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 05 07:48:17.548786 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540821    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 05 07:48:17.548786 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540824    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 05 07:48:17.549305 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540826    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 05 07:48:17.549305 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540829    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 05 07:48:17.549305 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540831    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 05 07:48:17.549305 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540834    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 05 07:48:17.549305 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540836    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 05 07:48:17.549305 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540858    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 05 07:48:17.549305 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540861    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 05 07:48:17.549305 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540863    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 05 07:48:17.549305 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540866    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 05 07:48:17.549305 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540868    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 05 07:48:17.549305 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540871    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 05 07:48:17.549305 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540874    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 05 07:48:17.549305 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540876    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 05 07:48:17.549305 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540879    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 05 07:48:17.549305 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540881    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 05 07:48:17.549305 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540885    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 05 07:48:17.549305 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540889    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 05 07:48:17.549305 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540891    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 05 07:48:17.549305 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540894    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 05 07:48:17.549305 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540897    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 05 07:48:17.549823 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540899    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 05 07:48:17.549823 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540902    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 05 07:48:17.549823 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540905    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 05 07:48:17.549823 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540907    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 05 07:48:17.549823 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540910    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 05 07:48:17.549823 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540912    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 05 07:48:17.549823 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540915    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 05 07:48:17.549823 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540919    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 05 07:48:17.549823 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540922    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 05 07:48:17.549823 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540925    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 05 07:48:17.549823 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540928    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 05 07:48:17.549823 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540930    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 05 07:48:17.549823 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540933    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 05 07:48:17.549823 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540935    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 05 07:48:17.549823 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540938    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 05 07:48:17.549823 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540942    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 05 07:48:17.549823 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540944    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 05 07:48:17.549823 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540947    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 05 07:48:17.549823 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540949    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 05 07:48:17.549823 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540952    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 05 07:48:17.550363 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540954    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 05 07:48:17.550363 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540957    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 05 07:48:17.550363 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540959    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 05 07:48:17.550363 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540962    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 05 07:48:17.550363 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540964    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 05 07:48:17.550363 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540967    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 05 07:48:17.550363 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540969    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 05 07:48:17.550363 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540973    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 05 07:48:17.550363 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540976    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 05 07:48:17.550363 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540979    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 05 07:48:17.550363 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540981    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 05 07:48:17.550363 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540984    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 05 07:48:17.550363 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540986    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 05 07:48:17.550363 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540989    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 05 07:48:17.550363 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540991    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 05 07:48:17.550363 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540993    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 05 07:48:17.550363 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540996    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 05 07:48:17.550363 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.540998    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 05 07:48:17.550363 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.541001    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 05 07:48:17.550363 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.541003    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 05 07:48:17.550862 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.541005    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 05 07:48:17.550862 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.541008    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 05 07:48:17.550862 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.541010    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 05 07:48:17.550862 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.541013    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 05 07:48:17.550862 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.541016    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 05 07:48:17.550862 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.541020    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 05 07:48:17.550862 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.541024    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 05 07:48:17.550862 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.541027    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 05 07:48:17.550862 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.541030    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 05 07:48:17.550862 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.541033    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 05 07:48:17.550862 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.541035    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 05 07:48:17.550862 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.541040    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 05 07:48:17.550862 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.548386    2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 05 07:48:17.550862 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.548402    2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 05 07:48:17.551208 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.550572    2563 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 05 07:48:17.551208 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.550580    2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 05 07:48:17.551208 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550644    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 05 07:48:17.551208 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550650    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 05 07:48:17.551208 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550653    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 05 07:48:17.551208 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550656    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 05 07:48:17.551208 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550659    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 05 07:48:17.551208 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550662    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 05 07:48:17.551208 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550665    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 05 07:48:17.551208 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550668    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 05 07:48:17.551208 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550671    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 05 07:48:17.551208 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550673    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 05 07:48:17.551208 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550676    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 05 07:48:17.551208 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550679    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 05 07:48:17.551208 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550683    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 05 07:48:17.551208 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550686    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 05 07:48:17.551208 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550689    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 05 07:48:17.551208 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550692    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 05 07:48:17.551208 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550695    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 05 07:48:17.551673 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550697    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 05 07:48:17.551673 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550700    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 05 07:48:17.551673 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550702    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 05 07:48:17.551673 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550705    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 05 07:48:17.551673 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550707    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 05 07:48:17.551673 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550710    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 05 07:48:17.551673 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550712    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 05 07:48:17.551673 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550716    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 05 07:48:17.551673 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550720    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 05 07:48:17.551673 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550724    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 05 07:48:17.551673 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550727    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 05 07:48:17.551673 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550730    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 05 07:48:17.551673 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550733    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 05 07:48:17.551673 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550735    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 05 07:48:17.551673 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550738    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 05 07:48:17.551673 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550741    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 05 07:48:17.551673 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550743    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 05 07:48:17.551673 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550746    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 05 07:48:17.551673 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550749    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 05 07:48:17.552209 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550751    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 05 07:48:17.552209 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550754    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 05 07:48:17.552209 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550756    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 05 07:48:17.552209 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550759    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 05 07:48:17.552209 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550761    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 05 07:48:17.552209 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550765    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 05 07:48:17.552209 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550768    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 05 07:48:17.552209 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550771    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 05 07:48:17.552209 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550774    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 05 07:48:17.552209 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550777    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 05 07:48:17.552209 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550779    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 05 07:48:17.552209 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550782    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 05 07:48:17.552209 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550784    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 05 07:48:17.552209 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550787    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 05 07:48:17.552209 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550789    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 05 07:48:17.552209 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550792    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 05 07:48:17.552209 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550794    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 05 07:48:17.552209 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550797    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 05 07:48:17.552209 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550800    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 05 07:48:17.552653 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550802    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 05 07:48:17.552653 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550804    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 05 07:48:17.552653 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550807    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 05 07:48:17.552653 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550810    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 05 07:48:17.552653 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550812    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 05 07:48:17.552653 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550814    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 05 07:48:17.552653 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550817    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 05 07:48:17.552653 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550820    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 05 07:48:17.552653 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550822    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 05 07:48:17.552653 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550825    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 05 07:48:17.552653 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550827    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 05 07:48:17.552653 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550830    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 05 07:48:17.552653 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550832    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 05 07:48:17.552653 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550834    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 05 07:48:17.552653 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550837    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 05 07:48:17.552653 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550851    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 05 07:48:17.552653 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550854    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 05 07:48:17.552653 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550857    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 05 07:48:17.552653 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550859    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 05 07:48:17.552653 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550862    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 05 07:48:17.553152 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550865    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 05 07:48:17.553152 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550867    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 05 07:48:17.553152 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550870    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 05 07:48:17.553152 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550872    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 05 07:48:17.553152 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550875    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 05 07:48:17.553152 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550877    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 05 07:48:17.553152 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550880    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 05 07:48:17.553152 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550883    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 05 07:48:17.553152 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550885    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 05 07:48:17.553152 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550888    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 05 07:48:17.553152 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550890    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 05 07:48:17.553152 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550892    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 05 07:48:17.553152 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550895    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 05 07:48:17.553152 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550897    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 05 07:48:17.553152 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550900    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 05 07:48:17.553152 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550903    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 05 07:48:17.553152 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550906    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 05 07:48:17.553152 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550908    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 05 07:48:17.553152 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550911    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 05 07:48:17.553152 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550913    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 05 07:48:17.553668 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550916    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 05 07:48:17.553668 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550918    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 05 07:48:17.553668 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550921    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 05 07:48:17.553668 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.550923    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 05 07:48:17.553668 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.550929    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 05 07:48:17.553668 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551054    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 05 07:48:17.553668 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551059    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 05 07:48:17.553668 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551062    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 05 07:48:17.553668 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551064    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 05 07:48:17.553668 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551067    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 05 07:48:17.553668 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551070    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 05 07:48:17.553668 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551073    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 05 07:48:17.553668 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551075    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 05 07:48:17.553668 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551077    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 05 07:48:17.553668 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551080    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 05 07:48:17.554052 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551082    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 05 07:48:17.554052 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551085    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 05 07:48:17.554052 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551087    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 05 07:48:17.554052 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551090    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 05 07:48:17.554052 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551092    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 05 07:48:17.554052 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551095    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 05 07:48:17.554052 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551097    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 05 07:48:17.554052 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551099    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 05 07:48:17.554052 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551102    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 05 07:48:17.554052 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551104    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 05 07:48:17.554052 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551106    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 05 07:48:17.554052 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551109    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 05 07:48:17.554052 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551111    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 05 07:48:17.554052 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551114    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 05 07:48:17.554052 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551116    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 05 07:48:17.554052 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551119    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 05 07:48:17.554052 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551121    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 05 07:48:17.554052 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551124    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 05 07:48:17.554052 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551126    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 05 07:48:17.554052 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551129    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 05 07:48:17.554518 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551131    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 05 07:48:17.554518 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551133    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 05 07:48:17.554518 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551136    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 05 07:48:17.554518 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551138    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 05 07:48:17.554518 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551140    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 05 07:48:17.554518 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551143    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 05 07:48:17.554518 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551145    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 05 07:48:17.554518 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551147    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 05 07:48:17.554518 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551150    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 05 07:48:17.554518 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551152    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 05 07:48:17.554518 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551155    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 05 07:48:17.554518 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551157    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 05 07:48:17.554518 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551160    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 05 07:48:17.554518 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551162    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 05 07:48:17.554518 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551165    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 05 07:48:17.554518 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551167    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 05 07:48:17.554518 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551170    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 05 07:48:17.554518 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551172    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 05 07:48:17.554518 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551175    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 05 07:48:17.554518 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551177    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 05 07:48:17.555011 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551180    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 05 07:48:17.555011 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551182    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 05 07:48:17.555011 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551185    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 05 07:48:17.555011 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551188    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 05 07:48:17.555011 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551191    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 05 07:48:17.555011 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551195    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 05 07:48:17.555011 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551198    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 05 07:48:17.555011 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551201    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 05 07:48:17.555011 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551203    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 05 07:48:17.555011 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551206    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 05 07:48:17.555011 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551209    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 05 07:48:17.555011 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551212    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 05 07:48:17.555011 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551215    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 05 07:48:17.555011 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551217    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 05 07:48:17.555011 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551220    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 05 07:48:17.555011 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551223    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 05 07:48:17.555011 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551225    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 05 07:48:17.555011 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551230    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 05 07:48:17.555011 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551233    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 05 07:48:17.555461 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551236    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 05 07:48:17.555461 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551239    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 05 07:48:17.555461 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551241    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 05 07:48:17.555461 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551244    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 05 07:48:17.555461 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551247    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 05 07:48:17.555461 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551249    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 05 07:48:17.555461 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551253    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 05 07:48:17.555461 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551256    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 05 07:48:17.555461 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551259    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 05 07:48:17.555461 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551261    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 05 07:48:17.555461 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551264    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 05 07:48:17.555461 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551266    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 05 07:48:17.555461 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551269    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 05 07:48:17.555461 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551271    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 05 07:48:17.555461 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551273    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 05 07:48:17.555461 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551276    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 05 07:48:17.555461 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551279    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 05 07:48:17.555461 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551281    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 05 07:48:17.555461 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551283    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 05 07:48:17.556016 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551286    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 05 07:48:17.556016 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551289    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 05 07:48:17.556016 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551291    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 05 07:48:17.556016 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551294    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 05 07:48:17.556016 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551296    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 05 07:48:17.556016 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551298    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 05 07:48:17.556016 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551301    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 05 07:48:17.556016 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551303    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 05 07:48:17.556016 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551305    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 05 07:48:17.556016 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551308    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 05 07:48:17.556016 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:17.551310    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 05 07:48:17.556016 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.551315    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 05 07:48:17.556016 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.551412    2563 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 05 07:48:17.556016 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.553504    2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 05 07:48:17.556016 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.554527    2563 server.go:1019] "Starting client certificate rotation"
Sep 05 07:48:17.556376 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.554626    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 05 07:48:17.556376 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.554659    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 05 07:48:17.557395 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.557381    2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 05 07:48:17.558970 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.558952    2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 05 07:48:17.570837 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.570822    2563 log.go:25] "Validated CRI v1 runtime API"
Sep 05 07:48:17.575395 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.575378    2563 log.go:25] "Validated CRI v1 image API"
Sep 05 07:48:17.576280 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.576265    2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 05 07:48:17.580486 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.580469    2563 fs.go:135] Filesystem UUIDs: map[32386d1f-a772-474f-a3b7-8d97bc42292c:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 88835c5c-15bf-4a94-ac36-08a67245fa62:/dev/nvme0n1p4]
Sep 05 07:48:17.580536 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.580487    2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 05 07:48:17.585316 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.585297    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 05 07:48:17.585539 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.585438    2563 manager.go:216] Machine: {Timestamp:2026-09-05 07:48:17.584949969 +0000 UTC m=+0.355092316 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3106586 MemoryCapacity:32812154880 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2494df7224a7e0bee9e350bcd2d769 SystemUUID:ec2494df-7224-a7e0-bee9-e350bcd2d769 BootID:63312813-c420-4841-b515-84943ae8bafe Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406077440 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:df:c4:66:db:e9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:df:c4:66:db:e9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:46:f4:f3:49:76:8c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812154880 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 05 07:48:17.585539 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.585539    2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 05 07:48:17.585648 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.585637    2563 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 05 07:48:17.588486 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.588463    2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 05 07:48:17.588621 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.588488    2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-14.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 05 07:48:17.589959 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.589950    2563 topology_manager.go:138] "Creating topology manager with none policy"
Sep 05 07:48:17.589992 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.589961    2563 container_manager_linux.go:309] "Creating device plugin manager"
Sep 05 07:48:17.589992 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.589968    2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 05 07:48:17.590894 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.590884    2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 05 07:48:17.592562 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.592553    2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 05 07:48:17.597712 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.597701    2563 state_mem.go:36] "Initialized new in-memory state store"
Sep 05 07:48:17.597824 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.597815    2563 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 05 07:48:17.597875 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.597855    2563 kubelet.go:486] "Attempting to sync node with API server"
Sep 05 07:48:17.597875 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.597866    2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 05 07:48:17.597931 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.597879    2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 05 07:48:17.597931 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.597887    2563 kubelet.go:398] "Adding apiserver pod source"
Sep 05 07:48:17.597931 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.597895    2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 05 07:48:17.598323 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.598307    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 05 07:48:17.598369 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.598333    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 05 07:48:17.602291 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.602273    2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 05 07:48:17.605208 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.605194    2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 05 07:48:17.605268 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.605223    2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 05 07:48:17.605392 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.605379    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 05 07:48:17.605434 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.605402    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 05 07:48:17.605434 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.605412    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 05 07:48:17.605434 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.605419    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 05 07:48:17.605434 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.605427    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 05 07:48:17.605434 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.605432    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 05 07:48:17.606105 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.606095    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 05 07:48:17.606105 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.606105    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 05 07:48:17.606174 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.606112    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 05 07:48:17.606174 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.606119    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 05 07:48:17.606174 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.606127    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 05 07:48:17.606174 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.606134    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 05 07:48:17.606888 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.606878    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 05 07:48:17.606888 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.606888    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 05 07:48:17.608073 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:17.608053    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 05 07:48:17.608153 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:17.608086    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-14.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 05 07:48:17.610680 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.610666    2563 server.go:1268] "Started kubelet"
Sep 05 07:48:17.610854 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.610805    2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 05 07:48:17.611113 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.610939    2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 05 07:48:17.611282 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.611264    2563 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 05 07:48:17.611494 ip-10-0-133-14 systemd[1]: Started Kubernetes Kubelet.
Sep 05 07:48:17.612789 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.612753    2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 05 07:48:17.614134 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.614116    2563 server.go:310] "Adding debug handlers to kubelet server"
Sep 05 07:48:17.615856 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:17.615812    2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 05 07:48:17.618190 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.618173    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-14.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 05 07:48:17.619050 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:17.618007    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-14.ec2.internal.18d25d381d4ce36e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-14.ec2.internal,UID:ip-10-0-133-14.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-14.ec2.internal,},FirstTimestamp:2026-09-05 07:48:17.61063819 +0000 UTC m=+0.380780518,LastTimestamp:2026-09-05 07:48:17.61063819 +0000 UTC m=+0.380780518,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-14.ec2.internal,}"
Sep 05 07:48:17.619578 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.619560    2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 05 07:48:17.619660 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.619567    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 05 07:48:17.619660 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:17.619640    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-14.ec2.internal\" not found"
Sep 05 07:48:17.620282 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.620262    2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 05 07:48:17.620282 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.620264    2563 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 05 07:48:17.620421 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.620290    2563 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 05 07:48:17.620479 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.620422    2563 reconstruct.go:97] "Volume reconstruction finished"
Sep 05 07:48:17.620479 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.620430    2563 reconciler.go:29] "Reconciler: start to sync state"
Sep 05 07:48:17.620607 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.620594    2563 factory.go:153] Registering CRI-O factory
Sep 05 07:48:17.620674 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.620610    2563 factory.go:223] Registration of the crio container factory successfully
Sep 05 07:48:17.620674 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.620657    2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 05 07:48:17.620674 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.620667    2563 factory.go:55] Registering systemd factory
Sep 05 07:48:17.620674 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.620675    2563 factory.go:223] Registration of the systemd container factory successfully
Sep 05 07:48:17.620871 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.620689    2563 factory.go:103] Registering Raw factory
Sep 05 07:48:17.620871 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.620702    2563 manager.go:1196] Started watching for new ooms in manager
Sep 05 07:48:17.621120 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.621107    2563 manager.go:349] Starting recovery of all containers
Sep 05 07:48:17.625586 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:17.625558    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 05 07:48:17.625669 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:17.625616    2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-14.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 05 07:48:17.629879 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.629861    2563 manager.go:354] Recovery completed
Sep 05 07:48:17.634231 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.634218    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 05 07:48:17.636178 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.636162    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-14.ec2.internal" event="NodeHasSufficientMemory"
Sep 05 07:48:17.636246 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.636191    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-14.ec2.internal" event="NodeHasNoDiskPressure"
Sep 05 07:48:17.636246 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.636202    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-14.ec2.internal" event="NodeHasSufficientPID"
Sep 05 07:48:17.636731 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.636715    2563 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 05 07:48:17.636731 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.636725    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 05 07:48:17.636731 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.636730    2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 05 07:48:17.636933 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.636747    2563 state_mem.go:36] "Initialized new in-memory state store"
Sep 05 07:48:17.637394 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.637378    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-14.ec2.internal" event="NodeHasSufficientMemory"
Sep 05 07:48:17.637455 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.637403    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-14.ec2.internal" event="NodeHasNoDiskPressure"
Sep 05 07:48:17.637455 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.637416    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-14.ec2.internal" event="NodeHasSufficientPID"
Sep 05 07:48:17.637653 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:17.637595    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-14.ec2.internal.18d25d381ed2980e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-14.ec2.internal,UID:ip-10-0-133-14.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-14.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-14.ec2.internal,},FirstTimestamp:2026-09-05 07:48:17.636177934 +0000 UTC m=+0.406320261,LastTimestamp:2026-09-05 07:48:17.636177934 +0000 UTC m=+0.406320261,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-14.ec2.internal,}"
Sep 05 07:48:17.638064 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.638050    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-x8zt4"
Sep 05 07:48:17.638876 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.638865    2563 policy_none.go:49] "None policy: Start"
Sep 05 07:48:17.638911 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.638881    2563 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 05 07:48:17.638911 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.638891    2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 05 07:48:17.640738 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.640727    2563 policy_none.go:47] "Start"
Sep 05 07:48:17.644975 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.644955    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-x8zt4"
Sep 05 07:48:17.647578 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:17.647296    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-14.ec2.internal.18d25d381ed2e038  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-14.ec2.internal,UID:ip-10-0-133-14.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-14.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-14.ec2.internal,},FirstTimestamp:2026-09-05 07:48:17.636196408 +0000 UTC m=+0.406338736,LastTimestamp:2026-09-05 07:48:17.636196408 +0000 UTC m=+0.406338736,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-14.ec2.internal,}"
Sep 05 07:48:17.679416 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.679398    2563 manager.go:337] "Starting Device Plugin manager"
Sep 05 07:48:17.704947 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:17.679441    2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 05 07:48:17.704947 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.679452    2563 server.go:87] "Starting device plugin registration server"
Sep 05 07:48:17.704947 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.679697    2563 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 05 07:48:17.704947 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.679707    2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 05 07:48:17.704947 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.679802    2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 05 07:48:17.704947 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.679905    2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 05 07:48:17.704947 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.679912    2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 05 07:48:17.704947 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:17.680385    2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 05 07:48:17.704947 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:17.680426    2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-14.ec2.internal\" not found"
Sep 05 07:48:17.721960 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.721933    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 05 07:48:17.723261 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.723241    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 05 07:48:17.723338 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.723282    2563 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 05 07:48:17.723338 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.723302    2563 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 05 07:48:17.723338 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:17.723334    2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 05 07:48:17.725970 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.725949    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 05 07:48:17.780062 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.779968    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 05 07:48:17.780935 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.780913    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-14.ec2.internal" event="NodeHasSufficientMemory"
Sep 05 07:48:17.781029 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.780950    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-14.ec2.internal" event="NodeHasNoDiskPressure"
Sep 05 07:48:17.781029 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.780962    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-14.ec2.internal" event="NodeHasSufficientPID"
Sep 05 07:48:17.781029 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.780984    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-14.ec2.internal"
Sep 05 07:48:17.790851 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.790824    2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-14.ec2.internal"
Sep 05 07:48:17.790911 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:17.790860    2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-14.ec2.internal\": node \"ip-10-0-133-14.ec2.internal\" not found"
Sep 05 07:48:17.817996 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:17.817976    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-14.ec2.internal\" not found"
Sep 05 07:48:17.824231 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.824212    2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-14.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-14.ec2.internal"]
Sep 05 07:48:17.824290 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.824271    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 05 07:48:17.825767 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.825750    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-14.ec2.internal" event="NodeHasSufficientMemory"
Sep 05 07:48:17.825862 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.825780    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-14.ec2.internal" event="NodeHasNoDiskPressure"
Sep 05 07:48:17.825862 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.825790    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-14.ec2.internal" event="NodeHasSufficientPID"
Sep 05 07:48:17.827941 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.827929    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 05 07:48:17.828060 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.828045    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-14.ec2.internal"
Sep 05 07:48:17.828094 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.828082    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 05 07:48:17.828618 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.828600    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-14.ec2.internal" event="NodeHasSufficientMemory"
Sep 05 07:48:17.828681 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.828625    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-14.ec2.internal" event="NodeHasNoDiskPressure"
Sep 05 07:48:17.828681 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.828634    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-14.ec2.internal" event="NodeHasSufficientPID"
Sep 05 07:48:17.828681 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.828637    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-14.ec2.internal" event="NodeHasSufficientMemory"
Sep 05 07:48:17.828681 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.828654    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-14.ec2.internal" event="NodeHasNoDiskPressure"
Sep 05 07:48:17.828681 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.828664    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-14.ec2.internal" event="NodeHasSufficientPID"
Sep 05 07:48:17.830673 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.830661    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-14.ec2.internal"
Sep 05 07:48:17.830722 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.830685    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 05 07:48:17.831328 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.831304    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-14.ec2.internal" event="NodeHasSufficientMemory"
Sep 05 07:48:17.831382 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.831338    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-14.ec2.internal" event="NodeHasNoDiskPressure"
Sep 05 07:48:17.831382 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.831350    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-14.ec2.internal" event="NodeHasSufficientPID"
Sep 05 07:48:17.843415 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:17.843397    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-14.ec2.internal\" not found" node="ip-10-0-133-14.ec2.internal"
Sep 05 07:48:17.847305 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:17.847290    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-14.ec2.internal\" not found" node="ip-10-0-133-14.ec2.internal"
Sep 05 07:48:17.918329 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:17.918306    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-14.ec2.internal\" not found"
Sep 05 07:48:17.921563 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.921546    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0ff80311f74673ce3c7117e23d0a13e1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-14.ec2.internal\" (UID: \"0ff80311f74673ce3c7117e23d0a13e1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-14.ec2.internal"
Sep 05 07:48:17.921630 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.921571    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0ff80311f74673ce3c7117e23d0a13e1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-14.ec2.internal\" (UID: \"0ff80311f74673ce3c7117e23d0a13e1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-14.ec2.internal"
Sep 05 07:48:17.921630 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:17.921598    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5bcee7ae8d0b6018ec80e28e7bc76ca5-config\") pod \"kube-apiserver-proxy-ip-10-0-133-14.ec2.internal\" (UID: \"5bcee7ae8d0b6018ec80e28e7bc76ca5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-14.ec2.internal"
Sep 05 07:48:18.019388 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:18.019347    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-14.ec2.internal\" not found"
Sep 05 07:48:18.022614 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:18.022589    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0ff80311f74673ce3c7117e23d0a13e1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-14.ec2.internal\" (UID: \"0ff80311f74673ce3c7117e23d0a13e1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-14.ec2.internal"
Sep 05 07:48:18.022660 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:18.022623    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0ff80311f74673ce3c7117e23d0a13e1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-14.ec2.internal\" (UID: \"0ff80311f74673ce3c7117e23d0a13e1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-14.ec2.internal"
Sep 05 07:48:18.022660 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:18.022640    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5bcee7ae8d0b6018ec80e28e7bc76ca5-config\") pod \"kube-apiserver-proxy-ip-10-0-133-14.ec2.internal\" (UID: \"5bcee7ae8d0b6018ec80e28e7bc76ca5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-14.ec2.internal"
Sep 05 07:48:18.022753 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:18.022735    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0ff80311f74673ce3c7117e23d0a13e1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-14.ec2.internal\" (UID: \"0ff80311f74673ce3c7117e23d0a13e1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-14.ec2.internal"
Sep 05 07:48:18.022787 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:18.022768    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0ff80311f74673ce3c7117e23d0a13e1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-14.ec2.internal\" (UID: \"0ff80311f74673ce3c7117e23d0a13e1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-14.ec2.internal"
Sep 05 07:48:18.022819 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:18.022804    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5bcee7ae8d0b6018ec80e28e7bc76ca5-config\") pod \"kube-apiserver-proxy-ip-10-0-133-14.ec2.internal\" (UID: \"5bcee7ae8d0b6018ec80e28e7bc76ca5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-14.ec2.internal"
Sep 05 07:48:18.120135 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:18.120066    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-14.ec2.internal\" not found"
Sep 05 07:48:18.145393 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:18.145369    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-14.ec2.internal"
Sep 05 07:48:18.150301 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:18.150009    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-14.ec2.internal"
Sep 05 07:48:18.220564 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:18.220517    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-14.ec2.internal\" not found"
Sep 05 07:48:18.320987 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:18.320952    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-14.ec2.internal\" not found"
Sep 05 07:48:18.421446 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:18.421372    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-14.ec2.internal\" not found"
Sep 05 07:48:18.521838 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:18.521796    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-14.ec2.internal\" not found"
Sep 05 07:48:18.554120 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:18.554094    2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 05 07:48:18.554531 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:18.554243    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 05 07:48:18.622422 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:18.622397    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-14.ec2.internal\" not found"
Sep 05 07:48:18.648045 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:18.648015    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-04 07:43:17 +0000 UTC" deadline="2028-03-17 02:11:58.663210967 +0000 UTC"
Sep 05 07:48:18.648045 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:18.648045    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13410h23m40.015172235s"
Sep 05 07:48:18.715392 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:18.715362    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 05 07:48:18.716641 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:18.716621    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-14.ec2.internal" event="NodeHasSufficientMemory"
Sep 05 07:48:18.716743 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:18.716658    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-14.ec2.internal" event="NodeHasNoDiskPressure"
Sep 05 07:48:18.716743 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:18.716671    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-14.ec2.internal" event="NodeHasSufficientPID"
Sep 05 07:48:18.722707 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:18.722515    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-14.ec2.internal\" not found"
Sep 05 07:48:18.773155 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:18.773134    2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 05 07:48:18.807018 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:18.807001    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 05 07:48:18.817311 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:18.817281    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bcee7ae8d0b6018ec80e28e7bc76ca5.slice/crio-bcf20a1e6419b35f1e1594e572a966c86bd7fed0172bb81c95f2ccde465d4039 WatchSource:0}: Error finding container bcf20a1e6419b35f1e1594e572a966c86bd7fed0172bb81c95f2ccde465d4039: Status 404 returned error can't find the container with id bcf20a1e6419b35f1e1594e572a966c86bd7fed0172bb81c95f2ccde465d4039
Sep 05 07:48:18.817479 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:18.817463    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ff80311f74673ce3c7117e23d0a13e1.slice/crio-17652c3d3b1fa8f84eee36cec480ebf381f3837f61ef22910e6a56f11461a170 WatchSource:0}: Error finding container 17652c3d3b1fa8f84eee36cec480ebf381f3837f61ef22910e6a56f11461a170: Status 404 returned error can't find the container with id 17652c3d3b1fa8f84eee36cec480ebf381f3837f61ef22910e6a56f11461a170
Sep 05 07:48:18.820436 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:18.820411    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-14.ec2.internal"
Sep 05 07:48:18.822521 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:18.822507    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 05 07:48:18.833033 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:18.833017    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 05 07:48:18.834163 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:18.834151    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-14.ec2.internal"
Sep 05 07:48:18.840187 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:18.840171    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 05 07:48:19.206125 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.206092    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 05 07:48:19.522133 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.522050    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 05 07:48:19.599147 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.599027    2563 apiserver.go:52] "Watching apiserver"
Sep 05 07:48:19.608761 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.608511    2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 05 07:48:19.609752 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.609447    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-133-14.ec2.internal","openshift-dns/node-resolver-fstjt","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-14.ec2.internal","openshift-multus/multus-additional-cni-plugins-cd8wl","openshift-network-diagnostics/network-check-target-rbtht","openshift-network-operator/iptables-alerter-mvqcl","kube-system/global-pull-secret-syncer-lx5db","kube-system/konnectivity-agent-8phbx","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d26m8","openshift-cluster-node-tuning-operator/tuned-kqd45","openshift-image-registry/node-ca-m7775","openshift-multus/multus-cvxlh","openshift-multus/network-metrics-daemon-l9lgq","openshift-ovn-kubernetes/ovnkube-node-2bzp8"]
Sep 05 07:48:19.612050 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.611968    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lx5db"
Sep 05 07:48:19.612156 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:19.612048    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lx5db" podUID="7e352b34-8143-4a94-86e7-11a9460d2946"
Sep 05 07:48:19.614900 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.614851    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9lgq"
Sep 05 07:48:19.615030 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:19.614964    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9lgq" podUID="404c81cf-16aa-49f0-9059-fb97a5791ed7"
Sep 05 07:48:19.617857 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.617817    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.619281 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.619197    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cd8wl"
Sep 05 07:48:19.620309 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.620285    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 05 07:48:19.620523 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.620503    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-bxzsd\""
Sep 05 07:48:19.620800 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.620779    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 05 07:48:19.621798 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.621773    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 05 07:48:19.621925 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.621825    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 05 07:48:19.622014 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.621987    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 05 07:48:19.622118 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.622096    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 05 07:48:19.622349 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.622332    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 05 07:48:19.622435 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.622407    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 05 07:48:19.622972 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.622633    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 05 07:48:19.622972 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.622697    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 05 07:48:19.622972 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.622718    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-j5b74\""
Sep 05 07:48:19.622972 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.622640    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 05 07:48:19.622972 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.622957    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rbtht"
Sep 05 07:48:19.623318 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:19.623011    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rbtht" podUID="b1e5c942-8397-439a-96b6-13149a974b18"
Sep 05 07:48:19.623318 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.622957    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 05 07:48:19.624070 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.624029    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mvqcl"
Sep 05 07:48:19.626094 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.626072    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 05 07:48:19.626699 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.626678    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 05 07:48:19.626937 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.626907    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-fc4kx\""
Sep 05 07:48:19.627106 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.627089    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 05 07:48:19.630179 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.629896    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d26m8"
Sep 05 07:48:19.631404 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.631072    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/569cffae-1c5c-4b02-af57-9ca53c455e4c-ovn-node-metrics-cert\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.631404 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.631124    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-run-systemd\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.631404 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.631152    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-run-ovn\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.631404 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.631180    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wt7mt\" (UniqueName: \"kubernetes.io/projected/b1e5c942-8397-439a-96b6-13149a974b18-kube-api-access-wt7mt\") pod \"network-check-target-rbtht\" (UID: \"b1e5c942-8397-439a-96b6-13149a974b18\") " pod="openshift-network-diagnostics/network-check-target-rbtht"
Sep 05 07:48:19.631404 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.631206    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-run-openvswitch\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.631404 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.631249    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/569cffae-1c5c-4b02-af57-9ca53c455e4c-env-overrides\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.631404 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.631276    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4d02d57b-ec85-4cd4-8da2-0ffee0574984-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cd8wl\" (UID: \"4d02d57b-ec85-4cd4-8da2-0ffee0574984\") " pod="openshift-multus/multus-additional-cni-plugins-cd8wl"
Sep 05 07:48:19.631404 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.631303    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nf55b\" (UniqueName: \"kubernetes.io/projected/4d02d57b-ec85-4cd4-8da2-0ffee0574984-kube-api-access-nf55b\") pod \"multus-additional-cni-plugins-cd8wl\" (UID: \"4d02d57b-ec85-4cd4-8da2-0ffee0574984\") " pod="openshift-multus/multus-additional-cni-plugins-cd8wl"
Sep 05 07:48:19.631404 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.631327    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-etc-openvswitch\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.631404 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.631350    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/569cffae-1c5c-4b02-af57-9ca53c455e4c-ovnkube-config\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.631404 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.631378    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4d02d57b-ec85-4cd4-8da2-0ffee0574984-system-cni-dir\") pod \"multus-additional-cni-plugins-cd8wl\" (UID: \"4d02d57b-ec85-4cd4-8da2-0ffee0574984\") " pod="openshift-multus/multus-additional-cni-plugins-cd8wl"
Sep 05 07:48:19.632125 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.631417    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4d02d57b-ec85-4cd4-8da2-0ffee0574984-cnibin\") pod \"multus-additional-cni-plugins-cd8wl\" (UID: \"4d02d57b-ec85-4cd4-8da2-0ffee0574984\") " pod="openshift-multus/multus-additional-cni-plugins-cd8wl"
Sep 05 07:48:19.632125 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.631468    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4d02d57b-ec85-4cd4-8da2-0ffee0574984-cni-binary-copy\") pod \"multus-additional-cni-plugins-cd8wl\" (UID: \"4d02d57b-ec85-4cd4-8da2-0ffee0574984\") " pod="openshift-multus/multus-additional-cni-plugins-cd8wl"
Sep 05 07:48:19.632125 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.631514    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/404c81cf-16aa-49f0-9059-fb97a5791ed7-metrics-certs\") pod \"network-metrics-daemon-l9lgq\" (UID: \"404c81cf-16aa-49f0-9059-fb97a5791ed7\") " pod="openshift-multus/network-metrics-daemon-l9lgq"
Sep 05 07:48:19.632125 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.631545    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-host-kubelet\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.632125 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.631570    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-host-run-netns\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.632125 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.631600    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.632125 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.631630    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e352b34-8143-4a94-86e7-11a9460d2946-original-pull-secret\") pod \"global-pull-secret-syncer-lx5db\" (UID: \"7e352b34-8143-4a94-86e7-11a9460d2946\") " pod="kube-system/global-pull-secret-syncer-lx5db"
Sep 05 07:48:19.632125 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.631654    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-systemd-units\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.632125 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.631698    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-host-slash\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.632125 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.631733    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-log-socket\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.632125 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.631756    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-node-log\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.632125 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.631781    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-host-run-ovn-kubernetes\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.632125 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.631804    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-host-cni-bin\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.632125 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.631827    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-host-cni-netd\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.632125 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.631849    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/569cffae-1c5c-4b02-af57-9ca53c455e4c-ovnkube-script-lib\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.632125 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.631920    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt4ll\" (UniqueName: \"kubernetes.io/projected/569cffae-1c5c-4b02-af57-9ca53c455e4c-kube-api-access-mt4ll\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.632750 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.631946    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4d02d57b-ec85-4cd4-8da2-0ffee0574984-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cd8wl\" (UID: \"4d02d57b-ec85-4cd4-8da2-0ffee0574984\") " pod="openshift-multus/multus-additional-cni-plugins-cd8wl"
Sep 05 07:48:19.632750 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.631994    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4d02d57b-ec85-4cd4-8da2-0ffee0574984-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cd8wl\" (UID: \"4d02d57b-ec85-4cd4-8da2-0ffee0574984\") " pod="openshift-multus/multus-additional-cni-plugins-cd8wl"
Sep 05 07:48:19.632750 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.632027    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7e352b34-8143-4a94-86e7-11a9460d2946-kubelet-config\") pod \"global-pull-secret-syncer-lx5db\" (UID: \"7e352b34-8143-4a94-86e7-11a9460d2946\") " pod="kube-system/global-pull-secret-syncer-lx5db"
Sep 05 07:48:19.632750 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.632060    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7e352b34-8143-4a94-86e7-11a9460d2946-dbus\") pod \"global-pull-secret-syncer-lx5db\" (UID: \"7e352b34-8143-4a94-86e7-11a9460d2946\") " pod="kube-system/global-pull-secret-syncer-lx5db"
Sep 05 07:48:19.632750 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.632097    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n86hz\" (UniqueName: \"kubernetes.io/projected/404c81cf-16aa-49f0-9059-fb97a5791ed7-kube-api-access-n86hz\") pod \"network-metrics-daemon-l9lgq\" (UID: \"404c81cf-16aa-49f0-9059-fb97a5791ed7\") " pod="openshift-multus/network-metrics-daemon-l9lgq"
Sep 05 07:48:19.632750 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.632125    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-var-lib-openvswitch\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.632750 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.632347    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8phbx"
Sep 05 07:48:19.636431 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.633350    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-s45kq\""
Sep 05 07:48:19.636431 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.633610    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 05 07:48:19.636431 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.633651    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 05 07:48:19.636431 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.634709    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 05 07:48:19.636431 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.635483    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 05 07:48:19.636431 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.635625    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fstjt"
Sep 05 07:48:19.636431 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.636001    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-crlpv\""
Sep 05 07:48:19.636431 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.636040    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 05 07:48:19.638601 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.637501    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.638601 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.638564    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-7mc4j\""
Sep 05 07:48:19.638887 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.638800    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 05 07:48:19.642592 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.639073    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 05 07:48:19.642592 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.639602    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 05 07:48:19.642592 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.640608    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 05 07:48:19.642592 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.640790    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 05 07:48:19.642592 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.640840    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-m7775"
Sep 05 07:48:19.642592 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.641072    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-l7hb5\""
Sep 05 07:48:19.643081 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.642823    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.643398 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.643304    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 05 07:48:19.643398 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.643327    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 05 07:48:19.643948 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.643738    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 05 07:48:19.643948 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.643881    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vrbgv\""
Sep 05 07:48:19.645475 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.645454    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 05 07:48:19.645791 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.645770    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rtrjr\""
Sep 05 07:48:19.662668 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.662648    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-p494d"
Sep 05 07:48:19.672790 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.672758    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-p494d"
Sep 05 07:48:19.721344 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.721324    2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 05 07:48:19.730820 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.730743    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-14.ec2.internal" event={"ID":"5bcee7ae8d0b6018ec80e28e7bc76ca5","Type":"ContainerStarted","Data":"bcf20a1e6419b35f1e1594e572a966c86bd7fed0172bb81c95f2ccde465d4039"}
Sep 05 07:48:19.732538 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.732511    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e352b34-8143-4a94-86e7-11a9460d2946-original-pull-secret\") pod \"global-pull-secret-syncer-lx5db\" (UID: \"7e352b34-8143-4a94-86e7-11a9460d2946\") " pod="kube-system/global-pull-secret-syncer-lx5db"
Sep 05 07:48:19.732640 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.732557    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-systemd-units\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.732640 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.732583    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-host-slash\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.732640 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.732608    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-log-socket\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.732803 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.732639    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kp8jv\" (UniqueName: \"kubernetes.io/projected/5ff68b54-382f-46b6-8ee5-3d55b4aa0444-kube-api-access-kp8jv\") pod \"iptables-alerter-mvqcl\" (UID: \"5ff68b54-382f-46b6-8ee5-3d55b4aa0444\") " pod="openshift-network-operator/iptables-alerter-mvqcl"
Sep 05 07:48:19.732803 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.732666    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3e67240a-02be-4224-b626-99289ca9f76d-etc-modprobe-d\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.732803 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.732689    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3e67240a-02be-4224-b626-99289ca9f76d-etc-sysctl-d\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.732803 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.732713    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e67240a-02be-4224-b626-99289ca9f76d-var-lib-kubelet\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.732803 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.732746    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-node-log\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.732803 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.732771    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-host-cni-bin\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.732803 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.732794    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-host-cni-netd\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.733165 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.732820    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/569cffae-1c5c-4b02-af57-9ca53c455e4c-ovnkube-script-lib\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.733165 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.732845    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt4ll\" (UniqueName: \"kubernetes.io/projected/569cffae-1c5c-4b02-af57-9ca53c455e4c-kube-api-access-mt4ll\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.733165 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.732870    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4d02d57b-ec85-4cd4-8da2-0ffee0574984-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cd8wl\" (UID: \"4d02d57b-ec85-4cd4-8da2-0ffee0574984\") " pod="openshift-multus/multus-additional-cni-plugins-cd8wl"
Sep 05 07:48:19.733165 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.732913    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f08f40f6-ff0d-419e-9268-fe051987663e-multus-cni-dir\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.733165 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.732949    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7e352b34-8143-4a94-86e7-11a9460d2946-kubelet-config\") pod \"global-pull-secret-syncer-lx5db\" (UID: \"7e352b34-8143-4a94-86e7-11a9460d2946\") " pod="kube-system/global-pull-secret-syncer-lx5db"
Sep 05 07:48:19.733165 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.732975    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnswt\" (UniqueName: \"kubernetes.io/projected/8ab06bbb-6881-4038-b482-327ff71f7bda-kube-api-access-gnswt\") pod \"aws-ebs-csi-driver-node-d26m8\" (UID: \"8ab06bbb-6881-4038-b482-327ff71f7bda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d26m8"
Sep 05 07:48:19.733165 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.733001    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3e67240a-02be-4224-b626-99289ca9f76d-sys\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.733165 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.733025    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f08f40f6-ff0d-419e-9268-fe051987663e-cni-binary-copy\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.733165 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.733059    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/569cffae-1c5c-4b02-af57-9ca53c455e4c-ovn-node-metrics-cert\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.733165 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.733084    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8ab06bbb-6881-4038-b482-327ff71f7bda-sys-fs\") pod \"aws-ebs-csi-driver-node-d26m8\" (UID: \"8ab06bbb-6881-4038-b482-327ff71f7bda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d26m8"
Sep 05 07:48:19.733165 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.733107    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3e67240a-02be-4224-b626-99289ca9f76d-etc-tuned\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.733165 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.733130    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f08f40f6-ff0d-419e-9268-fe051987663e-cnibin\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.733165 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.733153    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f08f40f6-ff0d-419e-9268-fe051987663e-host-var-lib-cni-bin\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.733702 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.733179    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-run-ovn\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.733702 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.733203    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f08f40f6-ff0d-419e-9268-fe051987663e-host-run-netns\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.733702 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.733226    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f08f40f6-ff0d-419e-9268-fe051987663e-host-var-lib-kubelet\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.733702 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.733251    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f08f40f6-ff0d-419e-9268-fe051987663e-multus-daemon-config\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.733702 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.733275    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f08f40f6-ff0d-419e-9268-fe051987663e-host-run-multus-certs\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.733702 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.733301    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/569cffae-1c5c-4b02-af57-9ca53c455e4c-env-overrides\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.733702 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.733335    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4d02d57b-ec85-4cd4-8da2-0ffee0574984-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cd8wl\" (UID: \"4d02d57b-ec85-4cd4-8da2-0ffee0574984\") " pod="openshift-multus/multus-additional-cni-plugins-cd8wl"
Sep 05 07:48:19.733702 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.733362    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sgcg\" (UniqueName: \"kubernetes.io/projected/3e67240a-02be-4224-b626-99289ca9f76d-kube-api-access-8sgcg\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.733702 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.733386    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ljvk\" (UniqueName: \"kubernetes.io/projected/35c73ae3-8ad8-443c-ac44-c9cd3396e526-kube-api-access-8ljvk\") pod \"node-ca-m7775\" (UID: \"35c73ae3-8ad8-443c-ac44-c9cd3396e526\") " pod="openshift-image-registry/node-ca-m7775"
Sep 05 07:48:19.733702 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.733412    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/155f061b-e464-41b8-bf23-4183a33dd370-agent-certs\") pod \"konnectivity-agent-8phbx\" (UID: \"155f061b-e464-41b8-bf23-4183a33dd370\") " pod="kube-system/konnectivity-agent-8phbx"
Sep 05 07:48:19.733702 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.733463    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/569cffae-1c5c-4b02-af57-9ca53c455e4c-ovnkube-config\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.733702 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.733511    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4d02d57b-ec85-4cd4-8da2-0ffee0574984-cnibin\") pod \"multus-additional-cni-plugins-cd8wl\" (UID: \"4d02d57b-ec85-4cd4-8da2-0ffee0574984\") " pod="openshift-multus/multus-additional-cni-plugins-cd8wl"
Sep 05 07:48:19.733702 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.733537    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e67240a-02be-4224-b626-99289ca9f76d-etc-kubernetes\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.733702 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.733561    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/35c73ae3-8ad8-443c-ac44-c9cd3396e526-serviceca\") pod \"node-ca-m7775\" (UID: \"35c73ae3-8ad8-443c-ac44-c9cd3396e526\") " pod="openshift-image-registry/node-ca-m7775"
Sep 05 07:48:19.733702 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.733597    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/404c81cf-16aa-49f0-9059-fb97a5791ed7-metrics-certs\") pod \"network-metrics-daemon-l9lgq\" (UID: \"404c81cf-16aa-49f0-9059-fb97a5791ed7\") " pod="openshift-multus/network-metrics-daemon-l9lgq"
Sep 05 07:48:19.733702 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.733633    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-host-kubelet\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.733702 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.733659    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.734267 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.733684    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3e67240a-02be-4224-b626-99289ca9f76d-etc-sysconfig\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.734267 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.733714    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8pcf\" (UniqueName: \"kubernetes.io/projected/f0fd7e0b-f07e-43a2-a169-667bc753a36b-kube-api-access-s8pcf\") pod \"node-resolver-fstjt\" (UID: \"f0fd7e0b-f07e-43a2-a169-667bc753a36b\") " pod="openshift-dns/node-resolver-fstjt"
Sep 05 07:48:19.734267 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.733738    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3e67240a-02be-4224-b626-99289ca9f76d-etc-sysctl-conf\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.734267 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.733782    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f0fd7e0b-f07e-43a2-a169-667bc753a36b-hosts-file\") pod \"node-resolver-fstjt\" (UID: \"f0fd7e0b-f07e-43a2-a169-667bc753a36b\") " pod="openshift-dns/node-resolver-fstjt"
Sep 05 07:48:19.734267 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.733808    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-host-run-ovn-kubernetes\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.734267 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.733837    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4d02d57b-ec85-4cd4-8da2-0ffee0574984-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cd8wl\" (UID: \"4d02d57b-ec85-4cd4-8da2-0ffee0574984\") " pod="openshift-multus/multus-additional-cni-plugins-cd8wl"
Sep 05 07:48:19.734267 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.733862    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8ab06bbb-6881-4038-b482-327ff71f7bda-socket-dir\") pod \"aws-ebs-csi-driver-node-d26m8\" (UID: \"8ab06bbb-6881-4038-b482-327ff71f7bda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d26m8"
Sep 05 07:48:19.734267 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.733897    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0fd7e0b-f07e-43a2-a169-667bc753a36b-tmp-dir\") pod \"node-resolver-fstjt\" (UID: \"f0fd7e0b-f07e-43a2-a169-667bc753a36b\") " pod="openshift-dns/node-resolver-fstjt"
Sep 05 07:48:19.734267 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.733964    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f08f40f6-ff0d-419e-9268-fe051987663e-hostroot\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.734267 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.733992    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7e352b34-8143-4a94-86e7-11a9460d2946-dbus\") pod \"global-pull-secret-syncer-lx5db\" (UID: \"7e352b34-8143-4a94-86e7-11a9460d2946\") " pod="kube-system/global-pull-secret-syncer-lx5db"
Sep 05 07:48:19.734267 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.734023    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n86hz\" (UniqueName: \"kubernetes.io/projected/404c81cf-16aa-49f0-9059-fb97a5791ed7-kube-api-access-n86hz\") pod \"network-metrics-daemon-l9lgq\" (UID: \"404c81cf-16aa-49f0-9059-fb97a5791ed7\") " pod="openshift-multus/network-metrics-daemon-l9lgq"
Sep 05 07:48:19.734267 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.734048    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8ab06bbb-6881-4038-b482-327ff71f7bda-device-dir\") pod \"aws-ebs-csi-driver-node-d26m8\" (UID: \"8ab06bbb-6881-4038-b482-327ff71f7bda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d26m8"
Sep 05 07:48:19.734267 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.734072    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3e67240a-02be-4224-b626-99289ca9f76d-lib-modules\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.734267 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.734095    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/35c73ae3-8ad8-443c-ac44-c9cd3396e526-host\") pod \"node-ca-m7775\" (UID: \"35c73ae3-8ad8-443c-ac44-c9cd3396e526\") " pod="openshift-image-registry/node-ca-m7775"
Sep 05 07:48:19.734267 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.734119    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f08f40f6-ff0d-419e-9268-fe051987663e-etc-kubernetes\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.734267 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.734155    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-var-lib-openvswitch\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.734267 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.734180    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5ff68b54-382f-46b6-8ee5-3d55b4aa0444-host-slash\") pod \"iptables-alerter-mvqcl\" (UID: \"5ff68b54-382f-46b6-8ee5-3d55b4aa0444\") " pod="openshift-network-operator/iptables-alerter-mvqcl"
Sep 05 07:48:19.734812 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.734205    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8ab06bbb-6881-4038-b482-327ff71f7bda-etc-selinux\") pod \"aws-ebs-csi-driver-node-d26m8\" (UID: \"8ab06bbb-6881-4038-b482-327ff71f7bda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d26m8"
Sep 05 07:48:19.734812 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.734235    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f08f40f6-ff0d-419e-9268-fe051987663e-host-var-lib-cni-multus\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.734812 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.734275    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-run-systemd\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.734812 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.734309    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wt7mt\" (UniqueName: \"kubernetes.io/projected/b1e5c942-8397-439a-96b6-13149a974b18-kube-api-access-wt7mt\") pod \"network-check-target-rbtht\" (UID: \"b1e5c942-8397-439a-96b6-13149a974b18\") " pod="openshift-network-diagnostics/network-check-target-rbtht"
Sep 05 07:48:19.734812 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.734347    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8ab06bbb-6881-4038-b482-327ff71f7bda-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d26m8\" (UID: \"8ab06bbb-6881-4038-b482-327ff71f7bda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d26m8"
Sep 05 07:48:19.734812 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.734374    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-run-openvswitch\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.734812 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.734401    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nf55b\" (UniqueName: \"kubernetes.io/projected/4d02d57b-ec85-4cd4-8da2-0ffee0574984-kube-api-access-nf55b\") pod \"multus-additional-cni-plugins-cd8wl\" (UID: \"4d02d57b-ec85-4cd4-8da2-0ffee0574984\") " pod="openshift-multus/multus-additional-cni-plugins-cd8wl"
Sep 05 07:48:19.734812 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.734405    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-14.ec2.internal" event={"ID":"0ff80311f74673ce3c7117e23d0a13e1","Type":"ContainerStarted","Data":"17652c3d3b1fa8f84eee36cec480ebf381f3837f61ef22910e6a56f11461a170"}
Sep 05 07:48:19.734812 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.734426    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3e67240a-02be-4224-b626-99289ca9f76d-etc-systemd\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.734812 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.734450    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3e67240a-02be-4224-b626-99289ca9f76d-run\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.734812 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.734488    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f08f40f6-ff0d-419e-9268-fe051987663e-host-run-k8s-cni-cncf-io\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.734812 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.734521    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx6rj\" (UniqueName: \"kubernetes.io/projected/f08f40f6-ff0d-419e-9268-fe051987663e-kube-api-access-cx6rj\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.734812 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.734548    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-etc-openvswitch\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.734812 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.734577    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4d02d57b-ec85-4cd4-8da2-0ffee0574984-system-cni-dir\") pod \"multus-additional-cni-plugins-cd8wl\" (UID: \"4d02d57b-ec85-4cd4-8da2-0ffee0574984\") " pod="openshift-multus/multus-additional-cni-plugins-cd8wl"
Sep 05 07:48:19.734812 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.734604    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4d02d57b-ec85-4cd4-8da2-0ffee0574984-cni-binary-copy\") pod \"multus-additional-cni-plugins-cd8wl\" (UID: \"4d02d57b-ec85-4cd4-8da2-0ffee0574984\") " pod="openshift-multus/multus-additional-cni-plugins-cd8wl"
Sep 05 07:48:19.734812 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.734629    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5ff68b54-382f-46b6-8ee5-3d55b4aa0444-iptables-alerter-script\") pod \"iptables-alerter-mvqcl\" (UID: \"5ff68b54-382f-46b6-8ee5-3d55b4aa0444\") " pod="openshift-network-operator/iptables-alerter-mvqcl"
Sep 05 07:48:19.735442 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.734662    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8ab06bbb-6881-4038-b482-327ff71f7bda-registration-dir\") pod \"aws-ebs-csi-driver-node-d26m8\" (UID: \"8ab06bbb-6881-4038-b482-327ff71f7bda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d26m8"
Sep 05 07:48:19.735442 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.734687    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3e67240a-02be-4224-b626-99289ca9f76d-host\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.735442 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.734711    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f08f40f6-ff0d-419e-9268-fe051987663e-multus-socket-dir-parent\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.735442 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.734738    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f08f40f6-ff0d-419e-9268-fe051987663e-multus-conf-dir\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.735442 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.734766    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-host-run-netns\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.735442 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.734792    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e67240a-02be-4224-b626-99289ca9f76d-tmp\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.735442 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.734818    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/155f061b-e464-41b8-bf23-4183a33dd370-konnectivity-ca\") pod \"konnectivity-agent-8phbx\" (UID: \"155f061b-e464-41b8-bf23-4183a33dd370\") " pod="kube-system/konnectivity-agent-8phbx"
Sep 05 07:48:19.735442 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.734847    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f08f40f6-ff0d-419e-9268-fe051987663e-system-cni-dir\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.735442 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.734964    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-host-kubelet\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.735442 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.735017    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.735442 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.735102    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-host-run-ovn-kubernetes\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.735442 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.735242    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4d02d57b-ec85-4cd4-8da2-0ffee0574984-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cd8wl\" (UID: \"4d02d57b-ec85-4cd4-8da2-0ffee0574984\") " pod="openshift-multus/multus-additional-cni-plugins-cd8wl"
Sep 05 07:48:19.735880 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.735450    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7e352b34-8143-4a94-86e7-11a9460d2946-dbus\") pod \"global-pull-secret-syncer-lx5db\" (UID: \"7e352b34-8143-4a94-86e7-11a9460d2946\") " pod="kube-system/global-pull-secret-syncer-lx5db"
Sep 05 07:48:19.735880 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.735480    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7e352b34-8143-4a94-86e7-11a9460d2946-kubelet-config\") pod \"global-pull-secret-syncer-lx5db\" (UID: \"7e352b34-8143-4a94-86e7-11a9460d2946\") " pod="kube-system/global-pull-secret-syncer-lx5db"
Sep 05 07:48:19.735880 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.735764    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-var-lib-openvswitch\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.735880 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.735797    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-systemd-units\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.735880 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.735861    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-run-systemd\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.736153 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.735912    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-log-socket\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.736153 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.736008    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-node-log\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.736153 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.736049    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-host-cni-bin\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.736153 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.736091    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-host-cni-netd\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.736353 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.736159    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-run-openvswitch\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.736353 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.736255    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-etc-openvswitch\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.736353 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:19.734548    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 07:48:19.736353 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.736309    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/569cffae-1c5c-4b02-af57-9ca53c455e4c-ovnkube-script-lib\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.736597 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.735851    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-host-slash\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.736597 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:19.736393    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/404c81cf-16aa-49f0-9059-fb97a5791ed7-metrics-certs podName:404c81cf-16aa-49f0-9059-fb97a5791ed7 nodeName:}" failed. No retries permitted until 2026-09-05 07:48:20.236369731 +0000 UTC m=+3.006512053 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/404c81cf-16aa-49f0-9059-fb97a5791ed7-metrics-certs") pod "network-metrics-daemon-l9lgq" (UID: "404c81cf-16aa-49f0-9059-fb97a5791ed7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 07:48:19.736597 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:19.735745    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 05 07:48:19.736597 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:19.736452    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e352b34-8143-4a94-86e7-11a9460d2946-original-pull-secret podName:7e352b34-8143-4a94-86e7-11a9460d2946 nodeName:}" failed. No retries permitted until 2026-09-05 07:48:20.236442736 +0000 UTC m=+3.006585057 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7e352b34-8143-4a94-86e7-11a9460d2946-original-pull-secret") pod "global-pull-secret-syncer-lx5db" (UID: "7e352b34-8143-4a94-86e7-11a9460d2946") : object "kube-system"/"original-pull-secret" not registered
Sep 05 07:48:19.736597 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.736413    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-host-run-netns\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.736597 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.736545    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4d02d57b-ec85-4cd4-8da2-0ffee0574984-cnibin\") pod \"multus-additional-cni-plugins-cd8wl\" (UID: \"4d02d57b-ec85-4cd4-8da2-0ffee0574984\") " pod="openshift-multus/multus-additional-cni-plugins-cd8wl"
Sep 05 07:48:19.736944 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.736598    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/569cffae-1c5c-4b02-af57-9ca53c455e4c-run-ovn\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.736944 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.736666    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4d02d57b-ec85-4cd4-8da2-0ffee0574984-system-cni-dir\") pod \"multus-additional-cni-plugins-cd8wl\" (UID: \"4d02d57b-ec85-4cd4-8da2-0ffee0574984\") " pod="openshift-multus/multus-additional-cni-plugins-cd8wl"
Sep 05 07:48:19.736944 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.736884    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/569cffae-1c5c-4b02-af57-9ca53c455e4c-ovnkube-config\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.737176 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.737154    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4d02d57b-ec85-4cd4-8da2-0ffee0574984-cni-binary-copy\") pod \"multus-additional-cni-plugins-cd8wl\" (UID: \"4d02d57b-ec85-4cd4-8da2-0ffee0574984\") " pod="openshift-multus/multus-additional-cni-plugins-cd8wl"
Sep 05 07:48:19.737293 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.737273    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/569cffae-1c5c-4b02-af57-9ca53c455e4c-env-overrides\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.737362 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.737282    2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 05 07:48:19.737362 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.737344    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4d02d57b-ec85-4cd4-8da2-0ffee0574984-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cd8wl\" (UID: \"4d02d57b-ec85-4cd4-8da2-0ffee0574984\") " pod="openshift-multus/multus-additional-cni-plugins-cd8wl"
Sep 05 07:48:19.738154 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.737818    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4d02d57b-ec85-4cd4-8da2-0ffee0574984-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cd8wl\" (UID: \"4d02d57b-ec85-4cd4-8da2-0ffee0574984\") " pod="openshift-multus/multus-additional-cni-plugins-cd8wl"
Sep 05 07:48:19.742698 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.742636    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/569cffae-1c5c-4b02-af57-9ca53c455e4c-ovn-node-metrics-cert\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.748845 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:19.748594    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 05 07:48:19.748845 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:19.748615    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 05 07:48:19.748845 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:19.748640    2563 projected.go:196] Error preparing data for projected volume kube-api-access-wt7mt for pod openshift-network-diagnostics/network-check-target-rbtht: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 07:48:19.748845 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:19.748688    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b1e5c942-8397-439a-96b6-13149a974b18-kube-api-access-wt7mt podName:b1e5c942-8397-439a-96b6-13149a974b18 nodeName:}" failed. No retries permitted until 2026-09-05 07:48:20.248675673 +0000 UTC m=+3.018817991 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wt7mt" (UniqueName: "kubernetes.io/projected/b1e5c942-8397-439a-96b6-13149a974b18-kube-api-access-wt7mt") pod "network-check-target-rbtht" (UID: "b1e5c942-8397-439a-96b6-13149a974b18") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 07:48:19.754029 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.754005    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nf55b\" (UniqueName: \"kubernetes.io/projected/4d02d57b-ec85-4cd4-8da2-0ffee0574984-kube-api-access-nf55b\") pod \"multus-additional-cni-plugins-cd8wl\" (UID: \"4d02d57b-ec85-4cd4-8da2-0ffee0574984\") " pod="openshift-multus/multus-additional-cni-plugins-cd8wl"
Sep 05 07:48:19.754246 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.754220    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n86hz\" (UniqueName: \"kubernetes.io/projected/404c81cf-16aa-49f0-9059-fb97a5791ed7-kube-api-access-n86hz\") pod \"network-metrics-daemon-l9lgq\" (UID: \"404c81cf-16aa-49f0-9059-fb97a5791ed7\") " pod="openshift-multus/network-metrics-daemon-l9lgq"
Sep 05 07:48:19.761556 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.761521    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt4ll\" (UniqueName: \"kubernetes.io/projected/569cffae-1c5c-4b02-af57-9ca53c455e4c-kube-api-access-mt4ll\") pod \"ovnkube-node-2bzp8\" (UID: \"569cffae-1c5c-4b02-af57-9ca53c455e4c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.835355 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.835273    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3e67240a-02be-4224-b626-99289ca9f76d-lib-modules\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.835355 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.835314    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/35c73ae3-8ad8-443c-ac44-c9cd3396e526-host\") pod \"node-ca-m7775\" (UID: \"35c73ae3-8ad8-443c-ac44-c9cd3396e526\") " pod="openshift-image-registry/node-ca-m7775"
Sep 05 07:48:19.835355 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.835336    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f08f40f6-ff0d-419e-9268-fe051987663e-etc-kubernetes\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.835609 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.835362    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5ff68b54-382f-46b6-8ee5-3d55b4aa0444-host-slash\") pod \"iptables-alerter-mvqcl\" (UID: \"5ff68b54-382f-46b6-8ee5-3d55b4aa0444\") " pod="openshift-network-operator/iptables-alerter-mvqcl"
Sep 05 07:48:19.835609 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.835412    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5ff68b54-382f-46b6-8ee5-3d55b4aa0444-host-slash\") pod \"iptables-alerter-mvqcl\" (UID: \"5ff68b54-382f-46b6-8ee5-3d55b4aa0444\") " pod="openshift-network-operator/iptables-alerter-mvqcl"
Sep 05 07:48:19.835609 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.835412    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f08f40f6-ff0d-419e-9268-fe051987663e-etc-kubernetes\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.835609 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.835412    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/35c73ae3-8ad8-443c-ac44-c9cd3396e526-host\") pod \"node-ca-m7775\" (UID: \"35c73ae3-8ad8-443c-ac44-c9cd3396e526\") " pod="openshift-image-registry/node-ca-m7775"
Sep 05 07:48:19.835609 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.835463    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3e67240a-02be-4224-b626-99289ca9f76d-lib-modules\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.835609 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.835469    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8ab06bbb-6881-4038-b482-327ff71f7bda-etc-selinux\") pod \"aws-ebs-csi-driver-node-d26m8\" (UID: \"8ab06bbb-6881-4038-b482-327ff71f7bda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d26m8"
Sep 05 07:48:19.835609 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.835504    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f08f40f6-ff0d-419e-9268-fe051987663e-host-var-lib-cni-multus\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.835609 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.835548    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8ab06bbb-6881-4038-b482-327ff71f7bda-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d26m8\" (UID: \"8ab06bbb-6881-4038-b482-327ff71f7bda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d26m8"
Sep 05 07:48:19.835609 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.835569    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8ab06bbb-6881-4038-b482-327ff71f7bda-etc-selinux\") pod \"aws-ebs-csi-driver-node-d26m8\" (UID: \"8ab06bbb-6881-4038-b482-327ff71f7bda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d26m8"
Sep 05 07:48:19.835609 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.835574    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3e67240a-02be-4224-b626-99289ca9f76d-etc-systemd\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.835609 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.835598    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f08f40f6-ff0d-419e-9268-fe051987663e-host-var-lib-cni-multus\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.835609 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.835599    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3e67240a-02be-4224-b626-99289ca9f76d-run\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.836243 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.835637    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3e67240a-02be-4224-b626-99289ca9f76d-run\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.836243 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.835631    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f08f40f6-ff0d-419e-9268-fe051987663e-host-run-k8s-cni-cncf-io\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.836243 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.835654    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8ab06bbb-6881-4038-b482-327ff71f7bda-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d26m8\" (UID: \"8ab06bbb-6881-4038-b482-327ff71f7bda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d26m8"
Sep 05 07:48:19.836243 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.835674    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx6rj\" (UniqueName: \"kubernetes.io/projected/f08f40f6-ff0d-419e-9268-fe051987663e-kube-api-access-cx6rj\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.836243 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.835676    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f08f40f6-ff0d-419e-9268-fe051987663e-host-run-k8s-cni-cncf-io\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.836243 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.835705    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5ff68b54-382f-46b6-8ee5-3d55b4aa0444-iptables-alerter-script\") pod \"iptables-alerter-mvqcl\" (UID: \"5ff68b54-382f-46b6-8ee5-3d55b4aa0444\") " pod="openshift-network-operator/iptables-alerter-mvqcl"
Sep 05 07:48:19.836243 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.835726    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8ab06bbb-6881-4038-b482-327ff71f7bda-registration-dir\") pod \"aws-ebs-csi-driver-node-d26m8\" (UID: \"8ab06bbb-6881-4038-b482-327ff71f7bda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d26m8"
Sep 05 07:48:19.836243 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.835707    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3e67240a-02be-4224-b626-99289ca9f76d-etc-systemd\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.836243 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.835746    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3e67240a-02be-4224-b626-99289ca9f76d-host\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.836243 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.835799    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3e67240a-02be-4224-b626-99289ca9f76d-host\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.836243 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.835801    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f08f40f6-ff0d-419e-9268-fe051987663e-multus-socket-dir-parent\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.836243 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.835807    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8ab06bbb-6881-4038-b482-327ff71f7bda-registration-dir\") pod \"aws-ebs-csi-driver-node-d26m8\" (UID: \"8ab06bbb-6881-4038-b482-327ff71f7bda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d26m8"
Sep 05 07:48:19.836243 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.835829    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f08f40f6-ff0d-419e-9268-fe051987663e-multus-conf-dir\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.836243 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.835866    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f08f40f6-ff0d-419e-9268-fe051987663e-multus-socket-dir-parent\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.836243 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.835871    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e67240a-02be-4224-b626-99289ca9f76d-tmp\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.836243 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.835877    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f08f40f6-ff0d-419e-9268-fe051987663e-multus-conf-dir\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.836243 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.835898    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/155f061b-e464-41b8-bf23-4183a33dd370-konnectivity-ca\") pod \"konnectivity-agent-8phbx\" (UID: \"155f061b-e464-41b8-bf23-4183a33dd370\") " pod="kube-system/konnectivity-agent-8phbx"
Sep 05 07:48:19.836243 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.835926    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f08f40f6-ff0d-419e-9268-fe051987663e-system-cni-dir\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.836809 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.835958    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp8jv\" (UniqueName: \"kubernetes.io/projected/5ff68b54-382f-46b6-8ee5-3d55b4aa0444-kube-api-access-kp8jv\") pod \"iptables-alerter-mvqcl\" (UID: \"5ff68b54-382f-46b6-8ee5-3d55b4aa0444\") " pod="openshift-network-operator/iptables-alerter-mvqcl"
Sep 05 07:48:19.836809 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.835975    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3e67240a-02be-4224-b626-99289ca9f76d-etc-modprobe-d\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.836809 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.836072    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f08f40f6-ff0d-419e-9268-fe051987663e-system-cni-dir\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.836809 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.836110    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3e67240a-02be-4224-b626-99289ca9f76d-etc-sysctl-d\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.836809 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.836146    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e67240a-02be-4224-b626-99289ca9f76d-var-lib-kubelet\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.836809 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.836189    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f08f40f6-ff0d-419e-9268-fe051987663e-multus-cni-dir\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.836809 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.836218    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnswt\" (UniqueName: \"kubernetes.io/projected/8ab06bbb-6881-4038-b482-327ff71f7bda-kube-api-access-gnswt\") pod \"aws-ebs-csi-driver-node-d26m8\" (UID: \"8ab06bbb-6881-4038-b482-327ff71f7bda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d26m8"
Sep 05 07:48:19.836809 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.836245    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e67240a-02be-4224-b626-99289ca9f76d-var-lib-kubelet\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.836809 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.836250    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3e67240a-02be-4224-b626-99289ca9f76d-sys\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.836809 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.836295    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3e67240a-02be-4224-b626-99289ca9f76d-sys\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.836809 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.836308    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f08f40f6-ff0d-419e-9268-fe051987663e-cni-binary-copy\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.836809 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.836345    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8ab06bbb-6881-4038-b482-327ff71f7bda-sys-fs\") pod \"aws-ebs-csi-driver-node-d26m8\" (UID: \"8ab06bbb-6881-4038-b482-327ff71f7bda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d26m8"
Sep 05 07:48:19.836809 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.836377    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3e67240a-02be-4224-b626-99289ca9f76d-etc-tuned\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.836809 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.836406    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f08f40f6-ff0d-419e-9268-fe051987663e-cnibin\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.836809 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.836439    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f08f40f6-ff0d-419e-9268-fe051987663e-multus-cni-dir\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.836809 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.836463    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/155f061b-e464-41b8-bf23-4183a33dd370-konnectivity-ca\") pod \"konnectivity-agent-8phbx\" (UID: \"155f061b-e464-41b8-bf23-4183a33dd370\") " pod="kube-system/konnectivity-agent-8phbx"
Sep 05 07:48:19.836809 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.836523    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8ab06bbb-6881-4038-b482-327ff71f7bda-sys-fs\") pod \"aws-ebs-csi-driver-node-d26m8\" (UID: \"8ab06bbb-6881-4038-b482-327ff71f7bda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d26m8"
Sep 05 07:48:19.836809 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.836080    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3e67240a-02be-4224-b626-99289ca9f76d-etc-modprobe-d\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.837631 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.836729    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f08f40f6-ff0d-419e-9268-fe051987663e-cnibin\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.837631 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.836838    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3e67240a-02be-4224-b626-99289ca9f76d-etc-sysctl-d\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.837631 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.836883    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f08f40f6-ff0d-419e-9268-fe051987663e-host-var-lib-cni-bin\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.837631 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.836997    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f08f40f6-ff0d-419e-9268-fe051987663e-host-var-lib-cni-bin\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.837631 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.837038    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f08f40f6-ff0d-419e-9268-fe051987663e-host-run-netns\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.837631 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.837064    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f08f40f6-ff0d-419e-9268-fe051987663e-host-var-lib-kubelet\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.837631 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.837088    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f08f40f6-ff0d-419e-9268-fe051987663e-multus-daemon-config\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.837631 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.837113    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f08f40f6-ff0d-419e-9268-fe051987663e-cni-binary-copy\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.837631 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.837114    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f08f40f6-ff0d-419e-9268-fe051987663e-host-run-multus-certs\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.837631 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.837151    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f08f40f6-ff0d-419e-9268-fe051987663e-host-var-lib-kubelet\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.837631 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.837156    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f08f40f6-ff0d-419e-9268-fe051987663e-host-run-multus-certs\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.837631 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.837168    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f08f40f6-ff0d-419e-9268-fe051987663e-host-run-netns\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.837631 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.837159    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sgcg\" (UniqueName: \"kubernetes.io/projected/3e67240a-02be-4224-b626-99289ca9f76d-kube-api-access-8sgcg\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.837631 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.837222    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ljvk\" (UniqueName: \"kubernetes.io/projected/35c73ae3-8ad8-443c-ac44-c9cd3396e526-kube-api-access-8ljvk\") pod \"node-ca-m7775\" (UID: \"35c73ae3-8ad8-443c-ac44-c9cd3396e526\") " pod="openshift-image-registry/node-ca-m7775"
Sep 05 07:48:19.837631 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.837247    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/155f061b-e464-41b8-bf23-4183a33dd370-agent-certs\") pod \"konnectivity-agent-8phbx\" (UID: \"155f061b-e464-41b8-bf23-4183a33dd370\") " pod="kube-system/konnectivity-agent-8phbx"
Sep 05 07:48:19.837631 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.837275    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e67240a-02be-4224-b626-99289ca9f76d-etc-kubernetes\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.837631 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.837305    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/35c73ae3-8ad8-443c-ac44-c9cd3396e526-serviceca\") pod \"node-ca-m7775\" (UID: \"35c73ae3-8ad8-443c-ac44-c9cd3396e526\") " pod="openshift-image-registry/node-ca-m7775"
Sep 05 07:48:19.837631 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.837347    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3e67240a-02be-4224-b626-99289ca9f76d-etc-sysconfig\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.838485 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.837371    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8pcf\" (UniqueName: \"kubernetes.io/projected/f0fd7e0b-f07e-43a2-a169-667bc753a36b-kube-api-access-s8pcf\") pod \"node-resolver-fstjt\" (UID: \"f0fd7e0b-f07e-43a2-a169-667bc753a36b\") " pod="openshift-dns/node-resolver-fstjt"
Sep 05 07:48:19.838485 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.837396    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3e67240a-02be-4224-b626-99289ca9f76d-etc-sysctl-conf\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.838485 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.837535    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3e67240a-02be-4224-b626-99289ca9f76d-etc-sysctl-conf\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.838485 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.837047    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5ff68b54-382f-46b6-8ee5-3d55b4aa0444-iptables-alerter-script\") pod \"iptables-alerter-mvqcl\" (UID: \"5ff68b54-382f-46b6-8ee5-3d55b4aa0444\") " pod="openshift-network-operator/iptables-alerter-mvqcl"
Sep 05 07:48:19.838485 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.837619    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f08f40f6-ff0d-419e-9268-fe051987663e-multus-daemon-config\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.838485 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.837554    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3e67240a-02be-4224-b626-99289ca9f76d-etc-sysconfig\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.838485 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.837645    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e67240a-02be-4224-b626-99289ca9f76d-etc-kubernetes\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.838485 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.837684    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f0fd7e0b-f07e-43a2-a169-667bc753a36b-hosts-file\") pod \"node-resolver-fstjt\" (UID: \"f0fd7e0b-f07e-43a2-a169-667bc753a36b\") " pod="openshift-dns/node-resolver-fstjt"
Sep 05 07:48:19.838485 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.837717    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8ab06bbb-6881-4038-b482-327ff71f7bda-socket-dir\") pod \"aws-ebs-csi-driver-node-d26m8\" (UID: \"8ab06bbb-6881-4038-b482-327ff71f7bda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d26m8"
Sep 05 07:48:19.838485 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.837745    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0fd7e0b-f07e-43a2-a169-667bc753a36b-tmp-dir\") pod \"node-resolver-fstjt\" (UID: \"f0fd7e0b-f07e-43a2-a169-667bc753a36b\") " pod="openshift-dns/node-resolver-fstjt"
Sep 05 07:48:19.838485 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.837769    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f08f40f6-ff0d-419e-9268-fe051987663e-hostroot\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.838485 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.837784    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f0fd7e0b-f07e-43a2-a169-667bc753a36b-hosts-file\") pod \"node-resolver-fstjt\" (UID: \"f0fd7e0b-f07e-43a2-a169-667bc753a36b\") " pod="openshift-dns/node-resolver-fstjt"
Sep 05 07:48:19.838485 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.837817    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f08f40f6-ff0d-419e-9268-fe051987663e-hostroot\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.838485 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.837847    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/35c73ae3-8ad8-443c-ac44-c9cd3396e526-serviceca\") pod \"node-ca-m7775\" (UID: \"35c73ae3-8ad8-443c-ac44-c9cd3396e526\") " pod="openshift-image-registry/node-ca-m7775"
Sep 05 07:48:19.838485 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.837843    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8ab06bbb-6881-4038-b482-327ff71f7bda-socket-dir\") pod \"aws-ebs-csi-driver-node-d26m8\" (UID: \"8ab06bbb-6881-4038-b482-327ff71f7bda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d26m8"
Sep 05 07:48:19.838485 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.837877    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8ab06bbb-6881-4038-b482-327ff71f7bda-device-dir\") pod \"aws-ebs-csi-driver-node-d26m8\" (UID: \"8ab06bbb-6881-4038-b482-327ff71f7bda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d26m8"
Sep 05 07:48:19.838485 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.837957    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8ab06bbb-6881-4038-b482-327ff71f7bda-device-dir\") pod \"aws-ebs-csi-driver-node-d26m8\" (UID: \"8ab06bbb-6881-4038-b482-327ff71f7bda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d26m8"
Sep 05 07:48:19.838485 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.838102    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0fd7e0b-f07e-43a2-a169-667bc753a36b-tmp-dir\") pod \"node-resolver-fstjt\" (UID: \"f0fd7e0b-f07e-43a2-a169-667bc753a36b\") " pod="openshift-dns/node-resolver-fstjt"
Sep 05 07:48:19.840482 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.840457    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e67240a-02be-4224-b626-99289ca9f76d-tmp\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.841175 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.840628    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/155f061b-e464-41b8-bf23-4183a33dd370-agent-certs\") pod \"konnectivity-agent-8phbx\" (UID: \"155f061b-e464-41b8-bf23-4183a33dd370\") " pod="kube-system/konnectivity-agent-8phbx"
Sep 05 07:48:19.850881 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.847205    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3e67240a-02be-4224-b626-99289ca9f76d-etc-tuned\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.850881 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.849614    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sgcg\" (UniqueName: \"kubernetes.io/projected/3e67240a-02be-4224-b626-99289ca9f76d-kube-api-access-8sgcg\") pod \"tuned-kqd45\" (UID: \"3e67240a-02be-4224-b626-99289ca9f76d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:19.850881 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.850174    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ljvk\" (UniqueName: \"kubernetes.io/projected/35c73ae3-8ad8-443c-ac44-c9cd3396e526-kube-api-access-8ljvk\") pod \"node-ca-m7775\" (UID: \"35c73ae3-8ad8-443c-ac44-c9cd3396e526\") " pod="openshift-image-registry/node-ca-m7775"
Sep 05 07:48:19.852206 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.852181    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kp8jv\" (UniqueName: \"kubernetes.io/projected/5ff68b54-382f-46b6-8ee5-3d55b4aa0444-kube-api-access-kp8jv\") pod \"iptables-alerter-mvqcl\" (UID: \"5ff68b54-382f-46b6-8ee5-3d55b4aa0444\") " pod="openshift-network-operator/iptables-alerter-mvqcl"
Sep 05 07:48:19.853247 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.853178    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnswt\" (UniqueName: \"kubernetes.io/projected/8ab06bbb-6881-4038-b482-327ff71f7bda-kube-api-access-gnswt\") pod \"aws-ebs-csi-driver-node-d26m8\" (UID: \"8ab06bbb-6881-4038-b482-327ff71f7bda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d26m8"
Sep 05 07:48:19.853975 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.853943    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8pcf\" (UniqueName: \"kubernetes.io/projected/f0fd7e0b-f07e-43a2-a169-667bc753a36b-kube-api-access-s8pcf\") pod \"node-resolver-fstjt\" (UID: \"f0fd7e0b-f07e-43a2-a169-667bc753a36b\") " pod="openshift-dns/node-resolver-fstjt"
Sep 05 07:48:19.854320 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.854296    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx6rj\" (UniqueName: \"kubernetes.io/projected/f08f40f6-ff0d-419e-9268-fe051987663e-kube-api-access-cx6rj\") pod \"multus-cvxlh\" (UID: \"f08f40f6-ff0d-419e-9268-fe051987663e\") " pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:19.934766 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.934726    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:19.947232 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.947206    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cd8wl"
Sep 05 07:48:19.965023 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.964997    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mvqcl"
Sep 05 07:48:19.976030 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.976006    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d26m8"
Sep 05 07:48:19.984040 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.984016    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8phbx"
Sep 05 07:48:19.993001 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:19.992976    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fstjt"
Sep 05 07:48:20.001920 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:20.001901    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kqd45"
Sep 05 07:48:20.010756 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:20.010739    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-m7775"
Sep 05 07:48:20.017643 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:20.017624    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cvxlh"
Sep 05 07:48:20.241685 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:20.241610    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/404c81cf-16aa-49f0-9059-fb97a5791ed7-metrics-certs\") pod \"network-metrics-daemon-l9lgq\" (UID: \"404c81cf-16aa-49f0-9059-fb97a5791ed7\") " pod="openshift-multus/network-metrics-daemon-l9lgq"
Sep 05 07:48:20.241826 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:20.241696    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e352b34-8143-4a94-86e7-11a9460d2946-original-pull-secret\") pod \"global-pull-secret-syncer-lx5db\" (UID: \"7e352b34-8143-4a94-86e7-11a9460d2946\") " pod="kube-system/global-pull-secret-syncer-lx5db"
Sep 05 07:48:20.241826 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:20.241802    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 05 07:48:20.241826 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:20.241806    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 07:48:20.241976 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:20.241854    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e352b34-8143-4a94-86e7-11a9460d2946-original-pull-secret podName:7e352b34-8143-4a94-86e7-11a9460d2946 nodeName:}" failed. No retries permitted until 2026-09-05 07:48:21.241840471 +0000 UTC m=+4.011982803 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7e352b34-8143-4a94-86e7-11a9460d2946-original-pull-secret") pod "global-pull-secret-syncer-lx5db" (UID: "7e352b34-8143-4a94-86e7-11a9460d2946") : object "kube-system"/"original-pull-secret" not registered
Sep 05 07:48:20.241976 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:20.241899    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/404c81cf-16aa-49f0-9059-fb97a5791ed7-metrics-certs podName:404c81cf-16aa-49f0-9059-fb97a5791ed7 nodeName:}" failed. No retries permitted until 2026-09-05 07:48:21.241877553 +0000 UTC m=+4.012019889 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/404c81cf-16aa-49f0-9059-fb97a5791ed7-metrics-certs") pod "network-metrics-daemon-l9lgq" (UID: "404c81cf-16aa-49f0-9059-fb97a5791ed7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 07:48:20.342126 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:20.342084    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wt7mt\" (UniqueName: \"kubernetes.io/projected/b1e5c942-8397-439a-96b6-13149a974b18-kube-api-access-wt7mt\") pod \"network-check-target-rbtht\" (UID: \"b1e5c942-8397-439a-96b6-13149a974b18\") " pod="openshift-network-diagnostics/network-check-target-rbtht"
Sep 05 07:48:20.342317 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:20.342293    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 05 07:48:20.342397 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:20.342327    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 05 07:48:20.342397 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:20.342342    2563 projected.go:196] Error preparing data for projected volume kube-api-access-wt7mt for pod openshift-network-diagnostics/network-check-target-rbtht: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 07:48:20.342469 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:20.342411    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b1e5c942-8397-439a-96b6-13149a974b18-kube-api-access-wt7mt podName:b1e5c942-8397-439a-96b6-13149a974b18 nodeName:}" failed. No retries permitted until 2026-09-05 07:48:21.342389895 +0000 UTC m=+4.112532230 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wt7mt" (UniqueName: "kubernetes.io/projected/b1e5c942-8397-439a-96b6-13149a974b18-kube-api-access-wt7mt") pod "network-check-target-rbtht" (UID: "b1e5c942-8397-439a-96b6-13149a974b18") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 07:48:20.676819 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:20.676720    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-04 07:43:19 +0000 UTC" deadline="2028-02-26 01:06:50.784088626 +0000 UTC"
Sep 05 07:48:20.676819 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:20.676770    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12929h18m30.107322592s"
Sep 05 07:48:20.724562 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:20.724533    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lx5db"
Sep 05 07:48:20.724562 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:20.724550    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rbtht"
Sep 05 07:48:20.724769 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:20.724653    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lx5db" podUID="7e352b34-8143-4a94-86e7-11a9460d2946"
Sep 05 07:48:20.724827 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:20.724790    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rbtht" podUID="b1e5c942-8397-439a-96b6-13149a974b18"
Sep 05 07:48:20.969626 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:20.969588    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0fd7e0b_f07e_43a2_a169_667bc753a36b.slice/crio-e08b0e9d602631bda2f5a2d96d533fb552ede76cbadb0389ef7286401b10242f WatchSource:0}: Error finding container e08b0e9d602631bda2f5a2d96d533fb552ede76cbadb0389ef7286401b10242f: Status 404 returned error can't find the container with id e08b0e9d602631bda2f5a2d96d533fb552ede76cbadb0389ef7286401b10242f
Sep 05 07:48:20.970150 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:20.970113    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ff68b54_382f_46b6_8ee5_3d55b4aa0444.slice/crio-dbd5424c8b1cb19ad468d32ae473bd779928b72b7e5c62168af3535e36f98708 WatchSource:0}: Error finding container dbd5424c8b1cb19ad468d32ae473bd779928b72b7e5c62168af3535e36f98708: Status 404 returned error can't find the container with id dbd5424c8b1cb19ad468d32ae473bd779928b72b7e5c62168af3535e36f98708
Sep 05 07:48:20.993229 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:20.993204    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e67240a_02be_4224_b626_99289ca9f76d.slice/crio-69137f13e29643f4fa855f6f9ef49c47e438f90610acb09e52a52476684d7063 WatchSource:0}: Error finding container 69137f13e29643f4fa855f6f9ef49c47e438f90610acb09e52a52476684d7063: Status 404 returned error can't find the container with id 69137f13e29643f4fa855f6f9ef49c47e438f90610acb09e52a52476684d7063
Sep 05 07:48:20.996025 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:20.995998    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod569cffae_1c5c_4b02_af57_9ca53c455e4c.slice/crio-26fe25142cca57487f1fdcc015edb0786d4668b6902ebb49dd9b11bfb647e156 WatchSource:0}: Error finding container 26fe25142cca57487f1fdcc015edb0786d4668b6902ebb49dd9b11bfb647e156: Status 404 returned error can't find the container with id 26fe25142cca57487f1fdcc015edb0786d4668b6902ebb49dd9b11bfb647e156
Sep 05 07:48:21.249310 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:21.249229    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/404c81cf-16aa-49f0-9059-fb97a5791ed7-metrics-certs\") pod \"network-metrics-daemon-l9lgq\" (UID: \"404c81cf-16aa-49f0-9059-fb97a5791ed7\") " pod="openshift-multus/network-metrics-daemon-l9lgq"
Sep 05 07:48:21.249310 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:21.249291    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e352b34-8143-4a94-86e7-11a9460d2946-original-pull-secret\") pod \"global-pull-secret-syncer-lx5db\" (UID: \"7e352b34-8143-4a94-86e7-11a9460d2946\") " pod="kube-system/global-pull-secret-syncer-lx5db"
Sep 05 07:48:21.249513 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:21.249411    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 05 07:48:21.249513 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:21.249428    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 07:48:21.249513 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:21.249474    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e352b34-8143-4a94-86e7-11a9460d2946-original-pull-secret podName:7e352b34-8143-4a94-86e7-11a9460d2946 nodeName:}" failed. No retries permitted until 2026-09-05 07:48:23.249454723 +0000 UTC m=+6.019597042 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7e352b34-8143-4a94-86e7-11a9460d2946-original-pull-secret") pod "global-pull-secret-syncer-lx5db" (UID: "7e352b34-8143-4a94-86e7-11a9460d2946") : object "kube-system"/"original-pull-secret" not registered
Sep 05 07:48:21.249513 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:21.249505    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/404c81cf-16aa-49f0-9059-fb97a5791ed7-metrics-certs podName:404c81cf-16aa-49f0-9059-fb97a5791ed7 nodeName:}" failed. No retries permitted until 2026-09-05 07:48:23.249481808 +0000 UTC m=+6.019624146 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/404c81cf-16aa-49f0-9059-fb97a5791ed7-metrics-certs") pod "network-metrics-daemon-l9lgq" (UID: "404c81cf-16aa-49f0-9059-fb97a5791ed7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 07:48:21.350441 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:21.350415    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wt7mt\" (UniqueName: \"kubernetes.io/projected/b1e5c942-8397-439a-96b6-13149a974b18-kube-api-access-wt7mt\") pod \"network-check-target-rbtht\" (UID: \"b1e5c942-8397-439a-96b6-13149a974b18\") " pod="openshift-network-diagnostics/network-check-target-rbtht"
Sep 05 07:48:21.350598 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:21.350576    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 05 07:48:21.350656 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:21.350604    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 05 07:48:21.350656 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:21.350624    2563 projected.go:196] Error preparing data for projected volume kube-api-access-wt7mt for pod openshift-network-diagnostics/network-check-target-rbtht: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 07:48:21.350723 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:21.350677    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b1e5c942-8397-439a-96b6-13149a974b18-kube-api-access-wt7mt podName:b1e5c942-8397-439a-96b6-13149a974b18 nodeName:}" failed. No retries permitted until 2026-09-05 07:48:23.350662367 +0000 UTC m=+6.120804683 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wt7mt" (UniqueName: "kubernetes.io/projected/b1e5c942-8397-439a-96b6-13149a974b18-kube-api-access-wt7mt") pod "network-check-target-rbtht" (UID: "b1e5c942-8397-439a-96b6-13149a974b18") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 07:48:21.677550 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:21.677455    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-04 07:43:19 +0000 UTC" deadline="2028-03-03 05:29:26.432381121 +0000 UTC"
Sep 05 07:48:21.677550 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:21.677497    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13077h41m4.75488929s"
Sep 05 07:48:21.726269 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:21.726224    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rbtht"
Sep 05 07:48:21.726424 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:21.726366    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rbtht" podUID="b1e5c942-8397-439a-96b6-13149a974b18"
Sep 05 07:48:21.729538 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:21.729307    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9lgq"
Sep 05 07:48:21.729672 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:21.729650    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9lgq" podUID="404c81cf-16aa-49f0-9059-fb97a5791ed7"
Sep 05 07:48:21.746110 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:21.746078    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cvxlh" event={"ID":"f08f40f6-ff0d-419e-9268-fe051987663e","Type":"ContainerStarted","Data":"6c07df68fc1c16dc6d410d905a31cac6a8b5b12501a99b3eb68ceca28c1efce1"}
Sep 05 07:48:21.749837 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:21.749807    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8" event={"ID":"569cffae-1c5c-4b02-af57-9ca53c455e4c","Type":"ContainerStarted","Data":"26fe25142cca57487f1fdcc015edb0786d4668b6902ebb49dd9b11bfb647e156"}
Sep 05 07:48:21.751822 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:21.751796    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mvqcl" event={"ID":"5ff68b54-382f-46b6-8ee5-3d55b4aa0444","Type":"ContainerStarted","Data":"dbd5424c8b1cb19ad468d32ae473bd779928b72b7e5c62168af3535e36f98708"}
Sep 05 07:48:21.758026 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:21.757992    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-14.ec2.internal" event={"ID":"5bcee7ae8d0b6018ec80e28e7bc76ca5","Type":"ContainerStarted","Data":"36d24ee44a6ff3e4113992697b0bb25e2cb6c8314418396f79e48aca55313d04"}
Sep 05 07:48:21.761925 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:21.761890    2563 generic.go:356] "Generic (PLEG): container finished" podID="0ff80311f74673ce3c7117e23d0a13e1" containerID="3fc2f50c0f09aa2ed76a501a2aabf6a5c2e01191a6e21885eaf59da4130de440" exitCode=0
Sep 05 07:48:21.762025 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:21.762012    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-14.ec2.internal" event={"ID":"0ff80311f74673ce3c7117e23d0a13e1","Type":"ContainerDied","Data":"3fc2f50c0f09aa2ed76a501a2aabf6a5c2e01191a6e21885eaf59da4130de440"}
Sep 05 07:48:21.768084 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:21.768057    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kqd45" event={"ID":"3e67240a-02be-4224-b626-99289ca9f76d","Type":"ContainerStarted","Data":"69137f13e29643f4fa855f6f9ef49c47e438f90610acb09e52a52476684d7063"}
Sep 05 07:48:21.777662 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:21.777610    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-14.ec2.internal" podStartSLOduration=2.777592102 podStartE2EDuration="2.777592102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 07:48:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:48:21.776748297 +0000 UTC m=+4.546890639" watchObservedRunningTime="2026-09-05 07:48:21.777592102 +0000 UTC m=+4.547734457"
Sep 05 07:48:21.788346 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:21.788307    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fstjt" event={"ID":"f0fd7e0b-f07e-43a2-a169-667bc753a36b","Type":"ContainerStarted","Data":"e08b0e9d602631bda2f5a2d96d533fb552ede76cbadb0389ef7286401b10242f"}
Sep 05 07:48:21.802659 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:21.799815    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8phbx" event={"ID":"155f061b-e464-41b8-bf23-4183a33dd370","Type":"ContainerStarted","Data":"33b5bedc9c7e405ddb34b5e3f63aca368511611818ce5244ddb543955df05aa8"}
Sep 05 07:48:21.814489 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:21.814459    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d26m8" event={"ID":"8ab06bbb-6881-4038-b482-327ff71f7bda","Type":"ContainerStarted","Data":"466ac145e8594839f952b1f08bec8952bf94685057672998a39c7b426692ad43"}
Sep 05 07:48:21.826090 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:21.826021    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cd8wl" event={"ID":"4d02d57b-ec85-4cd4-8da2-0ffee0574984","Type":"ContainerStarted","Data":"a034d3c1a869e2355d99729ff3dd62ea3332bce1d14f654c37791a2680b9a256"}
Sep 05 07:48:21.833098 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:21.833032    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-m7775" event={"ID":"35c73ae3-8ad8-443c-ac44-c9cd3396e526","Type":"ContainerStarted","Data":"37a60dfa321fd96533171fe42ec3c7c3cac1fbae73a2cfb5a14e22aa3499a031"}
Sep 05 07:48:22.725098 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:22.724572    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lx5db"
Sep 05 07:48:22.725098 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:22.724710    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lx5db" podUID="7e352b34-8143-4a94-86e7-11a9460d2946"
Sep 05 07:48:22.855706 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:22.855652    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-14.ec2.internal" event={"ID":"0ff80311f74673ce3c7117e23d0a13e1","Type":"ContainerStarted","Data":"8a23c897a1a9affa4ed0e3b5c7c2365e4c13efe9a686f30b5cf3b906e52510af"}
Sep 05 07:48:22.872096 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:22.872017    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-14.ec2.internal" podStartSLOduration=3.871993683 podStartE2EDuration="3.871993683s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 07:48:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:48:22.871447386 +0000 UTC m=+5.641589728" watchObservedRunningTime="2026-09-05 07:48:22.871993683 +0000 UTC m=+5.642136029"
Sep 05 07:48:23.269109 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:23.269068    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e352b34-8143-4a94-86e7-11a9460d2946-original-pull-secret\") pod \"global-pull-secret-syncer-lx5db\" (UID: \"7e352b34-8143-4a94-86e7-11a9460d2946\") " pod="kube-system/global-pull-secret-syncer-lx5db"
Sep 05 07:48:23.269294 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:23.269165    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/404c81cf-16aa-49f0-9059-fb97a5791ed7-metrics-certs\") pod \"network-metrics-daemon-l9lgq\" (UID: \"404c81cf-16aa-49f0-9059-fb97a5791ed7\") " pod="openshift-multus/network-metrics-daemon-l9lgq"
Sep 05 07:48:23.269381 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:23.269325    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 07:48:23.269441 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:23.269389    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/404c81cf-16aa-49f0-9059-fb97a5791ed7-metrics-certs podName:404c81cf-16aa-49f0-9059-fb97a5791ed7 nodeName:}" failed. No retries permitted until 2026-09-05 07:48:27.269371231 +0000 UTC m=+10.039513552 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/404c81cf-16aa-49f0-9059-fb97a5791ed7-metrics-certs") pod "network-metrics-daemon-l9lgq" (UID: "404c81cf-16aa-49f0-9059-fb97a5791ed7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 07:48:23.269845 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:23.269824    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 05 07:48:23.269968 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:23.269880    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e352b34-8143-4a94-86e7-11a9460d2946-original-pull-secret podName:7e352b34-8143-4a94-86e7-11a9460d2946 nodeName:}" failed. No retries permitted until 2026-09-05 07:48:27.269865824 +0000 UTC m=+10.040008165 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7e352b34-8143-4a94-86e7-11a9460d2946-original-pull-secret") pod "global-pull-secret-syncer-lx5db" (UID: "7e352b34-8143-4a94-86e7-11a9460d2946") : object "kube-system"/"original-pull-secret" not registered
Sep 05 07:48:23.370279 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:23.370239    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wt7mt\" (UniqueName: \"kubernetes.io/projected/b1e5c942-8397-439a-96b6-13149a974b18-kube-api-access-wt7mt\") pod \"network-check-target-rbtht\" (UID: \"b1e5c942-8397-439a-96b6-13149a974b18\") " pod="openshift-network-diagnostics/network-check-target-rbtht"
Sep 05 07:48:23.370467 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:23.370438    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 05 07:48:23.370467 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:23.370458    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 05 07:48:23.370582 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:23.370473    2563 projected.go:196] Error preparing data for projected volume kube-api-access-wt7mt for pod openshift-network-diagnostics/network-check-target-rbtht: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 07:48:23.370582 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:23.370535    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b1e5c942-8397-439a-96b6-13149a974b18-kube-api-access-wt7mt podName:b1e5c942-8397-439a-96b6-13149a974b18 nodeName:}" failed. No retries permitted until 2026-09-05 07:48:27.370516211 +0000 UTC m=+10.140658528 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wt7mt" (UniqueName: "kubernetes.io/projected/b1e5c942-8397-439a-96b6-13149a974b18-kube-api-access-wt7mt") pod "network-check-target-rbtht" (UID: "b1e5c942-8397-439a-96b6-13149a974b18") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 07:48:23.726182 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:23.726061    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rbtht"
Sep 05 07:48:23.726656 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:23.726305    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9lgq"
Sep 05 07:48:23.726656 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:23.726293    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rbtht" podUID="b1e5c942-8397-439a-96b6-13149a974b18"
Sep 05 07:48:23.726656 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:23.726403    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9lgq" podUID="404c81cf-16aa-49f0-9059-fb97a5791ed7"
Sep 05 07:48:24.724257 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:24.723786    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lx5db"
Sep 05 07:48:24.724257 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:24.723925    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lx5db" podUID="7e352b34-8143-4a94-86e7-11a9460d2946"
Sep 05 07:48:25.726665 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:25.726629    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9lgq"
Sep 05 07:48:25.727236 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:25.726770    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9lgq" podUID="404c81cf-16aa-49f0-9059-fb97a5791ed7"
Sep 05 07:48:25.727236 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:25.727135    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rbtht"
Sep 05 07:48:25.727236 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:25.727225    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rbtht" podUID="b1e5c942-8397-439a-96b6-13149a974b18"
Sep 05 07:48:26.724098 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:26.724056    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lx5db"
Sep 05 07:48:26.724313 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:26.724205    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lx5db" podUID="7e352b34-8143-4a94-86e7-11a9460d2946"
Sep 05 07:48:27.304386 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:27.303762    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e352b34-8143-4a94-86e7-11a9460d2946-original-pull-secret\") pod \"global-pull-secret-syncer-lx5db\" (UID: \"7e352b34-8143-4a94-86e7-11a9460d2946\") " pod="kube-system/global-pull-secret-syncer-lx5db"
Sep 05 07:48:27.304386 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:27.303831    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/404c81cf-16aa-49f0-9059-fb97a5791ed7-metrics-certs\") pod \"network-metrics-daemon-l9lgq\" (UID: \"404c81cf-16aa-49f0-9059-fb97a5791ed7\") " pod="openshift-multus/network-metrics-daemon-l9lgq"
Sep 05 07:48:27.304386 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:27.303969    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 07:48:27.304386 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:27.303972    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 05 07:48:27.304386 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:27.304036    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/404c81cf-16aa-49f0-9059-fb97a5791ed7-metrics-certs podName:404c81cf-16aa-49f0-9059-fb97a5791ed7 nodeName:}" failed. No retries permitted until 2026-09-05 07:48:35.304017718 +0000 UTC m=+18.074160039 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/404c81cf-16aa-49f0-9059-fb97a5791ed7-metrics-certs") pod "network-metrics-daemon-l9lgq" (UID: "404c81cf-16aa-49f0-9059-fb97a5791ed7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 07:48:27.304386 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:27.304054    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e352b34-8143-4a94-86e7-11a9460d2946-original-pull-secret podName:7e352b34-8143-4a94-86e7-11a9460d2946 nodeName:}" failed. No retries permitted until 2026-09-05 07:48:35.304045287 +0000 UTC m=+18.074187608 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7e352b34-8143-4a94-86e7-11a9460d2946-original-pull-secret") pod "global-pull-secret-syncer-lx5db" (UID: "7e352b34-8143-4a94-86e7-11a9460d2946") : object "kube-system"/"original-pull-secret" not registered
Sep 05 07:48:27.404887 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:27.404836    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wt7mt\" (UniqueName: \"kubernetes.io/projected/b1e5c942-8397-439a-96b6-13149a974b18-kube-api-access-wt7mt\") pod \"network-check-target-rbtht\" (UID: \"b1e5c942-8397-439a-96b6-13149a974b18\") " pod="openshift-network-diagnostics/network-check-target-rbtht"
Sep 05 07:48:27.405062 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:27.404988    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 05 07:48:27.405062 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:27.405016    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 05 07:48:27.405062 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:27.405029    2563 projected.go:196] Error preparing data for projected volume kube-api-access-wt7mt for pod openshift-network-diagnostics/network-check-target-rbtht: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 07:48:27.405231 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:27.405094    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b1e5c942-8397-439a-96b6-13149a974b18-kube-api-access-wt7mt podName:b1e5c942-8397-439a-96b6-13149a974b18 nodeName:}" failed. No retries permitted until 2026-09-05 07:48:35.405076328 +0000 UTC m=+18.175218649 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-wt7mt" (UniqueName: "kubernetes.io/projected/b1e5c942-8397-439a-96b6-13149a974b18-kube-api-access-wt7mt") pod "network-check-target-rbtht" (UID: "b1e5c942-8397-439a-96b6-13149a974b18") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 07:48:27.725151 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:27.725081    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rbtht"
Sep 05 07:48:27.725278 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:27.725186    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rbtht" podUID="b1e5c942-8397-439a-96b6-13149a974b18"
Sep 05 07:48:27.725278 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:27.725256    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9lgq"
Sep 05 07:48:27.725366 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:27.725349    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9lgq" podUID="404c81cf-16aa-49f0-9059-fb97a5791ed7"
Sep 05 07:48:28.724350 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:28.724315    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lx5db"
Sep 05 07:48:28.724805 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:28.724462    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lx5db" podUID="7e352b34-8143-4a94-86e7-11a9460d2946"
Sep 05 07:48:29.724990 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:29.724961    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9lgq"
Sep 05 07:48:29.725418 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:29.725092    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9lgq" podUID="404c81cf-16aa-49f0-9059-fb97a5791ed7"
Sep 05 07:48:29.725418 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:29.725261    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rbtht"
Sep 05 07:48:29.725418 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:29.725366    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rbtht" podUID="b1e5c942-8397-439a-96b6-13149a974b18"
Sep 05 07:48:30.724311 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:30.723855    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lx5db"
Sep 05 07:48:30.724311 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:30.723990    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lx5db" podUID="7e352b34-8143-4a94-86e7-11a9460d2946"
Sep 05 07:48:31.723695 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:31.723665    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9lgq"
Sep 05 07:48:31.724126 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:31.723673    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rbtht"
Sep 05 07:48:31.724126 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:31.723796    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9lgq" podUID="404c81cf-16aa-49f0-9059-fb97a5791ed7"
Sep 05 07:48:31.724126 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:31.723904    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rbtht" podUID="b1e5c942-8397-439a-96b6-13149a974b18"
Sep 05 07:48:32.724484 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:32.724433    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lx5db"
Sep 05 07:48:32.724909 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:32.724580    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lx5db" podUID="7e352b34-8143-4a94-86e7-11a9460d2946"
Sep 05 07:48:33.723875 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:33.723825    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9lgq"
Sep 05 07:48:33.724045 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:33.723888    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rbtht"
Sep 05 07:48:33.724045 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:33.723987    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9lgq" podUID="404c81cf-16aa-49f0-9059-fb97a5791ed7"
Sep 05 07:48:33.724171 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:33.724071    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rbtht" podUID="b1e5c942-8397-439a-96b6-13149a974b18"
Sep 05 07:48:34.724025 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:34.723987    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lx5db"
Sep 05 07:48:34.724449 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:34.724108    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lx5db" podUID="7e352b34-8143-4a94-86e7-11a9460d2946"
Sep 05 07:48:35.367235 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:35.367208    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e352b34-8143-4a94-86e7-11a9460d2946-original-pull-secret\") pod \"global-pull-secret-syncer-lx5db\" (UID: \"7e352b34-8143-4a94-86e7-11a9460d2946\") " pod="kube-system/global-pull-secret-syncer-lx5db"
Sep 05 07:48:35.367325 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:35.367250    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/404c81cf-16aa-49f0-9059-fb97a5791ed7-metrics-certs\") pod \"network-metrics-daemon-l9lgq\" (UID: \"404c81cf-16aa-49f0-9059-fb97a5791ed7\") " pod="openshift-multus/network-metrics-daemon-l9lgq"
Sep 05 07:48:35.367369 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:35.367339    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 07:48:35.367369 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:35.367347    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 05 07:48:35.367424 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:35.367384    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/404c81cf-16aa-49f0-9059-fb97a5791ed7-metrics-certs podName:404c81cf-16aa-49f0-9059-fb97a5791ed7 nodeName:}" failed. No retries permitted until 2026-09-05 07:48:51.367372195 +0000 UTC m=+34.137514511 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/404c81cf-16aa-49f0-9059-fb97a5791ed7-metrics-certs") pod "network-metrics-daemon-l9lgq" (UID: "404c81cf-16aa-49f0-9059-fb97a5791ed7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 07:48:35.367424 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:35.367397    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e352b34-8143-4a94-86e7-11a9460d2946-original-pull-secret podName:7e352b34-8143-4a94-86e7-11a9460d2946 nodeName:}" failed. No retries permitted until 2026-09-05 07:48:51.36739082 +0000 UTC m=+34.137533135 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7e352b34-8143-4a94-86e7-11a9460d2946-original-pull-secret") pod "global-pull-secret-syncer-lx5db" (UID: "7e352b34-8143-4a94-86e7-11a9460d2946") : object "kube-system"/"original-pull-secret" not registered
Sep 05 07:48:35.467620 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:35.467593    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wt7mt\" (UniqueName: \"kubernetes.io/projected/b1e5c942-8397-439a-96b6-13149a974b18-kube-api-access-wt7mt\") pod \"network-check-target-rbtht\" (UID: \"b1e5c942-8397-439a-96b6-13149a974b18\") " pod="openshift-network-diagnostics/network-check-target-rbtht"
Sep 05 07:48:35.467771 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:35.467752    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 05 07:48:35.467814 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:35.467798    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 05 07:48:35.467814 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:35.467809    2563 projected.go:196] Error preparing data for projected volume kube-api-access-wt7mt for pod openshift-network-diagnostics/network-check-target-rbtht: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 07:48:35.467888 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:35.467871    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b1e5c942-8397-439a-96b6-13149a974b18-kube-api-access-wt7mt podName:b1e5c942-8397-439a-96b6-13149a974b18 nodeName:}" failed. No retries permitted until 2026-09-05 07:48:51.467857604 +0000 UTC m=+34.237999932 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-wt7mt" (UniqueName: "kubernetes.io/projected/b1e5c942-8397-439a-96b6-13149a974b18-kube-api-access-wt7mt") pod "network-check-target-rbtht" (UID: "b1e5c942-8397-439a-96b6-13149a974b18") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 07:48:35.725059 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:35.724976    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9lgq"
Sep 05 07:48:35.725059 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:35.725024    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rbtht"
Sep 05 07:48:35.725458 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:35.725135    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9lgq" podUID="404c81cf-16aa-49f0-9059-fb97a5791ed7"
Sep 05 07:48:35.725458 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:35.725247    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rbtht" podUID="b1e5c942-8397-439a-96b6-13149a974b18"
Sep 05 07:48:36.723644 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:36.723617    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lx5db"
Sep 05 07:48:36.723796 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:36.723719    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lx5db" podUID="7e352b34-8143-4a94-86e7-11a9460d2946"
Sep 05 07:48:37.725125 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:37.725095    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9lgq"
Sep 05 07:48:37.725556 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:37.725133    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rbtht"
Sep 05 07:48:37.725556 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:37.725209    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9lgq" podUID="404c81cf-16aa-49f0-9059-fb97a5791ed7"
Sep 05 07:48:37.725556 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:37.725312    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rbtht" podUID="b1e5c942-8397-439a-96b6-13149a974b18"
Sep 05 07:48:38.724590 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:38.724555    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lx5db"
Sep 05 07:48:38.724757 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:38.724689    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lx5db" podUID="7e352b34-8143-4a94-86e7-11a9460d2946"
Sep 05 07:48:39.724222 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:39.724053    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rbtht"
Sep 05 07:48:39.724605 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:39.724060    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9lgq"
Sep 05 07:48:39.724605 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:39.724293    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rbtht" podUID="b1e5c942-8397-439a-96b6-13149a974b18"
Sep 05 07:48:39.724605 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:39.724421    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9lgq" podUID="404c81cf-16aa-49f0-9059-fb97a5791ed7"
Sep 05 07:48:39.895967 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:39.895942    2563 generic.go:356] "Generic (PLEG): container finished" podID="4d02d57b-ec85-4cd4-8da2-0ffee0574984" containerID="d6ba31c785d100a72cd67667e899103f1c4f86b43e8aa6f371c35ff2b3ce4fd3" exitCode=0
Sep 05 07:48:39.896071 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:39.896018    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cd8wl" event={"ID":"4d02d57b-ec85-4cd4-8da2-0ffee0574984","Type":"ContainerDied","Data":"d6ba31c785d100a72cd67667e899103f1c4f86b43e8aa6f371c35ff2b3ce4fd3"}
Sep 05 07:48:39.897725 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:39.897703    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-m7775" event={"ID":"35c73ae3-8ad8-443c-ac44-c9cd3396e526","Type":"ContainerStarted","Data":"81ed11ea4d81d76b7ce490fe7a1b9e480b96f551a6acc03bd99fd86a569f25d7"}
Sep 05 07:48:39.899299 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:39.899270    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cvxlh" event={"ID":"f08f40f6-ff0d-419e-9268-fe051987663e","Type":"ContainerStarted","Data":"8edb680764de52e62579682619f50f0a78386f3b395882f6d339eefe682c93f8"}
Sep 05 07:48:39.901642 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:39.901626    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2bzp8_569cffae-1c5c-4b02-af57-9ca53c455e4c/ovn-acl-logging/0.log"
Sep 05 07:48:39.901953 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:39.901928    2563 generic.go:356] "Generic (PLEG): container finished" podID="569cffae-1c5c-4b02-af57-9ca53c455e4c" containerID="fff3988d8875fc30b4317152e8fb3ab40764e5215a88808ac8be74e3617bf9be" exitCode=1
Sep 05 07:48:39.902023 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:39.901952    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8" event={"ID":"569cffae-1c5c-4b02-af57-9ca53c455e4c","Type":"ContainerStarted","Data":"054dfae2270489f87cecc4e7ae890fe74ef3ed936b5a681feea41f9167de5b59"}
Sep 05 07:48:39.902023 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:39.902019    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8" event={"ID":"569cffae-1c5c-4b02-af57-9ca53c455e4c","Type":"ContainerStarted","Data":"2dda700c174125e71d2791a6f69614adb086b0749f82984687d99ab65dbe3ef9"}
Sep 05 07:48:39.902134 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:39.902052    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8" event={"ID":"569cffae-1c5c-4b02-af57-9ca53c455e4c","Type":"ContainerDied","Data":"fff3988d8875fc30b4317152e8fb3ab40764e5215a88808ac8be74e3617bf9be"}
Sep 05 07:48:39.902134 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:39.902082    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8" event={"ID":"569cffae-1c5c-4b02-af57-9ca53c455e4c","Type":"ContainerStarted","Data":"d26e8fa6b5f98d8ba51c7721e2d729b07864d73cc73e71286eebee6f14a8b703"}
Sep 05 07:48:39.903489 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:39.903466    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kqd45" event={"ID":"3e67240a-02be-4224-b626-99289ca9f76d","Type":"ContainerStarted","Data":"d66cd5dccf6e238a349cede0324dbff1e44b049be52d93ea404a7adb89fc01de"}
Sep 05 07:48:39.904889 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:39.904869    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fstjt" event={"ID":"f0fd7e0b-f07e-43a2-a169-667bc753a36b","Type":"ContainerStarted","Data":"25d3864ee1aa831f25683d53822a4c8160798dc40d54398923c8a564afaac3c7"}
Sep 05 07:48:39.906188 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:39.906165    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8phbx" event={"ID":"155f061b-e464-41b8-bf23-4183a33dd370","Type":"ContainerStarted","Data":"11e54115990782f76765941c96548bf7393a9af700f6faca3c7a6b14757d42d0"}
Sep 05 07:48:39.907425 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:39.907409    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d26m8" event={"ID":"8ab06bbb-6881-4038-b482-327ff71f7bda","Type":"ContainerStarted","Data":"cd3b6b55610c66a1c47fa63e5141a5492d821e52e111b592748308f387c131df"}
Sep 05 07:48:39.956623 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:39.956581    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-m7775" podStartSLOduration=3.698577715 podStartE2EDuration="21.956566489s" totalImagesPullingTime="18.257988774s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 07:48:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:48:39.956441106 +0000 UTC m=+22.726583454" watchObservedRunningTime="2026-09-05 07:48:39.956566489 +0000 UTC m=+22.726708827"
Sep 05 07:48:39.956945 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:39.956920    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-fstjt" podStartSLOduration=3.68743714 podStartE2EDuration="21.956909317s" totalImagesPullingTime="18.269472177s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 07:48:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:48:39.935186566 +0000 UTC m=+22.705328907" watchObservedRunningTime="2026-09-05 07:48:39.956909317 +0000 UTC m=+22.727051726"
Sep 05 07:48:39.992190 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:39.992146    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8phbx" podStartSLOduration=7.668716722 podStartE2EDuration="21.992127708s" totalImagesPullingTime="14.323410986s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:48:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:48:39.974568411 +0000 UTC m=+22.744710765" watchObservedRunningTime="2026-09-05 07:48:39.992127708 +0000 UTC m=+22.762270047"
Sep 05 07:48:39.992358 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:39.992325    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-kqd45" podStartSLOduration=3.7297982899999997 podStartE2EDuration="21.992318695s" totalImagesPullingTime="18.262520405s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 07:48:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:48:39.991769997 +0000 UTC m=+22.761912358" watchObservedRunningTime="2026-09-05 07:48:39.992318695 +0000 UTC m=+22.762461035"
Sep 05 07:48:40.012223 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:40.012060    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-cvxlh" podStartSLOduration=3.733000972 podStartE2EDuration="22.01204719s" totalImagesPullingTime="18.279046218s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 07:48:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:48:40.011734432 +0000 UTC m=+22.781876785" watchObservedRunningTime="2026-09-05 07:48:40.01204719 +0000 UTC m=+22.782189527"
Sep 05 07:48:40.723707 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:40.723681    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lx5db"
Sep 05 07:48:40.723822 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:40.723782    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lx5db" podUID="7e352b34-8143-4a94-86e7-11a9460d2946"
Sep 05 07:48:40.852527 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:40.852491    2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 05 07:48:40.912633 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:40.912590    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d26m8" event={"ID":"8ab06bbb-6881-4038-b482-327ff71f7bda","Type":"ContainerStarted","Data":"8ab835834fd0f7b2ac8bb3b39132545d8b2c2dac5c5f49bc6272e975175725df"}
Sep 05 07:48:40.915805 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:40.915782    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2bzp8_569cffae-1c5c-4b02-af57-9ca53c455e4c/ovn-acl-logging/0.log"
Sep 05 07:48:40.916237 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:40.916204    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8" event={"ID":"569cffae-1c5c-4b02-af57-9ca53c455e4c","Type":"ContainerStarted","Data":"4ac83f6ab661d4746c2d35a93d6fc9f7092515280248d6bac9b4d46565bbeca1"}
Sep 05 07:48:40.916332 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:40.916282    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8" event={"ID":"569cffae-1c5c-4b02-af57-9ca53c455e4c","Type":"ContainerStarted","Data":"6b4fa6d55502b91662776e03f662c732729bfc49931f4df6037f5e74a8174665"}
Sep 05 07:48:40.918012 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:40.917984    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mvqcl" event={"ID":"5ff68b54-382f-46b6-8ee5-3d55b4aa0444","Type":"ContainerStarted","Data":"f558d2761cb274b8c9cb84c3980f78fcfc50e5c8837421e2e39c477b5cbc8145"}
Sep 05 07:48:40.934206 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:40.934163    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-mvqcl" podStartSLOduration=4.664144946 podStartE2EDuration="22.934145574s" totalImagesPullingTime="18.270000628s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 07:48:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:48:40.933978535 +0000 UTC m=+23.704120897" watchObservedRunningTime="2026-09-05 07:48:40.934145574 +0000 UTC m=+23.704287911"
Sep 05 07:48:40.980246 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:40.980181    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8phbx"
Sep 05 07:48:40.980761 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:40.980744    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8phbx"
Sep 05 07:48:41.689690 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:41.689602    2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-05T07:48:40.852517885Z","UUID":"54d4f300-a74f-4563-83f6-d844294781ad","Handler":null,"Name":"","Endpoint":""}
Sep 05 07:48:41.691223 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:41.691201    2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 05 07:48:41.691335 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:41.691236    2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 05 07:48:41.724273 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:41.724247    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rbtht"
Sep 05 07:48:41.724449 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:41.724358    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rbtht" podUID="b1e5c942-8397-439a-96b6-13149a974b18"
Sep 05 07:48:41.724449 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:41.724385    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9lgq"
Sep 05 07:48:41.724563 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:41.724458    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9lgq" podUID="404c81cf-16aa-49f0-9059-fb97a5791ed7"
Sep 05 07:48:41.925700 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:41.925658    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d26m8" event={"ID":"8ab06bbb-6881-4038-b482-327ff71f7bda","Type":"ContainerStarted","Data":"ced02ec4bae6b8d63b613c5c9f1df912778a899b058b61b6c9d9ffa892da41b4"}
Sep 05 07:48:41.926178 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:41.926059    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8phbx"
Sep 05 07:48:41.926519 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:41.926492    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8phbx"
Sep 05 07:48:41.941790 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:41.941715    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d26m8" podStartSLOduration=3.586790049 podStartE2EDuration="23.941702596s" totalImagesPullingTime="20.354912547s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 07:48:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:48:41.941632438 +0000 UTC m=+24.711774775" watchObservedRunningTime="2026-09-05 07:48:41.941702596 +0000 UTC m=+24.711844962"
Sep 05 07:48:42.723596 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:42.723562    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lx5db"
Sep 05 07:48:42.723764 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:42.723687    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lx5db" podUID="7e352b34-8143-4a94-86e7-11a9460d2946"
Sep 05 07:48:42.931967 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:42.931940    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2bzp8_569cffae-1c5c-4b02-af57-9ca53c455e4c/ovn-acl-logging/0.log"
Sep 05 07:48:42.932397 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:42.932367    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8" event={"ID":"569cffae-1c5c-4b02-af57-9ca53c455e4c","Type":"ContainerStarted","Data":"2c95412e5cb284e032b001067a35ab833b326699af4ffb77afd370d4f6ffd271"}
Sep 05 07:48:43.724434 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:43.724401    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rbtht"
Sep 05 07:48:43.724609 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:43.724442    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9lgq"
Sep 05 07:48:43.724609 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:43.724527    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rbtht" podUID="b1e5c942-8397-439a-96b6-13149a974b18"
Sep 05 07:48:43.724717 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:43.724666    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9lgq" podUID="404c81cf-16aa-49f0-9059-fb97a5791ed7"
Sep 05 07:48:44.724613 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:44.724408    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lx5db"
Sep 05 07:48:44.725247 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:44.724650    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lx5db" podUID="7e352b34-8143-4a94-86e7-11a9460d2946"
Sep 05 07:48:44.939485 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:44.939459    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2bzp8_569cffae-1c5c-4b02-af57-9ca53c455e4c/ovn-acl-logging/0.log"
Sep 05 07:48:44.939893 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:44.939872    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8" event={"ID":"569cffae-1c5c-4b02-af57-9ca53c455e4c","Type":"ContainerStarted","Data":"7f1d2bec2a1c800b92fba3b633ffa77cd494dffdda10e50bcc4e1e45ac7a2bb4"}
Sep 05 07:48:44.940150 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:44.940130    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:44.940294 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:44.940271    2563 scope.go:117] "RemoveContainer" containerID="fff3988d8875fc30b4317152e8fb3ab40764e5215a88808ac8be74e3617bf9be"
Sep 05 07:48:44.942023 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:44.942002    2563 generic.go:356] "Generic (PLEG): container finished" podID="4d02d57b-ec85-4cd4-8da2-0ffee0574984" containerID="1256f1817daed779f13f83df94c4b2d2c8658676775aa04e4b5c0ec9d28a16ec" exitCode=0
Sep 05 07:48:44.942088 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:44.942038    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cd8wl" event={"ID":"4d02d57b-ec85-4cd4-8da2-0ffee0574984","Type":"ContainerDied","Data":"1256f1817daed779f13f83df94c4b2d2c8658676775aa04e4b5c0ec9d28a16ec"}
Sep 05 07:48:44.957697 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:44.957665    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:45.726466 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:45.726208    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9lgq"
Sep 05 07:48:45.726898 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:45.726412    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rbtht"
Sep 05 07:48:45.726898 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:45.726575    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9lgq" podUID="404c81cf-16aa-49f0-9059-fb97a5791ed7"
Sep 05 07:48:45.726898 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:45.726648    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rbtht" podUID="b1e5c942-8397-439a-96b6-13149a974b18"
Sep 05 07:48:45.947689 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:45.947607    2563 generic.go:356] "Generic (PLEG): container finished" podID="4d02d57b-ec85-4cd4-8da2-0ffee0574984" containerID="1c5224a846c07a030490bc98627b92a78fdd42d33393667d56d63c54750b03cb" exitCode=0
Sep 05 07:48:45.947900 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:45.947878    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cd8wl" event={"ID":"4d02d57b-ec85-4cd4-8da2-0ffee0574984","Type":"ContainerDied","Data":"1c5224a846c07a030490bc98627b92a78fdd42d33393667d56d63c54750b03cb"}
Sep 05 07:48:45.952070 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:45.952052    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2bzp8_569cffae-1c5c-4b02-af57-9ca53c455e4c/ovn-acl-logging/0.log"
Sep 05 07:48:45.953004 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:45.952981    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8" event={"ID":"569cffae-1c5c-4b02-af57-9ca53c455e4c","Type":"ContainerStarted","Data":"b16aee883fb0a52b8b20a1e0f86efcf4e94a1c7c7bec0224c9f2446cc28f894d"}
Sep 05 07:48:45.953214 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:45.953201    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:45.953308 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:45.953246    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:45.968922 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:45.968900    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:48:46.004859 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:46.004798    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8" podStartSLOduration=9.697022528 podStartE2EDuration="28.004782239s" totalImagesPullingTime="18.307759711s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 07:48:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:48:46.003751436 +0000 UTC m=+28.773893773" watchObservedRunningTime="2026-09-05 07:48:46.004782239 +0000 UTC m=+28.774924620"
Sep 05 07:48:46.623793 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:46.623760    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rbtht"]
Sep 05 07:48:46.623956 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:46.623937    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rbtht"
Sep 05 07:48:46.624077 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:46.624055    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rbtht" podUID="b1e5c942-8397-439a-96b6-13149a974b18"
Sep 05 07:48:46.626884 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:46.626859    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lx5db"]
Sep 05 07:48:46.626983 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:46.626964    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lx5db"
Sep 05 07:48:46.627070 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:46.627047    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lx5db" podUID="7e352b34-8143-4a94-86e7-11a9460d2946"
Sep 05 07:48:46.627392 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:46.627371    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-l9lgq"]
Sep 05 07:48:46.627474 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:46.627465    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9lgq"
Sep 05 07:48:46.627597 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:46.627567    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9lgq" podUID="404c81cf-16aa-49f0-9059-fb97a5791ed7"
Sep 05 07:48:46.958098 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:46.958005    2563 generic.go:356] "Generic (PLEG): container finished" podID="4d02d57b-ec85-4cd4-8da2-0ffee0574984" containerID="bdb0eafddbe470d81d901578d24f32080ccd1635b9f61105d885ea4bf6f957c2" exitCode=0
Sep 05 07:48:46.958451 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:46.958095    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cd8wl" event={"ID":"4d02d57b-ec85-4cd4-8da2-0ffee0574984","Type":"ContainerDied","Data":"bdb0eafddbe470d81d901578d24f32080ccd1635b9f61105d885ea4bf6f957c2"}
Sep 05 07:48:47.724790 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:47.724755    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9lgq"
Sep 05 07:48:47.724956 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:47.724896    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9lgq" podUID="404c81cf-16aa-49f0-9059-fb97a5791ed7"
Sep 05 07:48:48.724055 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:48.724022    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rbtht"
Sep 05 07:48:48.724436 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:48.724122    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rbtht" podUID="b1e5c942-8397-439a-96b6-13149a974b18"
Sep 05 07:48:48.724436 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:48.724171    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lx5db"
Sep 05 07:48:48.724436 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:48.724257    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lx5db" podUID="7e352b34-8143-4a94-86e7-11a9460d2946"
Sep 05 07:48:49.723930 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:49.723900    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9lgq"
Sep 05 07:48:49.724111 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:49.724043    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9lgq" podUID="404c81cf-16aa-49f0-9059-fb97a5791ed7"
Sep 05 07:48:50.724509 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:50.724479    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rbtht"
Sep 05 07:48:50.724965 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:50.724484    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lx5db"
Sep 05 07:48:50.724965 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:50.724605    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rbtht" podUID="b1e5c942-8397-439a-96b6-13149a974b18"
Sep 05 07:48:50.724965 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:50.724707    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lx5db" podUID="7e352b34-8143-4a94-86e7-11a9460d2946"
Sep 05 07:48:51.381521 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:51.381483    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/404c81cf-16aa-49f0-9059-fb97a5791ed7-metrics-certs\") pod \"network-metrics-daemon-l9lgq\" (UID: \"404c81cf-16aa-49f0-9059-fb97a5791ed7\") " pod="openshift-multus/network-metrics-daemon-l9lgq"
Sep 05 07:48:51.381704 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:51.381547    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e352b34-8143-4a94-86e7-11a9460d2946-original-pull-secret\") pod \"global-pull-secret-syncer-lx5db\" (UID: \"7e352b34-8143-4a94-86e7-11a9460d2946\") " pod="kube-system/global-pull-secret-syncer-lx5db"
Sep 05 07:48:51.381704 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:51.381613    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 07:48:51.381704 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:51.381685    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/404c81cf-16aa-49f0-9059-fb97a5791ed7-metrics-certs podName:404c81cf-16aa-49f0-9059-fb97a5791ed7 nodeName:}" failed. No retries permitted until 2026-09-05 07:49:23.381669394 +0000 UTC m=+66.151811709 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/404c81cf-16aa-49f0-9059-fb97a5791ed7-metrics-certs") pod "network-metrics-daemon-l9lgq" (UID: "404c81cf-16aa-49f0-9059-fb97a5791ed7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 07:48:51.381832 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:51.381620    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 05 07:48:51.381832 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:51.381753    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e352b34-8143-4a94-86e7-11a9460d2946-original-pull-secret podName:7e352b34-8143-4a94-86e7-11a9460d2946 nodeName:}" failed. No retries permitted until 2026-09-05 07:49:23.381739815 +0000 UTC m=+66.151882130 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7e352b34-8143-4a94-86e7-11a9460d2946-original-pull-secret") pod "global-pull-secret-syncer-lx5db" (UID: "7e352b34-8143-4a94-86e7-11a9460d2946") : object "kube-system"/"original-pull-secret" not registered
Sep 05 07:48:51.482395 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:51.482357    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wt7mt\" (UniqueName: \"kubernetes.io/projected/b1e5c942-8397-439a-96b6-13149a974b18-kube-api-access-wt7mt\") pod \"network-check-target-rbtht\" (UID: \"b1e5c942-8397-439a-96b6-13149a974b18\") " pod="openshift-network-diagnostics/network-check-target-rbtht"
Sep 05 07:48:51.482558 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:51.482511    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 05 07:48:51.482558 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:51.482532    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 05 07:48:51.482558 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:51.482541    2563 projected.go:196] Error preparing data for projected volume kube-api-access-wt7mt for pod openshift-network-diagnostics/network-check-target-rbtht: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 07:48:51.482662 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:51.482592    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b1e5c942-8397-439a-96b6-13149a974b18-kube-api-access-wt7mt podName:b1e5c942-8397-439a-96b6-13149a974b18 nodeName:}" failed. No retries permitted until 2026-09-05 07:49:23.482578654 +0000 UTC m=+66.252720972 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-wt7mt" (UniqueName: "kubernetes.io/projected/b1e5c942-8397-439a-96b6-13149a974b18-kube-api-access-wt7mt") pod "network-check-target-rbtht" (UID: "b1e5c942-8397-439a-96b6-13149a974b18") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 07:48:51.724467 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:51.724272    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9lgq"
Sep 05 07:48:51.724651 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:51.724564    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9lgq" podUID="404c81cf-16aa-49f0-9059-fb97a5791ed7"
Sep 05 07:48:51.971865 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:51.971763    2563 generic.go:356] "Generic (PLEG): container finished" podID="4d02d57b-ec85-4cd4-8da2-0ffee0574984" containerID="adf560c7dc94aaecd7a5fda27ccb5ee626920b9b20f3d2367dfc06c2aab8e35d" exitCode=0
Sep 05 07:48:51.971865 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:51.971819    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cd8wl" event={"ID":"4d02d57b-ec85-4cd4-8da2-0ffee0574984","Type":"ContainerDied","Data":"adf560c7dc94aaecd7a5fda27ccb5ee626920b9b20f3d2367dfc06c2aab8e35d"}
Sep 05 07:48:52.389209 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.389176    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:48:52.477121 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.477080    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:48:52.558145 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.558125    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-14.ec2.internal" event="NodeReady"
Sep 05 07:48:52.558276 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.558228    2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 05 07:48:52.605809 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.605778    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9wmv2"]
Sep 05 07:48:52.622358 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.622331    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-q6zgs"]
Sep 05 07:48:52.622494 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.622452    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9wmv2"
Sep 05 07:48:52.624967 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.624944    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 05 07:48:52.625120 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.625079    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 05 07:48:52.625246 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.625229    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-t6wxv\""
Sep 05 07:48:52.634458 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.634437    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9wmv2"]
Sep 05 07:48:52.634559 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.634476    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-q6zgs"]
Sep 05 07:48:52.634559 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.634547    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-q6zgs"
Sep 05 07:48:52.637471 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.637452    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 05 07:48:52.637786 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.637766    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-qqkv2\""
Sep 05 07:48:52.638283 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.638261    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 05 07:48:52.638380 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.638333    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 05 07:48:52.690947 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.690923    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2de9d5a3-384d-4f3e-866f-19f1b5d4126d-metrics-tls\") pod \"dns-default-9wmv2\" (UID: \"2de9d5a3-384d-4f3e-866f-19f1b5d4126d\") " pod="openshift-dns/dns-default-9wmv2"
Sep 05 07:48:52.691086 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.690949    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zk5tc\" (UniqueName: \"kubernetes.io/projected/2de9d5a3-384d-4f3e-866f-19f1b5d4126d-kube-api-access-zk5tc\") pod \"dns-default-9wmv2\" (UID: \"2de9d5a3-384d-4f3e-866f-19f1b5d4126d\") " pod="openshift-dns/dns-default-9wmv2"
Sep 05 07:48:52.691086 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.690983    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2de9d5a3-384d-4f3e-866f-19f1b5d4126d-config-volume\") pod \"dns-default-9wmv2\" (UID: \"2de9d5a3-384d-4f3e-866f-19f1b5d4126d\") " pod="openshift-dns/dns-default-9wmv2"
Sep 05 07:48:52.691186 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.691088    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2de9d5a3-384d-4f3e-866f-19f1b5d4126d-tmp-dir\") pod \"dns-default-9wmv2\" (UID: \"2de9d5a3-384d-4f3e-866f-19f1b5d4126d\") " pod="openshift-dns/dns-default-9wmv2"
Sep 05 07:48:52.723670 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.723651    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rbtht"
Sep 05 07:48:52.723757 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.723670    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lx5db"
Sep 05 07:48:52.725832 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.725817    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-g9xp6\""
Sep 05 07:48:52.726158 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.725816    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 05 07:48:52.726158 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.725818    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 05 07:48:52.726236 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.726158    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 05 07:48:52.732483 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.732464    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:48:52.792005 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.791978    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t642\" (UniqueName: \"kubernetes.io/projected/f2e61eb0-2689-4dfd-8b47-420342ee2916-kube-api-access-5t642\") pod \"ingress-canary-q6zgs\" (UID: \"f2e61eb0-2689-4dfd-8b47-420342ee2916\") " pod="openshift-ingress-canary/ingress-canary-q6zgs"
Sep 05 07:48:52.792130 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.792025    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2de9d5a3-384d-4f3e-866f-19f1b5d4126d-tmp-dir\") pod \"dns-default-9wmv2\" (UID: \"2de9d5a3-384d-4f3e-866f-19f1b5d4126d\") " pod="openshift-dns/dns-default-9wmv2"
Sep 05 07:48:52.792130 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.792090    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2de9d5a3-384d-4f3e-866f-19f1b5d4126d-metrics-tls\") pod \"dns-default-9wmv2\" (UID: \"2de9d5a3-384d-4f3e-866f-19f1b5d4126d\") " pod="openshift-dns/dns-default-9wmv2"
Sep 05 07:48:52.792130 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.792111    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk5tc\" (UniqueName: \"kubernetes.io/projected/2de9d5a3-384d-4f3e-866f-19f1b5d4126d-kube-api-access-zk5tc\") pod \"dns-default-9wmv2\" (UID: \"2de9d5a3-384d-4f3e-866f-19f1b5d4126d\") " pod="openshift-dns/dns-default-9wmv2"
Sep 05 07:48:52.792239 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.792136    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f2e61eb0-2689-4dfd-8b47-420342ee2916-cert\") pod \"ingress-canary-q6zgs\" (UID: \"f2e61eb0-2689-4dfd-8b47-420342ee2916\") " pod="openshift-ingress-canary/ingress-canary-q6zgs"
Sep 05 07:48:52.792239 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.792163    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2de9d5a3-384d-4f3e-866f-19f1b5d4126d-config-volume\") pod \"dns-default-9wmv2\" (UID: \"2de9d5a3-384d-4f3e-866f-19f1b5d4126d\") " pod="openshift-dns/dns-default-9wmv2"
Sep 05 07:48:52.792314 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:52.792272    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 05 07:48:52.792314 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.792289    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2de9d5a3-384d-4f3e-866f-19f1b5d4126d-tmp-dir\") pod \"dns-default-9wmv2\" (UID: \"2de9d5a3-384d-4f3e-866f-19f1b5d4126d\") " pod="openshift-dns/dns-default-9wmv2"
Sep 05 07:48:52.792393 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:52.792331    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2de9d5a3-384d-4f3e-866f-19f1b5d4126d-metrics-tls podName:2de9d5a3-384d-4f3e-866f-19f1b5d4126d nodeName:}" failed. No retries permitted until 2026-09-05 07:48:53.292314507 +0000 UTC m=+36.062456846 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2de9d5a3-384d-4f3e-866f-19f1b5d4126d-metrics-tls") pod "dns-default-9wmv2" (UID: "2de9d5a3-384d-4f3e-866f-19f1b5d4126d") : secret "dns-default-metrics-tls" not found
Sep 05 07:48:52.792686 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.792670    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2de9d5a3-384d-4f3e-866f-19f1b5d4126d-config-volume\") pod \"dns-default-9wmv2\" (UID: \"2de9d5a3-384d-4f3e-866f-19f1b5d4126d\") " pod="openshift-dns/dns-default-9wmv2"
Sep 05 07:48:52.802070 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.802043    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zk5tc\" (UniqueName: \"kubernetes.io/projected/2de9d5a3-384d-4f3e-866f-19f1b5d4126d-kube-api-access-zk5tc\") pod \"dns-default-9wmv2\" (UID: \"2de9d5a3-384d-4f3e-866f-19f1b5d4126d\") " pod="openshift-dns/dns-default-9wmv2"
Sep 05 07:48:52.834068 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.834017    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:48:52.892657 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.892628    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f2e61eb0-2689-4dfd-8b47-420342ee2916-cert\") pod \"ingress-canary-q6zgs\" (UID: \"f2e61eb0-2689-4dfd-8b47-420342ee2916\") " pod="openshift-ingress-canary/ingress-canary-q6zgs"
Sep 05 07:48:52.892780 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.892671    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t642\" (UniqueName: \"kubernetes.io/projected/f2e61eb0-2689-4dfd-8b47-420342ee2916-kube-api-access-5t642\") pod \"ingress-canary-q6zgs\" (UID: \"f2e61eb0-2689-4dfd-8b47-420342ee2916\") " pod="openshift-ingress-canary/ingress-canary-q6zgs"
Sep 05 07:48:52.895183 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.895158    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f2e61eb0-2689-4dfd-8b47-420342ee2916-cert\") pod \"ingress-canary-q6zgs\" (UID: \"f2e61eb0-2689-4dfd-8b47-420342ee2916\") " pod="openshift-ingress-canary/ingress-canary-q6zgs"
Sep 05 07:48:52.902098 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.902073    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t642\" (UniqueName: \"kubernetes.io/projected/f2e61eb0-2689-4dfd-8b47-420342ee2916-kube-api-access-5t642\") pod \"ingress-canary-q6zgs\" (UID: \"f2e61eb0-2689-4dfd-8b47-420342ee2916\") " pod="openshift-ingress-canary/ingress-canary-q6zgs"
Sep 05 07:48:52.942120 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.942094    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-q6zgs"
Sep 05 07:48:52.980411 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.980383    2563 generic.go:356] "Generic (PLEG): container finished" podID="4d02d57b-ec85-4cd4-8da2-0ffee0574984" containerID="cf402377cf583c7ef3c9e4363645057b75589d901122e3c3caa7292bc3ed8c68" exitCode=0
Sep 05 07:48:52.980526 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:52.980445    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cd8wl" event={"ID":"4d02d57b-ec85-4cd4-8da2-0ffee0574984","Type":"ContainerDied","Data":"cf402377cf583c7ef3c9e4363645057b75589d901122e3c3caa7292bc3ed8c68"}
Sep 05 07:48:53.259290 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:53.259094    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-q6zgs"]
Sep 05 07:48:53.263375 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:48:53.263352    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2e61eb0_2689_4dfd_8b47_420342ee2916.slice/crio-69851f8bfffea278b3f637c4e034c87f649931c70ac0b17fab99dd8b72c11b36 WatchSource:0}: Error finding container 69851f8bfffea278b3f637c4e034c87f649931c70ac0b17fab99dd8b72c11b36: Status 404 returned error can't find the container with id 69851f8bfffea278b3f637c4e034c87f649931c70ac0b17fab99dd8b72c11b36
Sep 05 07:48:53.296016 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:53.295991    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2de9d5a3-384d-4f3e-866f-19f1b5d4126d-metrics-tls\") pod \"dns-default-9wmv2\" (UID: \"2de9d5a3-384d-4f3e-866f-19f1b5d4126d\") " pod="openshift-dns/dns-default-9wmv2"
Sep 05 07:48:53.296122 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:53.296110    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 05 07:48:53.296165 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:53.296160    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2de9d5a3-384d-4f3e-866f-19f1b5d4126d-metrics-tls podName:2de9d5a3-384d-4f3e-866f-19f1b5d4126d nodeName:}" failed. No retries permitted until 2026-09-05 07:48:54.296147142 +0000 UTC m=+37.066289457 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2de9d5a3-384d-4f3e-866f-19f1b5d4126d-metrics-tls") pod "dns-default-9wmv2" (UID: "2de9d5a3-384d-4f3e-866f-19f1b5d4126d") : secret "dns-default-metrics-tls" not found
Sep 05 07:48:53.435489 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:53.435409    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:48:53.495164 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:53.495135    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:48:53.725104 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:53.724560    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9lgq"
Sep 05 07:48:53.727182 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:53.727148    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hp5x9\""
Sep 05 07:48:53.727960 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:53.727241    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 05 07:48:53.984762 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:53.984677    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-q6zgs" event={"ID":"f2e61eb0-2689-4dfd-8b47-420342ee2916","Type":"ContainerStarted","Data":"69851f8bfffea278b3f637c4e034c87f649931c70ac0b17fab99dd8b72c11b36"}
Sep 05 07:48:53.988805 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:53.988774    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cd8wl" event={"ID":"4d02d57b-ec85-4cd4-8da2-0ffee0574984","Type":"ContainerStarted","Data":"0af202dd13e1e67ab2e5969af3cb394bc22df3532b24d49e7279dcd2b02df0c0"}
Sep 05 07:48:54.009904 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:54.009826    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-cd8wl" podStartSLOduration=7.353281994 podStartE2EDuration="36.009806862s" totalImagesPullingTime="28.656524868s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 07:48:18 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:48:54.008826601 +0000 UTC m=+36.778968949" watchObservedRunningTime="2026-09-05 07:48:54.009806862 +0000 UTC m=+36.779949200"
Sep 05 07:48:54.134782 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:54.134749    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:48:54.192100 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:54.192061    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:48:54.304798 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:54.304776    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2de9d5a3-384d-4f3e-866f-19f1b5d4126d-metrics-tls\") pod \"dns-default-9wmv2\" (UID: \"2de9d5a3-384d-4f3e-866f-19f1b5d4126d\") " pod="openshift-dns/dns-default-9wmv2"
Sep 05 07:48:54.305039 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:54.304933    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 05 07:48:54.305039 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:54.304996    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2de9d5a3-384d-4f3e-866f-19f1b5d4126d-metrics-tls podName:2de9d5a3-384d-4f3e-866f-19f1b5d4126d nodeName:}" failed. No retries permitted until 2026-09-05 07:48:56.304980721 +0000 UTC m=+39.075123036 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2de9d5a3-384d-4f3e-866f-19f1b5d4126d-metrics-tls") pod "dns-default-9wmv2" (UID: "2de9d5a3-384d-4f3e-866f-19f1b5d4126d") : secret "dns-default-metrics-tls" not found
Sep 05 07:48:54.336304 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:54.336275    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:48:54.434649 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:54.434613    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:48:55.995005 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:55.994972    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-q6zgs" event={"ID":"f2e61eb0-2689-4dfd-8b47-420342ee2916","Type":"ContainerStarted","Data":"0849bdb96b1ac8e2be1c704251869d6ae736a8a653476803c0cb85f459f689ac"}
Sep 05 07:48:56.319916 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:56.319879    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2de9d5a3-384d-4f3e-866f-19f1b5d4126d-metrics-tls\") pod \"dns-default-9wmv2\" (UID: \"2de9d5a3-384d-4f3e-866f-19f1b5d4126d\") " pod="openshift-dns/dns-default-9wmv2"
Sep 05 07:48:56.320058 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:56.320039    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 05 07:48:56.320113 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:48:56.320102    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2de9d5a3-384d-4f3e-866f-19f1b5d4126d-metrics-tls podName:2de9d5a3-384d-4f3e-866f-19f1b5d4126d nodeName:}" failed. No retries permitted until 2026-09-05 07:49:00.320085662 +0000 UTC m=+43.090227998 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2de9d5a3-384d-4f3e-866f-19f1b5d4126d-metrics-tls") pod "dns-default-9wmv2" (UID: "2de9d5a3-384d-4f3e-866f-19f1b5d4126d") : secret "dns-default-metrics-tls" not found
Sep 05 07:48:57.396178 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:57.395997    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:48:57.458384 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:57.458357    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:48:57.531394 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:57.531364    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:48:57.632533 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:48:57.632500    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:00.348013 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:00.347979    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2de9d5a3-384d-4f3e-866f-19f1b5d4126d-metrics-tls\") pod \"dns-default-9wmv2\" (UID: \"2de9d5a3-384d-4f3e-866f-19f1b5d4126d\") " pod="openshift-dns/dns-default-9wmv2"
Sep 05 07:49:00.348414 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:00.348106    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 05 07:49:00.348414 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:00.348160    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2de9d5a3-384d-4f3e-866f-19f1b5d4126d-metrics-tls podName:2de9d5a3-384d-4f3e-866f-19f1b5d4126d nodeName:}" failed. No retries permitted until 2026-09-05 07:49:08.348145771 +0000 UTC m=+51.118288085 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2de9d5a3-384d-4f3e-866f-19f1b5d4126d-metrics-tls") pod "dns-default-9wmv2" (UID: "2de9d5a3-384d-4f3e-866f-19f1b5d4126d") : secret "dns-default-metrics-tls" not found
Sep 05 07:49:00.873574 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:00.873519    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-q6zgs" podStartSLOduration=6.333182328 podStartE2EDuration="8.873499789s" totalImagesPullingTime="2.540317461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:48:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:48:56.009294086 +0000 UTC m=+38.779436450" watchObservedRunningTime="2026-09-05 07:49:00.873499789 +0000 UTC m=+43.643642125"
Sep 05 07:49:00.873963 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:00.873947    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-vtq4p"]
Sep 05 07:49:00.912217 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:00.912182    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-dcbfv"]
Sep 05 07:49:00.912366 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:00.912323    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-vtq4p"
Sep 05 07:49:00.914567 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:00.914538    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 05 07:49:00.915129 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:00.915112    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 05 07:49:00.915187 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:00.915117    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-bfjqs\""
Sep 05 07:49:00.935764 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:00.935742    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-vtq4p"]
Sep 05 07:49:00.935888 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:00.935776    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-dcbfv"]
Sep 05 07:49:00.935953 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:00.935885    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-dcbfv"
Sep 05 07:49:00.937940 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:00.937918    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 05 07:49:00.938147 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:00.938134    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-4ff8m\""
Sep 05 07:49:00.938238 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:00.938226    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 05 07:49:00.938287 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:00.938269    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 05 07:49:00.938337 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:00.938229    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 05 07:49:00.956610 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:00.956593    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 05 07:49:00.977655 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:00.977637    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-hxgpj"]
Sep 05 07:49:00.990942 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:00.990923    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-rzj45"]
Sep 05 07:49:00.991048 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:00.991035    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-hxgpj"
Sep 05 07:49:00.995414 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:00.995389    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 05 07:49:00.995510 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:00.995407    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 05 07:49:00.995691 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:00.995671    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-dv72v\""
Sep 05 07:49:00.996271 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:00.996253    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 05 07:49:01.015949 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.015928    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-8557f77dd-lw4qb"]
Sep 05 07:49:01.016067 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.016043    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-rzj45"
Sep 05 07:49:01.018044 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.018026    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 05 07:49:01.018146 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.018074    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-v8bls\""
Sep 05 07:49:01.018336 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.018311    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 05 07:49:01.018336 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.018318    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 05 07:49:01.018478 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.018359    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 05 07:49:01.033707 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.033687    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-hxgpj"]
Sep 05 07:49:01.033798 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.033726    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-8557f77dd-lw4qb"
Sep 05 07:49:01.033798 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.033731    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-rzj45"]
Sep 05 07:49:01.033798 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.033769    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-8557f77dd-lw4qb"]
Sep 05 07:49:01.035791 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.035770    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 05 07:49:01.035902 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.035802    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-ml7q9\""
Sep 05 07:49:01.035902 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.035806    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 05 07:49:01.036081 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.036063    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 05 07:49:01.036180 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.036088    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 05 07:49:01.036180 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.036120    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 05 07:49:01.036180 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.036123    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 05 07:49:01.052928 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.052904    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/999f5178-4c4b-4888-a348-c0d434b38967-tmp\") pod \"insights-operator-76bcd6ccf5-dcbfv\" (UID: \"999f5178-4c4b-4888-a348-c0d434b38967\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dcbfv"
Sep 05 07:49:01.053031 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.052937    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/999f5178-4c4b-4888-a348-c0d434b38967-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-dcbfv\" (UID: \"999f5178-4c4b-4888-a348-c0d434b38967\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dcbfv"
Sep 05 07:49:01.053031 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.052956    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/999f5178-4c4b-4888-a348-c0d434b38967-snapshots\") pod \"insights-operator-76bcd6ccf5-dcbfv\" (UID: \"999f5178-4c4b-4888-a348-c0d434b38967\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dcbfv"
Sep 05 07:49:01.053142 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.053032    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsxwh\" (UniqueName: \"kubernetes.io/projected/9da5c2fc-7ecf-41c4-b814-ae36d6909e16-kube-api-access-zsxwh\") pod \"volume-data-source-validator-695c895bd6-vtq4p\" (UID: \"9da5c2fc-7ecf-41c4-b814-ae36d6909e16\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-vtq4p"
Sep 05 07:49:01.053142 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.053124    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/999f5178-4c4b-4888-a348-c0d434b38967-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-dcbfv\" (UID: \"999f5178-4c4b-4888-a348-c0d434b38967\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dcbfv"
Sep 05 07:49:01.053217 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.053145    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/999f5178-4c4b-4888-a348-c0d434b38967-serving-cert\") pod \"insights-operator-76bcd6ccf5-dcbfv\" (UID: \"999f5178-4c4b-4888-a348-c0d434b38967\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dcbfv"
Sep 05 07:49:01.053217 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.053180    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghnwf\" (UniqueName: \"kubernetes.io/projected/999f5178-4c4b-4888-a348-c0d434b38967-kube-api-access-ghnwf\") pod \"insights-operator-76bcd6ccf5-dcbfv\" (UID: \"999f5178-4c4b-4888-a348-c0d434b38967\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dcbfv"
Sep 05 07:49:01.145498 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.145417    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:01.154409 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.154385    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/999f5178-4c4b-4888-a348-c0d434b38967-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-dcbfv\" (UID: \"999f5178-4c4b-4888-a348-c0d434b38967\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dcbfv"
Sep 05 07:49:01.154487 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.154416    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/999f5178-4c4b-4888-a348-c0d434b38967-serving-cert\") pod \"insights-operator-76bcd6ccf5-dcbfv\" (UID: \"999f5178-4c4b-4888-a348-c0d434b38967\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dcbfv"
Sep 05 07:49:01.154487 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.154449    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghnwf\" (UniqueName: \"kubernetes.io/projected/999f5178-4c4b-4888-a348-c0d434b38967-kube-api-access-ghnwf\") pod \"insights-operator-76bcd6ccf5-dcbfv\" (UID: \"999f5178-4c4b-4888-a348-c0d434b38967\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dcbfv"
Sep 05 07:49:01.154714 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.154695    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bj2wp\" (UniqueName: \"kubernetes.io/projected/143aee12-5c44-440c-ac06-1a5146e2d401-kube-api-access-bj2wp\") pod \"kube-storage-version-migrator-operator-8d7dd4675-rzj45\" (UID: \"143aee12-5c44-440c-ac06-1a5146e2d401\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-rzj45"
Sep 05 07:49:01.154782 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.154741    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9163aed9-c681-4b4d-a7f9-63438c02656f-metrics-certs\") pod \"router-default-8557f77dd-lw4qb\" (UID: \"9163aed9-c681-4b4d-a7f9-63438c02656f\") " pod="openshift-ingress/router-default-8557f77dd-lw4qb"
Sep 05 07:49:01.154782 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.154773    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/143aee12-5c44-440c-ac06-1a5146e2d401-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-rzj45\" (UID: \"143aee12-5c44-440c-ac06-1a5146e2d401\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-rzj45"
Sep 05 07:49:01.154898 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.154820    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9b6376b1-fe4f-4e8f-b2aa-18b7d640d685-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-hxgpj\" (UID: \"9b6376b1-fe4f-4e8f-b2aa-18b7d640d685\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-hxgpj"
Sep 05 07:49:01.154898 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.154872    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/999f5178-4c4b-4888-a348-c0d434b38967-tmp\") pod \"insights-operator-76bcd6ccf5-dcbfv\" (UID: \"999f5178-4c4b-4888-a348-c0d434b38967\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dcbfv"
Sep 05 07:49:01.155004 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.154905    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdk7z\" (UniqueName: \"kubernetes.io/projected/9b6376b1-fe4f-4e8f-b2aa-18b7d640d685-kube-api-access-fdk7z\") pod \"cluster-samples-operator-7cdcfcf76c-hxgpj\" (UID: \"9b6376b1-fe4f-4e8f-b2aa-18b7d640d685\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-hxgpj"
Sep 05 07:49:01.155004 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.154937    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/999f5178-4c4b-4888-a348-c0d434b38967-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-dcbfv\" (UID: \"999f5178-4c4b-4888-a348-c0d434b38967\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dcbfv"
Sep 05 07:49:01.155004 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.154960    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/143aee12-5c44-440c-ac06-1a5146e2d401-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-rzj45\" (UID: \"143aee12-5c44-440c-ac06-1a5146e2d401\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-rzj45"
Sep 05 07:49:01.155142 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.155015    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/999f5178-4c4b-4888-a348-c0d434b38967-snapshots\") pod \"insights-operator-76bcd6ccf5-dcbfv\" (UID: \"999f5178-4c4b-4888-a348-c0d434b38967\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dcbfv"
Sep 05 07:49:01.155142 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.155077    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9163aed9-c681-4b4d-a7f9-63438c02656f-service-ca-bundle\") pod \"router-default-8557f77dd-lw4qb\" (UID: \"9163aed9-c681-4b4d-a7f9-63438c02656f\") " pod="openshift-ingress/router-default-8557f77dd-lw4qb"
Sep 05 07:49:01.155142 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.155116    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsxwh\" (UniqueName: \"kubernetes.io/projected/9da5c2fc-7ecf-41c4-b814-ae36d6909e16-kube-api-access-zsxwh\") pod \"volume-data-source-validator-695c895bd6-vtq4p\" (UID: \"9da5c2fc-7ecf-41c4-b814-ae36d6909e16\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-vtq4p"
Sep 05 07:49:01.155259 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.155148    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9163aed9-c681-4b4d-a7f9-63438c02656f-default-certificate\") pod \"router-default-8557f77dd-lw4qb\" (UID: \"9163aed9-c681-4b4d-a7f9-63438c02656f\") " pod="openshift-ingress/router-default-8557f77dd-lw4qb"
Sep 05 07:49:01.155259 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.155170    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9163aed9-c681-4b4d-a7f9-63438c02656f-stats-auth\") pod \"router-default-8557f77dd-lw4qb\" (UID: \"9163aed9-c681-4b4d-a7f9-63438c02656f\") " pod="openshift-ingress/router-default-8557f77dd-lw4qb"
Sep 05 07:49:01.155259 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.155193    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5zqk\" (UniqueName: \"kubernetes.io/projected/9163aed9-c681-4b4d-a7f9-63438c02656f-kube-api-access-s5zqk\") pod \"router-default-8557f77dd-lw4qb\" (UID: \"9163aed9-c681-4b4d-a7f9-63438c02656f\") " pod="openshift-ingress/router-default-8557f77dd-lw4qb"
Sep 05 07:49:01.155259 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.155222    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/999f5178-4c4b-4888-a348-c0d434b38967-tmp\") pod \"insights-operator-76bcd6ccf5-dcbfv\" (UID: \"999f5178-4c4b-4888-a348-c0d434b38967\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dcbfv"
Sep 05 07:49:01.155259 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.155234    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/999f5178-4c4b-4888-a348-c0d434b38967-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-dcbfv\" (UID: \"999f5178-4c4b-4888-a348-c0d434b38967\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dcbfv"
Sep 05 07:49:01.155553 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.155533    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/999f5178-4c4b-4888-a348-c0d434b38967-snapshots\") pod \"insights-operator-76bcd6ccf5-dcbfv\" (UID: \"999f5178-4c4b-4888-a348-c0d434b38967\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dcbfv"
Sep 05 07:49:01.155721 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.155703    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/999f5178-4c4b-4888-a348-c0d434b38967-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-dcbfv\" (UID: \"999f5178-4c4b-4888-a348-c0d434b38967\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dcbfv"
Sep 05 07:49:01.158447 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.158426    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/999f5178-4c4b-4888-a348-c0d434b38967-serving-cert\") pod \"insights-operator-76bcd6ccf5-dcbfv\" (UID: \"999f5178-4c4b-4888-a348-c0d434b38967\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dcbfv"
Sep 05 07:49:01.162583 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.162559    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghnwf\" (UniqueName: \"kubernetes.io/projected/999f5178-4c4b-4888-a348-c0d434b38967-kube-api-access-ghnwf\") pod \"insights-operator-76bcd6ccf5-dcbfv\" (UID: \"999f5178-4c4b-4888-a348-c0d434b38967\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dcbfv"
Sep 05 07:49:01.163511 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.163493    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsxwh\" (UniqueName: \"kubernetes.io/projected/9da5c2fc-7ecf-41c4-b814-ae36d6909e16-kube-api-access-zsxwh\") pod \"volume-data-source-validator-695c895bd6-vtq4p\" (UID: \"9da5c2fc-7ecf-41c4-b814-ae36d6909e16\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-vtq4p"
Sep 05 07:49:01.175181 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.175161    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-g27lz"]
Sep 05 07:49:01.202246 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.202222    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-79bddd854d-9vlsf"]
Sep 05 07:49:01.202355 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.202338    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-g27lz"
Sep 05 07:49:01.204427 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.204403    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 05 07:49:01.204878 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.204862    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 05 07:49:01.204961 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.204900    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-676t7\""
Sep 05 07:49:01.221888 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.221870    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-vtq4p"
Sep 05 07:49:01.229040 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.229023    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-g27lz"]
Sep 05 07:49:01.229097 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.229052    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-79bddd854d-9vlsf"]
Sep 05 07:49:01.229137 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.229123    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79bddd854d-9vlsf"
Sep 05 07:49:01.231713 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.231694    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 05 07:49:01.231806 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.231740    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 05 07:49:01.231806 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.231770    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 05 07:49:01.232162 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.232140    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-z2wx8\""
Sep 05 07:49:01.236483 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.236450    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 05 07:49:01.245135 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.245107    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-dcbfv"
Sep 05 07:49:01.252579 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.252555    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:01.256046 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.256026    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bj2wp\" (UniqueName: \"kubernetes.io/projected/143aee12-5c44-440c-ac06-1a5146e2d401-kube-api-access-bj2wp\") pod \"kube-storage-version-migrator-operator-8d7dd4675-rzj45\" (UID: \"143aee12-5c44-440c-ac06-1a5146e2d401\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-rzj45"
Sep 05 07:49:01.256126 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.256057    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9163aed9-c681-4b4d-a7f9-63438c02656f-metrics-certs\") pod \"router-default-8557f77dd-lw4qb\" (UID: \"9163aed9-c681-4b4d-a7f9-63438c02656f\") " pod="openshift-ingress/router-default-8557f77dd-lw4qb"
Sep 05 07:49:01.256126 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.256075    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/143aee12-5c44-440c-ac06-1a5146e2d401-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-rzj45\" (UID: \"143aee12-5c44-440c-ac06-1a5146e2d401\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-rzj45"
Sep 05 07:49:01.256126 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.256110    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9b6376b1-fe4f-4e8f-b2aa-18b7d640d685-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-hxgpj\" (UID: \"9b6376b1-fe4f-4e8f-b2aa-18b7d640d685\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-hxgpj"
Sep 05 07:49:01.256255 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.256134    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdk7z\" (UniqueName: \"kubernetes.io/projected/9b6376b1-fe4f-4e8f-b2aa-18b7d640d685-kube-api-access-fdk7z\") pod \"cluster-samples-operator-7cdcfcf76c-hxgpj\" (UID: \"9b6376b1-fe4f-4e8f-b2aa-18b7d640d685\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-hxgpj"
Sep 05 07:49:01.256255 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.256161    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/143aee12-5c44-440c-ac06-1a5146e2d401-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-rzj45\" (UID: \"143aee12-5c44-440c-ac06-1a5146e2d401\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-rzj45"
Sep 05 07:49:01.256255 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:01.256203    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 05 07:49:01.256255 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.256211    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9163aed9-c681-4b4d-a7f9-63438c02656f-service-ca-bundle\") pod \"router-default-8557f77dd-lw4qb\" (UID: \"9163aed9-c681-4b4d-a7f9-63438c02656f\") " pod="openshift-ingress/router-default-8557f77dd-lw4qb"
Sep 05 07:49:01.256255 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.256249    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9163aed9-c681-4b4d-a7f9-63438c02656f-default-certificate\") pod \"router-default-8557f77dd-lw4qb\" (UID: \"9163aed9-c681-4b4d-a7f9-63438c02656f\") " pod="openshift-ingress/router-default-8557f77dd-lw4qb"
Sep 05 07:49:01.256448 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:01.256276    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9163aed9-c681-4b4d-a7f9-63438c02656f-metrics-certs podName:9163aed9-c681-4b4d-a7f9-63438c02656f nodeName:}" failed. No retries permitted until 2026-09-05 07:49:01.756256337 +0000 UTC m=+44.526398651 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9163aed9-c681-4b4d-a7f9-63438c02656f-metrics-certs") pod "router-default-8557f77dd-lw4qb" (UID: "9163aed9-c681-4b4d-a7f9-63438c02656f") : secret "router-metrics-certs-default" not found
Sep 05 07:49:01.256448 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.256313    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9163aed9-c681-4b4d-a7f9-63438c02656f-stats-auth\") pod \"router-default-8557f77dd-lw4qb\" (UID: \"9163aed9-c681-4b4d-a7f9-63438c02656f\") " pod="openshift-ingress/router-default-8557f77dd-lw4qb"
Sep 05 07:49:01.256448 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.256349    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5zqk\" (UniqueName: \"kubernetes.io/projected/9163aed9-c681-4b4d-a7f9-63438c02656f-kube-api-access-s5zqk\") pod \"router-default-8557f77dd-lw4qb\" (UID: \"9163aed9-c681-4b4d-a7f9-63438c02656f\") " pod="openshift-ingress/router-default-8557f77dd-lw4qb"
Sep 05 07:49:01.256588 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:01.256533    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 05 07:49:01.256640 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:01.256597    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9163aed9-c681-4b4d-a7f9-63438c02656f-service-ca-bundle podName:9163aed9-c681-4b4d-a7f9-63438c02656f nodeName:}" failed. No retries permitted until 2026-09-05 07:49:01.756578514 +0000 UTC m=+44.526720845 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9163aed9-c681-4b4d-a7f9-63438c02656f-service-ca-bundle") pod "router-default-8557f77dd-lw4qb" (UID: "9163aed9-c681-4b4d-a7f9-63438c02656f") : configmap references non-existent config key: service-ca.crt
Sep 05 07:49:01.256640 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:01.256625    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b6376b1-fe4f-4e8f-b2aa-18b7d640d685-samples-operator-tls podName:9b6376b1-fe4f-4e8f-b2aa-18b7d640d685 nodeName:}" failed. No retries permitted until 2026-09-05 07:49:01.756613847 +0000 UTC m=+44.526756177 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9b6376b1-fe4f-4e8f-b2aa-18b7d640d685-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-hxgpj" (UID: "9b6376b1-fe4f-4e8f-b2aa-18b7d640d685") : secret "samples-operator-tls" not found
Sep 05 07:49:01.257323 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.257262    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/143aee12-5c44-440c-ac06-1a5146e2d401-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-rzj45\" (UID: \"143aee12-5c44-440c-ac06-1a5146e2d401\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-rzj45"
Sep 05 07:49:01.258593 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.258568    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/143aee12-5c44-440c-ac06-1a5146e2d401-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-rzj45\" (UID: \"143aee12-5c44-440c-ac06-1a5146e2d401\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-rzj45"
Sep 05 07:49:01.258826 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.258807    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9163aed9-c681-4b4d-a7f9-63438c02656f-default-certificate\") pod \"router-default-8557f77dd-lw4qb\" (UID: \"9163aed9-c681-4b4d-a7f9-63438c02656f\") " pod="openshift-ingress/router-default-8557f77dd-lw4qb"
Sep 05 07:49:01.259090 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.259071    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9163aed9-c681-4b4d-a7f9-63438c02656f-stats-auth\") pod \"router-default-8557f77dd-lw4qb\" (UID: \"9163aed9-c681-4b4d-a7f9-63438c02656f\") " pod="openshift-ingress/router-default-8557f77dd-lw4qb"
Sep 05 07:49:01.267069 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.267042    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bj2wp\" (UniqueName: \"kubernetes.io/projected/143aee12-5c44-440c-ac06-1a5146e2d401-kube-api-access-bj2wp\") pod \"kube-storage-version-migrator-operator-8d7dd4675-rzj45\" (UID: \"143aee12-5c44-440c-ac06-1a5146e2d401\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-rzj45"
Sep 05 07:49:01.267225 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.267207    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdk7z\" (UniqueName: \"kubernetes.io/projected/9b6376b1-fe4f-4e8f-b2aa-18b7d640d685-kube-api-access-fdk7z\") pod \"cluster-samples-operator-7cdcfcf76c-hxgpj\" (UID: \"9b6376b1-fe4f-4e8f-b2aa-18b7d640d685\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-hxgpj"
Sep 05 07:49:01.270773 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.270746    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5zqk\" (UniqueName: \"kubernetes.io/projected/9163aed9-c681-4b4d-a7f9-63438c02656f-kube-api-access-s5zqk\") pod \"router-default-8557f77dd-lw4qb\" (UID: \"9163aed9-c681-4b4d-a7f9-63438c02656f\") " pod="openshift-ingress/router-default-8557f77dd-lw4qb"
Sep 05 07:49:01.324645 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.324601    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-rzj45"
Sep 05 07:49:01.327412 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.327386    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:01.357454 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.357429    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cafd9630-0d63-4c63-9829-bb8f215e97ac-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-g27lz\" (UID: \"cafd9630-0d63-4c63-9829-bb8f215e97ac\") " pod="openshift-network-console/networking-console-plugin-79977f9844-g27lz"
Sep 05 07:49:01.357745 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.357465    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/27817b6a-02ee-4fde-856f-221f422e231f-ca-trust-extracted\") pod \"image-registry-79bddd854d-9vlsf\" (UID: \"27817b6a-02ee-4fde-856f-221f422e231f\") " pod="openshift-image-registry/image-registry-79bddd854d-9vlsf"
Sep 05 07:49:01.357745 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.357498    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/27817b6a-02ee-4fde-856f-221f422e231f-registry-certificates\") pod \"image-registry-79bddd854d-9vlsf\" (UID: \"27817b6a-02ee-4fde-856f-221f422e231f\") " pod="openshift-image-registry/image-registry-79bddd854d-9vlsf"
Sep 05 07:49:01.357745 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.357513    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tb54t\" (UniqueName: \"kubernetes.io/projected/27817b6a-02ee-4fde-856f-221f422e231f-kube-api-access-tb54t\") pod \"image-registry-79bddd854d-9vlsf\" (UID: \"27817b6a-02ee-4fde-856f-221f422e231f\") " pod="openshift-image-registry/image-registry-79bddd854d-9vlsf"
Sep 05 07:49:01.357745 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.357558    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/27817b6a-02ee-4fde-856f-221f422e231f-bound-sa-token\") pod \"image-registry-79bddd854d-9vlsf\" (UID: \"27817b6a-02ee-4fde-856f-221f422e231f\") " pod="openshift-image-registry/image-registry-79bddd854d-9vlsf"
Sep 05 07:49:01.357745 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.357601    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/27817b6a-02ee-4fde-856f-221f422e231f-registry-tls\") pod \"image-registry-79bddd854d-9vlsf\" (UID: \"27817b6a-02ee-4fde-856f-221f422e231f\") " pod="openshift-image-registry/image-registry-79bddd854d-9vlsf"
Sep 05 07:49:01.357745 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.357621    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cafd9630-0d63-4c63-9829-bb8f215e97ac-nginx-conf\") pod \"networking-console-plugin-79977f9844-g27lz\" (UID: \"cafd9630-0d63-4c63-9829-bb8f215e97ac\") " pod="openshift-network-console/networking-console-plugin-79977f9844-g27lz"
Sep 05 07:49:01.357745 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.357727    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/27817b6a-02ee-4fde-856f-221f422e231f-image-registry-private-configuration\") pod \"image-registry-79bddd854d-9vlsf\" (UID: \"27817b6a-02ee-4fde-856f-221f422e231f\") " pod="openshift-image-registry/image-registry-79bddd854d-9vlsf"
Sep 05 07:49:01.357998 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.357756    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/27817b6a-02ee-4fde-856f-221f422e231f-installation-pull-secrets\") pod \"image-registry-79bddd854d-9vlsf\" (UID: \"27817b6a-02ee-4fde-856f-221f422e231f\") " pod="openshift-image-registry/image-registry-79bddd854d-9vlsf"
Sep 05 07:49:01.357998 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.357791    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/27817b6a-02ee-4fde-856f-221f422e231f-trusted-ca\") pod \"image-registry-79bddd854d-9vlsf\" (UID: \"27817b6a-02ee-4fde-856f-221f422e231f\") " pod="openshift-image-registry/image-registry-79bddd854d-9vlsf"
Sep 05 07:49:01.426009 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.425971    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:01.458790 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.458759    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/27817b6a-02ee-4fde-856f-221f422e231f-trusted-ca\") pod \"image-registry-79bddd854d-9vlsf\" (UID: \"27817b6a-02ee-4fde-856f-221f422e231f\") " pod="openshift-image-registry/image-registry-79bddd854d-9vlsf"
Sep 05 07:49:01.458790 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.458793    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cafd9630-0d63-4c63-9829-bb8f215e97ac-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-g27lz\" (UID: \"cafd9630-0d63-4c63-9829-bb8f215e97ac\") " pod="openshift-network-console/networking-console-plugin-79977f9844-g27lz"
Sep 05 07:49:01.459022 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.458810    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/27817b6a-02ee-4fde-856f-221f422e231f-ca-trust-extracted\") pod \"image-registry-79bddd854d-9vlsf\" (UID: \"27817b6a-02ee-4fde-856f-221f422e231f\") " pod="openshift-image-registry/image-registry-79bddd854d-9vlsf"
Sep 05 07:49:01.459022 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.458833    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/27817b6a-02ee-4fde-856f-221f422e231f-registry-certificates\") pod \"image-registry-79bddd854d-9vlsf\" (UID: \"27817b6a-02ee-4fde-856f-221f422e231f\") " pod="openshift-image-registry/image-registry-79bddd854d-9vlsf"
Sep 05 07:49:01.459022 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.458868    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tb54t\" (UniqueName: \"kubernetes.io/projected/27817b6a-02ee-4fde-856f-221f422e231f-kube-api-access-tb54t\") pod \"image-registry-79bddd854d-9vlsf\" (UID: \"27817b6a-02ee-4fde-856f-221f422e231f\") " pod="openshift-image-registry/image-registry-79bddd854d-9vlsf"
Sep 05 07:49:01.459022 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.458893    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/27817b6a-02ee-4fde-856f-221f422e231f-bound-sa-token\") pod \"image-registry-79bddd854d-9vlsf\" (UID: \"27817b6a-02ee-4fde-856f-221f422e231f\") " pod="openshift-image-registry/image-registry-79bddd854d-9vlsf"
Sep 05 07:49:01.459022 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.458926    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/27817b6a-02ee-4fde-856f-221f422e231f-registry-tls\") pod \"image-registry-79bddd854d-9vlsf\" (UID: \"27817b6a-02ee-4fde-856f-221f422e231f\") " pod="openshift-image-registry/image-registry-79bddd854d-9vlsf"
Sep 05 07:49:01.459022 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.458960    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cafd9630-0d63-4c63-9829-bb8f215e97ac-nginx-conf\") pod \"networking-console-plugin-79977f9844-g27lz\" (UID: \"cafd9630-0d63-4c63-9829-bb8f215e97ac\") " pod="openshift-network-console/networking-console-plugin-79977f9844-g27lz"
Sep 05 07:49:01.459022 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:01.458961    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 05 07:49:01.459596 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:01.459045    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cafd9630-0d63-4c63-9829-bb8f215e97ac-networking-console-plugin-cert podName:cafd9630-0d63-4c63-9829-bb8f215e97ac nodeName:}" failed. No retries permitted until 2026-09-05 07:49:01.959023575 +0000 UTC m=+44.729165904 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cafd9630-0d63-4c63-9829-bb8f215e97ac-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-g27lz" (UID: "cafd9630-0d63-4c63-9829-bb8f215e97ac") : secret "networking-console-plugin-cert" not found
Sep 05 07:49:01.459596 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.459083    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/27817b6a-02ee-4fde-856f-221f422e231f-image-registry-private-configuration\") pod \"image-registry-79bddd854d-9vlsf\" (UID: \"27817b6a-02ee-4fde-856f-221f422e231f\") " pod="openshift-image-registry/image-registry-79bddd854d-9vlsf"
Sep 05 07:49:01.459596 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.459131    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/27817b6a-02ee-4fde-856f-221f422e231f-installation-pull-secrets\") pod \"image-registry-79bddd854d-9vlsf\" (UID: \"27817b6a-02ee-4fde-856f-221f422e231f\") " pod="openshift-image-registry/image-registry-79bddd854d-9vlsf"
Sep 05 07:49:01.459596 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:01.459255    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 05 07:49:01.459596 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:01.459272    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79bddd854d-9vlsf: secret "image-registry-tls" not found
Sep 05 07:49:01.459596 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.459272    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/27817b6a-02ee-4fde-856f-221f422e231f-ca-trust-extracted\") pod \"image-registry-79bddd854d-9vlsf\" (UID: \"27817b6a-02ee-4fde-856f-221f422e231f\") " pod="openshift-image-registry/image-registry-79bddd854d-9vlsf"
Sep 05 07:49:01.459596 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:01.459341    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27817b6a-02ee-4fde-856f-221f422e231f-registry-tls podName:27817b6a-02ee-4fde-856f-221f422e231f nodeName:}" failed. No retries permitted until 2026-09-05 07:49:01.959324695 +0000 UTC m=+44.729467009 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/27817b6a-02ee-4fde-856f-221f422e231f-registry-tls") pod "image-registry-79bddd854d-9vlsf" (UID: "27817b6a-02ee-4fde-856f-221f422e231f") : secret "image-registry-tls" not found
Sep 05 07:49:01.459596 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.459545    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cafd9630-0d63-4c63-9829-bb8f215e97ac-nginx-conf\") pod \"networking-console-plugin-79977f9844-g27lz\" (UID: \"cafd9630-0d63-4c63-9829-bb8f215e97ac\") " pod="openshift-network-console/networking-console-plugin-79977f9844-g27lz"
Sep 05 07:49:01.460157 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.460135    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/27817b6a-02ee-4fde-856f-221f422e231f-trusted-ca\") pod \"image-registry-79bddd854d-9vlsf\" (UID: \"27817b6a-02ee-4fde-856f-221f422e231f\") " pod="openshift-image-registry/image-registry-79bddd854d-9vlsf"
Sep 05 07:49:01.461575 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.461555    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/27817b6a-02ee-4fde-856f-221f422e231f-image-registry-private-configuration\") pod \"image-registry-79bddd854d-9vlsf\" (UID: \"27817b6a-02ee-4fde-856f-221f422e231f\") " pod="openshift-image-registry/image-registry-79bddd854d-9vlsf"
Sep 05 07:49:01.461656 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.461599    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/27817b6a-02ee-4fde-856f-221f422e231f-installation-pull-secrets\") pod \"image-registry-79bddd854d-9vlsf\" (UID: \"27817b6a-02ee-4fde-856f-221f422e231f\") " pod="openshift-image-registry/image-registry-79bddd854d-9vlsf"
Sep 05 07:49:01.470184 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.470163    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tb54t\" (UniqueName: \"kubernetes.io/projected/27817b6a-02ee-4fde-856f-221f422e231f-kube-api-access-tb54t\") pod \"image-registry-79bddd854d-9vlsf\" (UID: \"27817b6a-02ee-4fde-856f-221f422e231f\") " pod="openshift-image-registry/image-registry-79bddd854d-9vlsf"
Sep 05 07:49:01.470359 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.470344    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/27817b6a-02ee-4fde-856f-221f422e231f-bound-sa-token\") pod \"image-registry-79bddd854d-9vlsf\" (UID: \"27817b6a-02ee-4fde-856f-221f422e231f\") " pod="openshift-image-registry/image-registry-79bddd854d-9vlsf"
Sep 05 07:49:01.473165 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.473148    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/27817b6a-02ee-4fde-856f-221f422e231f-registry-certificates\") pod \"image-registry-79bddd854d-9vlsf\" (UID: \"27817b6a-02ee-4fde-856f-221f422e231f\") " pod="openshift-image-registry/image-registry-79bddd854d-9vlsf"
Sep 05 07:49:01.511101 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.511077    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-vtq4p"]
Sep 05 07:49:01.514258 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:49:01.514229    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9da5c2fc_7ecf_41c4_b814_ae36d6909e16.slice/crio-dfc0114dccca5f707fac72e821da3d6e0c680c8443fae6afc5ee7c7a496ae7c8 WatchSource:0}: Error finding container dfc0114dccca5f707fac72e821da3d6e0c680c8443fae6afc5ee7c7a496ae7c8: Status 404 returned error can't find the container with id dfc0114dccca5f707fac72e821da3d6e0c680c8443fae6afc5ee7c7a496ae7c8
Sep 05 07:49:01.515301 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.515278    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:01.550759 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.550731    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-dcbfv"]
Sep 05 07:49:01.555272 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:49:01.555244    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod999f5178_4c4b_4888_a348_c0d434b38967.slice/crio-20c31505e069f2727c28bae8045ba977ccd3360c01420fe3282058718282e363 WatchSource:0}: Error finding container 20c31505e069f2727c28bae8045ba977ccd3360c01420fe3282058718282e363: Status 404 returned error can't find the container with id 20c31505e069f2727c28bae8045ba977ccd3360c01420fe3282058718282e363
Sep 05 07:49:01.620095 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.620034    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-rzj45"]
Sep 05 07:49:01.624419 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:49:01.624388    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod143aee12_5c44_440c_ac06_1a5146e2d401.slice/crio-d0b8621dbb1e58c4778f29cbbd951e5c5b8b58a5f2a5db2210a0bea033288b7d WatchSource:0}: Error finding container d0b8621dbb1e58c4778f29cbbd951e5c5b8b58a5f2a5db2210a0bea033288b7d: Status 404 returned error can't find the container with id d0b8621dbb1e58c4778f29cbbd951e5c5b8b58a5f2a5db2210a0bea033288b7d
Sep 05 07:49:01.762328 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.762252    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9b6376b1-fe4f-4e8f-b2aa-18b7d640d685-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-hxgpj\" (UID: \"9b6376b1-fe4f-4e8f-b2aa-18b7d640d685\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-hxgpj"
Sep 05 07:49:01.762328 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.762295    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9163aed9-c681-4b4d-a7f9-63438c02656f-service-ca-bundle\") pod \"router-default-8557f77dd-lw4qb\" (UID: \"9163aed9-c681-4b4d-a7f9-63438c02656f\") " pod="openshift-ingress/router-default-8557f77dd-lw4qb"
Sep 05 07:49:01.762328 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.762317    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9163aed9-c681-4b4d-a7f9-63438c02656f-metrics-certs\") pod \"router-default-8557f77dd-lw4qb\" (UID: \"9163aed9-c681-4b4d-a7f9-63438c02656f\") " pod="openshift-ingress/router-default-8557f77dd-lw4qb"
Sep 05 07:49:01.762554 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:01.762399    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 05 07:49:01.762554 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:01.762424    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9163aed9-c681-4b4d-a7f9-63438c02656f-service-ca-bundle podName:9163aed9-c681-4b4d-a7f9-63438c02656f nodeName:}" failed. No retries permitted until 2026-09-05 07:49:02.762408283 +0000 UTC m=+45.532550597 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9163aed9-c681-4b4d-a7f9-63438c02656f-service-ca-bundle") pod "router-default-8557f77dd-lw4qb" (UID: "9163aed9-c681-4b4d-a7f9-63438c02656f") : configmap references non-existent config key: service-ca.crt
Sep 05 07:49:01.762554 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:01.762443    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 05 07:49:01.762554 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:01.762451    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b6376b1-fe4f-4e8f-b2aa-18b7d640d685-samples-operator-tls podName:9b6376b1-fe4f-4e8f-b2aa-18b7d640d685 nodeName:}" failed. No retries permitted until 2026-09-05 07:49:02.762439303 +0000 UTC m=+45.532581621 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9b6376b1-fe4f-4e8f-b2aa-18b7d640d685-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-hxgpj" (UID: "9b6376b1-fe4f-4e8f-b2aa-18b7d640d685") : secret "samples-operator-tls" not found
Sep 05 07:49:01.762554 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:01.762474    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9163aed9-c681-4b4d-a7f9-63438c02656f-metrics-certs podName:9163aed9-c681-4b4d-a7f9-63438c02656f nodeName:}" failed. No retries permitted until 2026-09-05 07:49:02.762465543 +0000 UTC m=+45.532607878 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9163aed9-c681-4b4d-a7f9-63438c02656f-metrics-certs") pod "router-default-8557f77dd-lw4qb" (UID: "9163aed9-c681-4b4d-a7f9-63438c02656f") : secret "router-metrics-certs-default" not found
Sep 05 07:49:01.801149 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.801119    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:01.897249 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.897217    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:01.963333 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.963300    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/27817b6a-02ee-4fde-856f-221f422e231f-registry-tls\") pod \"image-registry-79bddd854d-9vlsf\" (UID: \"27817b6a-02ee-4fde-856f-221f422e231f\") " pod="openshift-image-registry/image-registry-79bddd854d-9vlsf"
Sep 05 07:49:01.963473 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:01.963400    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cafd9630-0d63-4c63-9829-bb8f215e97ac-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-g27lz\" (UID: \"cafd9630-0d63-4c63-9829-bb8f215e97ac\") " pod="openshift-network-console/networking-console-plugin-79977f9844-g27lz"
Sep 05 07:49:01.963473 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:01.963448    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 05 07:49:01.963473 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:01.963469    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79bddd854d-9vlsf: secret "image-registry-tls" not found
Sep 05 07:49:01.963572 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:01.963502    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 05 07:49:01.963572 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:01.963528    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27817b6a-02ee-4fde-856f-221f422e231f-registry-tls podName:27817b6a-02ee-4fde-856f-221f422e231f nodeName:}" failed. No retries permitted until 2026-09-05 07:49:02.963510933 +0000 UTC m=+45.733653271 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/27817b6a-02ee-4fde-856f-221f422e231f-registry-tls") pod "image-registry-79bddd854d-9vlsf" (UID: "27817b6a-02ee-4fde-856f-221f422e231f") : secret "image-registry-tls" not found
Sep 05 07:49:01.963572 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:01.963545    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cafd9630-0d63-4c63-9829-bb8f215e97ac-networking-console-plugin-cert podName:cafd9630-0d63-4c63-9829-bb8f215e97ac nodeName:}" failed. No retries permitted until 2026-09-05 07:49:02.963534641 +0000 UTC m=+45.733676955 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cafd9630-0d63-4c63-9829-bb8f215e97ac-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-g27lz" (UID: "cafd9630-0d63-4c63-9829-bb8f215e97ac") : secret "networking-console-plugin-cert" not found
Sep 05 07:49:02.011981 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:02.011945    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-vtq4p" event={"ID":"9da5c2fc-7ecf-41c4-b814-ae36d6909e16","Type":"ContainerStarted","Data":"dfc0114dccca5f707fac72e821da3d6e0c680c8443fae6afc5ee7c7a496ae7c8"}
Sep 05 07:49:02.013410 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:02.013343    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-rzj45" event={"ID":"143aee12-5c44-440c-ac06-1a5146e2d401","Type":"ContainerStarted","Data":"d0b8621dbb1e58c4778f29cbbd951e5c5b8b58a5f2a5db2210a0bea033288b7d"}
Sep 05 07:49:02.016462 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:02.016434    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-dcbfv" event={"ID":"999f5178-4c4b-4888-a348-c0d434b38967","Type":"ContainerStarted","Data":"20c31505e069f2727c28bae8045ba977ccd3360c01420fe3282058718282e363"}
Sep 05 07:49:02.244534 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:02.244489    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:02.324167 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:02.324094    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:02.658209 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:02.658062    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:02.758958 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:02.758926    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:02.770110 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:02.770074    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9163aed9-c681-4b4d-a7f9-63438c02656f-metrics-certs\") pod \"router-default-8557f77dd-lw4qb\" (UID: \"9163aed9-c681-4b4d-a7f9-63438c02656f\") " pod="openshift-ingress/router-default-8557f77dd-lw4qb"
Sep 05 07:49:02.770283 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:02.770236    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 05 07:49:02.770334 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:02.770295    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9163aed9-c681-4b4d-a7f9-63438c02656f-metrics-certs podName:9163aed9-c681-4b4d-a7f9-63438c02656f nodeName:}" failed. No retries permitted until 2026-09-05 07:49:04.770278509 +0000 UTC m=+47.540420838 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9163aed9-c681-4b4d-a7f9-63438c02656f-metrics-certs") pod "router-default-8557f77dd-lw4qb" (UID: "9163aed9-c681-4b4d-a7f9-63438c02656f") : secret "router-metrics-certs-default" not found
Sep 05 07:49:02.770334 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:02.770302    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 05 07:49:02.770407 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:02.770343    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b6376b1-fe4f-4e8f-b2aa-18b7d640d685-samples-operator-tls podName:9b6376b1-fe4f-4e8f-b2aa-18b7d640d685 nodeName:}" failed. No retries permitted until 2026-09-05 07:49:04.770331181 +0000 UTC m=+47.540473522 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9b6376b1-fe4f-4e8f-b2aa-18b7d640d685-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-hxgpj" (UID: "9b6376b1-fe4f-4e8f-b2aa-18b7d640d685") : secret "samples-operator-tls" not found
Sep 05 07:49:02.770407 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:02.770237    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9b6376b1-fe4f-4e8f-b2aa-18b7d640d685-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-hxgpj\" (UID: \"9b6376b1-fe4f-4e8f-b2aa-18b7d640d685\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-hxgpj"
Sep 05 07:49:02.770407 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:02.770380    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9163aed9-c681-4b4d-a7f9-63438c02656f-service-ca-bundle\") pod \"router-default-8557f77dd-lw4qb\" (UID: \"9163aed9-c681-4b4d-a7f9-63438c02656f\") " pod="openshift-ingress/router-default-8557f77dd-lw4qb"
Sep 05 07:49:02.770578 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:02.770457    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9163aed9-c681-4b4d-a7f9-63438c02656f-service-ca-bundle podName:9163aed9-c681-4b4d-a7f9-63438c02656f nodeName:}" failed. No retries permitted until 2026-09-05 07:49:04.770448921 +0000 UTC m=+47.540591236 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9163aed9-c681-4b4d-a7f9-63438c02656f-service-ca-bundle") pod "router-default-8557f77dd-lw4qb" (UID: "9163aed9-c681-4b4d-a7f9-63438c02656f") : configmap references non-existent config key: service-ca.crt
Sep 05 07:49:02.972457 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:02.972385    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cafd9630-0d63-4c63-9829-bb8f215e97ac-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-g27lz\" (UID: \"cafd9630-0d63-4c63-9829-bb8f215e97ac\") " pod="openshift-network-console/networking-console-plugin-79977f9844-g27lz"
Sep 05 07:49:02.972457 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:02.972446    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/27817b6a-02ee-4fde-856f-221f422e231f-registry-tls\") pod \"image-registry-79bddd854d-9vlsf\" (UID: \"27817b6a-02ee-4fde-856f-221f422e231f\") " pod="openshift-image-registry/image-registry-79bddd854d-9vlsf"
Sep 05 07:49:02.972627 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:02.972540    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 05 07:49:02.972627 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:02.972609    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 05 07:49:02.972627 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:02.972623    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79bddd854d-9vlsf: secret "image-registry-tls" not found
Sep 05 07:49:02.972714 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:02.972612    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cafd9630-0d63-4c63-9829-bb8f215e97ac-networking-console-plugin-cert podName:cafd9630-0d63-4c63-9829-bb8f215e97ac nodeName:}" failed. No retries permitted until 2026-09-05 07:49:04.97259668 +0000 UTC m=+47.742738997 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cafd9630-0d63-4c63-9829-bb8f215e97ac-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-g27lz" (UID: "cafd9630-0d63-4c63-9829-bb8f215e97ac") : secret "networking-console-plugin-cert" not found
Sep 05 07:49:02.972714 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:02.972674    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27817b6a-02ee-4fde-856f-221f422e231f-registry-tls podName:27817b6a-02ee-4fde-856f-221f422e231f nodeName:}" failed. No retries permitted until 2026-09-05 07:49:04.97265826 +0000 UTC m=+47.742800576 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/27817b6a-02ee-4fde-856f-221f422e231f-registry-tls") pod "image-registry-79bddd854d-9vlsf" (UID: "27817b6a-02ee-4fde-856f-221f422e231f") : secret "image-registry-tls" not found
Sep 05 07:49:03.447329 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:03.447293    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:03.847624 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:03.847585    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:03.906907 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:03.906876    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:04.790474 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:04.790448    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9b6376b1-fe4f-4e8f-b2aa-18b7d640d685-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-hxgpj\" (UID: \"9b6376b1-fe4f-4e8f-b2aa-18b7d640d685\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-hxgpj"
Sep 05 07:49:04.790581 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:04.790500    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9163aed9-c681-4b4d-a7f9-63438c02656f-service-ca-bundle\") pod \"router-default-8557f77dd-lw4qb\" (UID: \"9163aed9-c681-4b4d-a7f9-63438c02656f\") " pod="openshift-ingress/router-default-8557f77dd-lw4qb"
Sep 05 07:49:04.790581 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:04.790535    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9163aed9-c681-4b4d-a7f9-63438c02656f-metrics-certs\") pod \"router-default-8557f77dd-lw4qb\" (UID: \"9163aed9-c681-4b4d-a7f9-63438c02656f\") " pod="openshift-ingress/router-default-8557f77dd-lw4qb"
Sep 05 07:49:04.790688 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:04.790657    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 05 07:49:04.790740 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:04.790730    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9163aed9-c681-4b4d-a7f9-63438c02656f-service-ca-bundle podName:9163aed9-c681-4b4d-a7f9-63438c02656f nodeName:}" failed. No retries permitted until 2026-09-05 07:49:08.790713921 +0000 UTC m=+51.560856249 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9163aed9-c681-4b4d-a7f9-63438c02656f-service-ca-bundle") pod "router-default-8557f77dd-lw4qb" (UID: "9163aed9-c681-4b4d-a7f9-63438c02656f") : configmap references non-existent config key: service-ca.crt
Sep 05 07:49:04.790801 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:04.790749    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b6376b1-fe4f-4e8f-b2aa-18b7d640d685-samples-operator-tls podName:9b6376b1-fe4f-4e8f-b2aa-18b7d640d685 nodeName:}" failed. No retries permitted until 2026-09-05 07:49:08.790740287 +0000 UTC m=+51.560882601 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9b6376b1-fe4f-4e8f-b2aa-18b7d640d685-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-hxgpj" (UID: "9b6376b1-fe4f-4e8f-b2aa-18b7d640d685") : secret "samples-operator-tls" not found
Sep 05 07:49:04.790801 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:04.790657    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 05 07:49:04.790801 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:04.790780    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9163aed9-c681-4b4d-a7f9-63438c02656f-metrics-certs podName:9163aed9-c681-4b4d-a7f9-63438c02656f nodeName:}" failed. No retries permitted until 2026-09-05 07:49:08.790772334 +0000 UTC m=+51.560914663 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9163aed9-c681-4b4d-a7f9-63438c02656f-metrics-certs") pod "router-default-8557f77dd-lw4qb" (UID: "9163aed9-c681-4b4d-a7f9-63438c02656f") : secret "router-metrics-certs-default" not found
Sep 05 07:49:04.992973 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:04.992889    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cafd9630-0d63-4c63-9829-bb8f215e97ac-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-g27lz\" (UID: \"cafd9630-0d63-4c63-9829-bb8f215e97ac\") " pod="openshift-network-console/networking-console-plugin-79977f9844-g27lz"
Sep 05 07:49:04.993312 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:04.992981    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/27817b6a-02ee-4fde-856f-221f422e231f-registry-tls\") pod \"image-registry-79bddd854d-9vlsf\" (UID: \"27817b6a-02ee-4fde-856f-221f422e231f\") " pod="openshift-image-registry/image-registry-79bddd854d-9vlsf"
Sep 05 07:49:04.993312 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:04.993019    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 05 07:49:04.993312 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:04.993083    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cafd9630-0d63-4c63-9829-bb8f215e97ac-networking-console-plugin-cert podName:cafd9630-0d63-4c63-9829-bb8f215e97ac nodeName:}" failed. No retries permitted until 2026-09-05 07:49:08.993066509 +0000 UTC m=+51.763208841 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cafd9630-0d63-4c63-9829-bb8f215e97ac-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-g27lz" (UID: "cafd9630-0d63-4c63-9829-bb8f215e97ac") : secret "networking-console-plugin-cert" not found
Sep 05 07:49:04.993312 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:04.993141    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 05 07:49:04.993312 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:04.993160    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79bddd854d-9vlsf: secret "image-registry-tls" not found
Sep 05 07:49:04.993312 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:04.993214    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27817b6a-02ee-4fde-856f-221f422e231f-registry-tls podName:27817b6a-02ee-4fde-856f-221f422e231f nodeName:}" failed. No retries permitted until 2026-09-05 07:49:08.993198394 +0000 UTC m=+51.763340712 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/27817b6a-02ee-4fde-856f-221f422e231f-registry-tls") pod "image-registry-79bddd854d-9vlsf" (UID: "27817b6a-02ee-4fde-856f-221f422e231f") : secret "image-registry-tls" not found
Sep 05 07:49:05.025618 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:05.025584    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-rzj45" event={"ID":"143aee12-5c44-440c-ac06-1a5146e2d401","Type":"ContainerStarted","Data":"f6946f6074b8e080daae5deaee1088c62c29b1c2a996904c77f2661801c809ca"}
Sep 05 07:49:05.027560 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:05.027256    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-dcbfv" event={"ID":"999f5178-4c4b-4888-a348-c0d434b38967","Type":"ContainerStarted","Data":"bc822850751b2f130084d7cb31a3f7a57007a0ffe2e36e10f0753d96c98bdd56"}
Sep 05 07:49:05.028835 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:05.028804    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-vtq4p" event={"ID":"9da5c2fc-7ecf-41c4-b814-ae36d6909e16","Type":"ContainerStarted","Data":"a66c9b68890c5999023401a0d62f1ccd4f705685fc7f80728ea5be0cbac5a084"}
Sep 05 07:49:05.040204 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:05.040151    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-rzj45" podStartSLOduration=1.964781262 podStartE2EDuration="5.040135128s" totalImagesPullingTime="3.075353866s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:49:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:05.039344579 +0000 UTC m=+47.809486929" watchObservedRunningTime="2026-09-05 07:49:05.040135128 +0000 UTC m=+47.810277462"
Sep 05 07:49:05.054038 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:05.053988    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-vtq4p" podStartSLOduration=1.8743303340000002 podStartE2EDuration="5.053970039s" totalImagesPullingTime="3.179639705s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:49:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:05.052500068 +0000 UTC m=+47.822642402" watchObservedRunningTime="2026-09-05 07:49:05.053970039 +0000 UTC m=+47.824112377"
Sep 05 07:49:05.070230 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:05.070181    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-dcbfv" podStartSLOduration=1.931859067 podStartE2EDuration="5.070161248s" totalImagesPullingTime="3.138302181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:49:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:05.069266746 +0000 UTC m=+47.839409106" watchObservedRunningTime="2026-09-05 07:49:05.070161248 +0000 UTC m=+47.840303585"
Sep 05 07:49:05.142085 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:05.142053    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:05.239400 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:05.239371    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:05.460619 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:05.460589    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:05.516467 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:05.516404    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:06.369305 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:06.369275    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:06.463437 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:06.463404    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:06.564104 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:06.564068    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:06.998777 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:06.998748    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mtq5b"]
Sep 05 07:49:07.007799 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:07.007779    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mtq5b"
Sep 05 07:49:07.010077 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:07.010050    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 05 07:49:07.010187 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:07.010071    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 05 07:49:07.010187 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:07.010090    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gx2xl\""
Sep 05 07:49:07.016507 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:07.016484    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mtq5b"]
Sep 05 07:49:07.020253 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:07.020232    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:07.093018 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:07.092987    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:07.111201 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:07.111177    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3387beaa-e6d7-4f0a-bc56-0c06ef410d80-data-volume\") pod \"insights-runtime-extractor-mtq5b\" (UID: \"3387beaa-e6d7-4f0a-bc56-0c06ef410d80\") " pod="openshift-insights/insights-runtime-extractor-mtq5b"
Sep 05 07:49:07.111330 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:07.111206    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3387beaa-e6d7-4f0a-bc56-0c06ef410d80-crio-socket\") pod \"insights-runtime-extractor-mtq5b\" (UID: \"3387beaa-e6d7-4f0a-bc56-0c06ef410d80\") " pod="openshift-insights/insights-runtime-extractor-mtq5b"
Sep 05 07:49:07.111330 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:07.111235    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3387beaa-e6d7-4f0a-bc56-0c06ef410d80-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mtq5b\" (UID: \"3387beaa-e6d7-4f0a-bc56-0c06ef410d80\") " pod="openshift-insights/insights-runtime-extractor-mtq5b"
Sep 05 07:49:07.111330 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:07.111263    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3387beaa-e6d7-4f0a-bc56-0c06ef410d80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mtq5b\" (UID: \"3387beaa-e6d7-4f0a-bc56-0c06ef410d80\") " pod="openshift-insights/insights-runtime-extractor-mtq5b"
Sep 05 07:49:07.111330 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:07.111315    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crzrd\" (UniqueName: \"kubernetes.io/projected/3387beaa-e6d7-4f0a-bc56-0c06ef410d80-kube-api-access-crzrd\") pod \"insights-runtime-extractor-mtq5b\" (UID: \"3387beaa-e6d7-4f0a-bc56-0c06ef410d80\") " pod="openshift-insights/insights-runtime-extractor-mtq5b"
Sep 05 07:49:07.211055 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:07.211025    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:07.212143 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:07.212111    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3387beaa-e6d7-4f0a-bc56-0c06ef410d80-data-volume\") pod \"insights-runtime-extractor-mtq5b\" (UID: \"3387beaa-e6d7-4f0a-bc56-0c06ef410d80\") " pod="openshift-insights/insights-runtime-extractor-mtq5b"
Sep 05 07:49:07.212260 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:07.212155    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3387beaa-e6d7-4f0a-bc56-0c06ef410d80-crio-socket\") pod \"insights-runtime-extractor-mtq5b\" (UID: \"3387beaa-e6d7-4f0a-bc56-0c06ef410d80\") " pod="openshift-insights/insights-runtime-extractor-mtq5b"
Sep 05 07:49:07.212260 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:07.212211    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3387beaa-e6d7-4f0a-bc56-0c06ef410d80-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mtq5b\" (UID: \"3387beaa-e6d7-4f0a-bc56-0c06ef410d80\") " pod="openshift-insights/insights-runtime-extractor-mtq5b"
Sep 05 07:49:07.212260 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:07.212242    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3387beaa-e6d7-4f0a-bc56-0c06ef410d80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mtq5b\" (UID: \"3387beaa-e6d7-4f0a-bc56-0c06ef410d80\") " pod="openshift-insights/insights-runtime-extractor-mtq5b"
Sep 05 07:49:07.212415 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:07.212267    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crzrd\" (UniqueName: \"kubernetes.io/projected/3387beaa-e6d7-4f0a-bc56-0c06ef410d80-kube-api-access-crzrd\") pod \"insights-runtime-extractor-mtq5b\" (UID: \"3387beaa-e6d7-4f0a-bc56-0c06ef410d80\") " pod="openshift-insights/insights-runtime-extractor-mtq5b"
Sep 05 07:49:07.212415 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:07.212302    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 05 07:49:07.212415 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:07.212356    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 05 07:49:07.212415 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:07.212372    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3387beaa-e6d7-4f0a-bc56-0c06ef410d80-kube-rbac-proxy-cm podName:3387beaa-e6d7-4f0a-bc56-0c06ef410d80 nodeName:}" failed. No retries permitted until 2026-09-05 07:49:07.712353018 +0000 UTC m=+50.482495346 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3387beaa-e6d7-4f0a-bc56-0c06ef410d80-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mtq5b" (UID: "3387beaa-e6d7-4f0a-bc56-0c06ef410d80") : configmap "kube-rbac-proxy" not found
Sep 05 07:49:07.212615 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:07.212425    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3387beaa-e6d7-4f0a-bc56-0c06ef410d80-insights-runtime-extractor-tls podName:3387beaa-e6d7-4f0a-bc56-0c06ef410d80 nodeName:}" failed. No retries permitted until 2026-09-05 07:49:07.712409251 +0000 UTC m=+50.482551584 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3387beaa-e6d7-4f0a-bc56-0c06ef410d80-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mtq5b" (UID: "3387beaa-e6d7-4f0a-bc56-0c06ef410d80") : secret "insights-runtime-extractor-tls" not found
Sep 05 07:49:07.212615 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:07.212486    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3387beaa-e6d7-4f0a-bc56-0c06ef410d80-data-volume\") pod \"insights-runtime-extractor-mtq5b\" (UID: \"3387beaa-e6d7-4f0a-bc56-0c06ef410d80\") " pod="openshift-insights/insights-runtime-extractor-mtq5b"
Sep 05 07:49:07.212615 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:07.212508    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3387beaa-e6d7-4f0a-bc56-0c06ef410d80-crio-socket\") pod \"insights-runtime-extractor-mtq5b\" (UID: \"3387beaa-e6d7-4f0a-bc56-0c06ef410d80\") " pod="openshift-insights/insights-runtime-extractor-mtq5b"
Sep 05 07:49:07.221670 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:07.221649    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crzrd\" (UniqueName: \"kubernetes.io/projected/3387beaa-e6d7-4f0a-bc56-0c06ef410d80-kube-api-access-crzrd\") pod \"insights-runtime-extractor-mtq5b\" (UID: \"3387beaa-e6d7-4f0a-bc56-0c06ef410d80\") " pod="openshift-insights/insights-runtime-extractor-mtq5b"
Sep 05 07:49:07.294216 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:07.294118    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:07.669138 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:07.669074    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:07.718137 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:07.718112    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3387beaa-e6d7-4f0a-bc56-0c06ef410d80-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mtq5b\" (UID: \"3387beaa-e6d7-4f0a-bc56-0c06ef410d80\") " pod="openshift-insights/insights-runtime-extractor-mtq5b"
Sep 05 07:49:07.718281 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:07.718145    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3387beaa-e6d7-4f0a-bc56-0c06ef410d80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mtq5b\" (UID: \"3387beaa-e6d7-4f0a-bc56-0c06ef410d80\") " pod="openshift-insights/insights-runtime-extractor-mtq5b"
Sep 05 07:49:07.718281 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:07.718243    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 05 07:49:07.718281 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:07.718241    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 05 07:49:07.718386 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:07.718298    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3387beaa-e6d7-4f0a-bc56-0c06ef410d80-insights-runtime-extractor-tls podName:3387beaa-e6d7-4f0a-bc56-0c06ef410d80 nodeName:}" failed. No retries permitted until 2026-09-05 07:49:08.718285342 +0000 UTC m=+51.488427658 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3387beaa-e6d7-4f0a-bc56-0c06ef410d80-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mtq5b" (UID: "3387beaa-e6d7-4f0a-bc56-0c06ef410d80") : secret "insights-runtime-extractor-tls" not found
Sep 05 07:49:07.718386 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:07.718347    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3387beaa-e6d7-4f0a-bc56-0c06ef410d80-kube-rbac-proxy-cm podName:3387beaa-e6d7-4f0a-bc56-0c06ef410d80 nodeName:}" failed. No retries permitted until 2026-09-05 07:49:08.718340437 +0000 UTC m=+51.488482752 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3387beaa-e6d7-4f0a-bc56-0c06ef410d80-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mtq5b" (UID: "3387beaa-e6d7-4f0a-bc56-0c06ef410d80") : configmap "kube-rbac-proxy" not found
Sep 05 07:49:07.736284 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:07.736254    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:07.931661 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:07.931586    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:08.031617 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:08.031586    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:08.424457 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:08.424423    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2de9d5a3-384d-4f3e-866f-19f1b5d4126d-metrics-tls\") pod \"dns-default-9wmv2\" (UID: \"2de9d5a3-384d-4f3e-866f-19f1b5d4126d\") " pod="openshift-dns/dns-default-9wmv2"
Sep 05 07:49:08.424622 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:08.424577    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 05 07:49:08.424663 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:08.424639    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2de9d5a3-384d-4f3e-866f-19f1b5d4126d-metrics-tls podName:2de9d5a3-384d-4f3e-866f-19f1b5d4126d nodeName:}" failed. No retries permitted until 2026-09-05 07:49:24.424623319 +0000 UTC m=+67.194765634 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2de9d5a3-384d-4f3e-866f-19f1b5d4126d-metrics-tls") pod "dns-default-9wmv2" (UID: "2de9d5a3-384d-4f3e-866f-19f1b5d4126d") : secret "dns-default-metrics-tls" not found
Sep 05 07:49:08.617151 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:08.617084    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:08.727323 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:08.727250    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3387beaa-e6d7-4f0a-bc56-0c06ef410d80-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mtq5b\" (UID: \"3387beaa-e6d7-4f0a-bc56-0c06ef410d80\") " pod="openshift-insights/insights-runtime-extractor-mtq5b"
Sep 05 07:49:08.727323 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:08.727285    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3387beaa-e6d7-4f0a-bc56-0c06ef410d80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mtq5b\" (UID: \"3387beaa-e6d7-4f0a-bc56-0c06ef410d80\") " pod="openshift-insights/insights-runtime-extractor-mtq5b"
Sep 05 07:49:08.727688 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:08.727385    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 05 07:49:08.727688 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:08.727447    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3387beaa-e6d7-4f0a-bc56-0c06ef410d80-insights-runtime-extractor-tls podName:3387beaa-e6d7-4f0a-bc56-0c06ef410d80 nodeName:}" failed. No retries permitted until 2026-09-05 07:49:10.727431968 +0000 UTC m=+53.497574287 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3387beaa-e6d7-4f0a-bc56-0c06ef410d80-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mtq5b" (UID: "3387beaa-e6d7-4f0a-bc56-0c06ef410d80") : secret "insights-runtime-extractor-tls" not found
Sep 05 07:49:08.727773 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:08.727754    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3387beaa-e6d7-4f0a-bc56-0c06ef410d80-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mtq5b\" (UID: \"3387beaa-e6d7-4f0a-bc56-0c06ef410d80\") " pod="openshift-insights/insights-runtime-extractor-mtq5b"
Sep 05 07:49:08.828620 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:08.828589    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9163aed9-c681-4b4d-a7f9-63438c02656f-metrics-certs\") pod \"router-default-8557f77dd-lw4qb\" (UID: \"9163aed9-c681-4b4d-a7f9-63438c02656f\") " pod="openshift-ingress/router-default-8557f77dd-lw4qb"
Sep 05 07:49:08.828745 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:08.828700    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9b6376b1-fe4f-4e8f-b2aa-18b7d640d685-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-hxgpj\" (UID: \"9b6376b1-fe4f-4e8f-b2aa-18b7d640d685\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-hxgpj"
Sep 05 07:49:08.828745 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:08.828726    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9163aed9-c681-4b4d-a7f9-63438c02656f-service-ca-bundle\") pod \"router-default-8557f77dd-lw4qb\" (UID: \"9163aed9-c681-4b4d-a7f9-63438c02656f\") " pod="openshift-ingress/router-default-8557f77dd-lw4qb"
Sep 05 07:49:08.828827 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:08.828735    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 05 07:49:08.828827 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:08.828820    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9163aed9-c681-4b4d-a7f9-63438c02656f-metrics-certs podName:9163aed9-c681-4b4d-a7f9-63438c02656f nodeName:}" failed. No retries permitted until 2026-09-05 07:49:16.828800173 +0000 UTC m=+59.598942490 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9163aed9-c681-4b4d-a7f9-63438c02656f-metrics-certs") pod "router-default-8557f77dd-lw4qb" (UID: "9163aed9-c681-4b4d-a7f9-63438c02656f") : secret "router-metrics-certs-default" not found
Sep 05 07:49:08.828920 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:08.828832    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 05 07:49:08.828920 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:08.828838    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9163aed9-c681-4b4d-a7f9-63438c02656f-service-ca-bundle podName:9163aed9-c681-4b4d-a7f9-63438c02656f nodeName:}" failed. No retries permitted until 2026-09-05 07:49:16.828828745 +0000 UTC m=+59.598971066 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9163aed9-c681-4b4d-a7f9-63438c02656f-service-ca-bundle") pod "router-default-8557f77dd-lw4qb" (UID: "9163aed9-c681-4b4d-a7f9-63438c02656f") : configmap references non-existent config key: service-ca.crt
Sep 05 07:49:08.828920 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:08.828893    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b6376b1-fe4f-4e8f-b2aa-18b7d640d685-samples-operator-tls podName:9b6376b1-fe4f-4e8f-b2aa-18b7d640d685 nodeName:}" failed. No retries permitted until 2026-09-05 07:49:16.828882688 +0000 UTC m=+59.599025032 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9b6376b1-fe4f-4e8f-b2aa-18b7d640d685-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-hxgpj" (UID: "9b6376b1-fe4f-4e8f-b2aa-18b7d640d685") : secret "samples-operator-tls" not found
Sep 05 07:49:09.030320 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:09.030235    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cafd9630-0d63-4c63-9829-bb8f215e97ac-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-g27lz\" (UID: \"cafd9630-0d63-4c63-9829-bb8f215e97ac\") " pod="openshift-network-console/networking-console-plugin-79977f9844-g27lz"
Sep 05 07:49:09.030320 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:09.030314    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/27817b6a-02ee-4fde-856f-221f422e231f-registry-tls\") pod \"image-registry-79bddd854d-9vlsf\" (UID: \"27817b6a-02ee-4fde-856f-221f422e231f\") " pod="openshift-image-registry/image-registry-79bddd854d-9vlsf"
Sep 05 07:49:09.030520 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:09.030370    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 05 07:49:09.030520 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:09.030432    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cafd9630-0d63-4c63-9829-bb8f215e97ac-networking-console-plugin-cert podName:cafd9630-0d63-4c63-9829-bb8f215e97ac nodeName:}" failed. No retries permitted until 2026-09-05 07:49:17.030417727 +0000 UTC m=+59.800560041 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cafd9630-0d63-4c63-9829-bb8f215e97ac-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-g27lz" (UID: "cafd9630-0d63-4c63-9829-bb8f215e97ac") : secret "networking-console-plugin-cert" not found
Sep 05 07:49:09.030520 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:09.030445    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 05 07:49:09.030520 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:09.030460    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79bddd854d-9vlsf: secret "image-registry-tls" not found
Sep 05 07:49:09.030520 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:09.030499    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27817b6a-02ee-4fde-856f-221f422e231f-registry-tls podName:27817b6a-02ee-4fde-856f-221f422e231f nodeName:}" failed. No retries permitted until 2026-09-05 07:49:17.030487123 +0000 UTC m=+59.800629438 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/27817b6a-02ee-4fde-856f-221f422e231f-registry-tls") pod "image-registry-79bddd854d-9vlsf" (UID: "27817b6a-02ee-4fde-856f-221f422e231f") : secret "image-registry-tls" not found
Sep 05 07:49:09.341620 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:09.341586    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:09.365806 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:09.365778    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fstjt_f0fd7e0b-f07e-43a2-a169-667bc753a36b/dns-node-resolver/0.log"
Sep 05 07:49:09.442217 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:09.442178    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:09.577132 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:09.577101    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:09.678092 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:09.678001    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:10.368820 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:10.368791    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-m7775_35c73ae3-8ad8-443c-ac44-c9cd3396e526/node-ca/0.log"
Sep 05 07:49:10.475613 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:10.475587    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:10.744776 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:10.744687    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3387beaa-e6d7-4f0a-bc56-0c06ef410d80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mtq5b\" (UID: \"3387beaa-e6d7-4f0a-bc56-0c06ef410d80\") " pod="openshift-insights/insights-runtime-extractor-mtq5b"
Sep 05 07:49:10.744967 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:10.744835    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 05 07:49:10.744967 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:10.744926    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3387beaa-e6d7-4f0a-bc56-0c06ef410d80-insights-runtime-extractor-tls podName:3387beaa-e6d7-4f0a-bc56-0c06ef410d80 nodeName:}" failed. No retries permitted until 2026-09-05 07:49:14.744908408 +0000 UTC m=+57.515050722 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3387beaa-e6d7-4f0a-bc56-0c06ef410d80-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mtq5b" (UID: "3387beaa-e6d7-4f0a-bc56-0c06ef410d80") : secret "insights-runtime-extractor-tls" not found
Sep 05 07:49:10.966169 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:10.966143    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-q6zgs_f2e61eb0-2689-4dfd-8b47-420342ee2916/serve-healthcheck-canary/0.log"
Sep 05 07:49:11.601376 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:11.601349    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:11.706305 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:11.706274    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:11.767237 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:11.767210    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-rzj45_143aee12-5c44-440c-ac06-1a5146e2d401/kube-storage-version-migrator-operator/0.log"
Sep 05 07:49:11.933462 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:11.933388    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:12.061315 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:12.061281    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:12.158316 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:12.158282    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:12.305264 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:12.305231    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:12.405374 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:12.405342    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:12.902395 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:12.902333    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:13.323011 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:13.322979    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:13.707130 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:13.707047    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:14.301763 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:14.301732    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:14.400674 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:14.400643    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:14.711758 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:14.711725    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:14.780348 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:14.780304    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3387beaa-e6d7-4f0a-bc56-0c06ef410d80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mtq5b\" (UID: \"3387beaa-e6d7-4f0a-bc56-0c06ef410d80\") " pod="openshift-insights/insights-runtime-extractor-mtq5b"
Sep 05 07:49:14.780515 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:14.780462    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 05 07:49:14.780555 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:14.780529    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3387beaa-e6d7-4f0a-bc56-0c06ef410d80-insights-runtime-extractor-tls podName:3387beaa-e6d7-4f0a-bc56-0c06ef410d80 nodeName:}" failed. No retries permitted until 2026-09-05 07:49:22.780510722 +0000 UTC m=+65.550653041 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3387beaa-e6d7-4f0a-bc56-0c06ef410d80-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mtq5b" (UID: "3387beaa-e6d7-4f0a-bc56-0c06ef410d80") : secret "insights-runtime-extractor-tls" not found
Sep 05 07:49:15.113435 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:15.113408    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:15.301575 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:15.301542    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:15.363888 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:15.363797    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:15.623831 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:15.623753    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:15.724724 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:15.724691    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:16.496057 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:16.496023    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:16.553307 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:16.553268    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:16.692608 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:16.692573    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:16.898054 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:16.898013    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9b6376b1-fe4f-4e8f-b2aa-18b7d640d685-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-hxgpj\" (UID: \"9b6376b1-fe4f-4e8f-b2aa-18b7d640d685\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-hxgpj"
Sep 05 07:49:16.898054 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:16.898060    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9163aed9-c681-4b4d-a7f9-63438c02656f-service-ca-bundle\") pod \"router-default-8557f77dd-lw4qb\" (UID: \"9163aed9-c681-4b4d-a7f9-63438c02656f\") " pod="openshift-ingress/router-default-8557f77dd-lw4qb"
Sep 05 07:49:16.898291 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:16.898089    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9163aed9-c681-4b4d-a7f9-63438c02656f-metrics-certs\") pod \"router-default-8557f77dd-lw4qb\" (UID: \"9163aed9-c681-4b4d-a7f9-63438c02656f\") " pod="openshift-ingress/router-default-8557f77dd-lw4qb"
Sep 05 07:49:16.898291 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:16.898210    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9163aed9-c681-4b4d-a7f9-63438c02656f-service-ca-bundle podName:9163aed9-c681-4b4d-a7f9-63438c02656f nodeName:}" failed. No retries permitted until 2026-09-05 07:49:32.898189769 +0000 UTC m=+75.668332105 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9163aed9-c681-4b4d-a7f9-63438c02656f-service-ca-bundle") pod "router-default-8557f77dd-lw4qb" (UID: "9163aed9-c681-4b4d-a7f9-63438c02656f") : configmap references non-existent config key: service-ca.crt
Sep 05 07:49:16.900331 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:16.900295    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:16.900828 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:16.900808    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9b6376b1-fe4f-4e8f-b2aa-18b7d640d685-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-hxgpj\" (UID: \"9b6376b1-fe4f-4e8f-b2aa-18b7d640d685\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-hxgpj"
Sep 05 07:49:16.900952 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:16.900936    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9163aed9-c681-4b4d-a7f9-63438c02656f-metrics-certs\") pod \"router-default-8557f77dd-lw4qb\" (UID: \"9163aed9-c681-4b4d-a7f9-63438c02656f\") " pod="openshift-ingress/router-default-8557f77dd-lw4qb"
Sep 05 07:49:16.904227 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:16.904215    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-hxgpj"
Sep 05 07:49:17.100409 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:17.100375    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cafd9630-0d63-4c63-9829-bb8f215e97ac-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-g27lz\" (UID: \"cafd9630-0d63-4c63-9829-bb8f215e97ac\") " pod="openshift-network-console/networking-console-plugin-79977f9844-g27lz"
Sep 05 07:49:17.100560 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:17.100427    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/27817b6a-02ee-4fde-856f-221f422e231f-registry-tls\") pod \"image-registry-79bddd854d-9vlsf\" (UID: \"27817b6a-02ee-4fde-856f-221f422e231f\") " pod="openshift-image-registry/image-registry-79bddd854d-9vlsf"
Sep 05 07:49:17.100560 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:17.100517    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 05 07:49:17.100633 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:17.100580    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cafd9630-0d63-4c63-9829-bb8f215e97ac-networking-console-plugin-cert podName:cafd9630-0d63-4c63-9829-bb8f215e97ac nodeName:}" failed. No retries permitted until 2026-09-05 07:49:33.100561295 +0000 UTC m=+75.870703614 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cafd9630-0d63-4c63-9829-bb8f215e97ac-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-g27lz" (UID: "cafd9630-0d63-4c63-9829-bb8f215e97ac") : secret "networking-console-plugin-cert" not found
Sep 05 07:49:17.102204 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:17.102173    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:17.103190 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:17.103169    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/27817b6a-02ee-4fde-856f-221f422e231f-registry-tls\") pod \"image-registry-79bddd854d-9vlsf\" (UID: \"27817b6a-02ee-4fde-856f-221f422e231f\") " pod="openshift-image-registry/image-registry-79bddd854d-9vlsf"
Sep 05 07:49:17.137410 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:17.137391    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79bddd854d-9vlsf"
Sep 05 07:49:17.170511 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:17.170407    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-hxgpj"]
Sep 05 07:49:17.410953 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:17.410922    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-79bddd854d-9vlsf"]
Sep 05 07:49:17.414484 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:49:17.414460    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27817b6a_02ee_4fde_856f_221f422e231f.slice/crio-610e9ab40f52f64112cb0196304ab4cc1048b1a15faf6195453de75bc33bbcee WatchSource:0}: Error finding container 610e9ab40f52f64112cb0196304ab4cc1048b1a15faf6195453de75bc33bbcee: Status 404 returned error can't find the container with id 610e9ab40f52f64112cb0196304ab4cc1048b1a15faf6195453de75bc33bbcee
Sep 05 07:49:17.972890 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:17.972858    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2bzp8"
Sep 05 07:49:18.067533 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:18.067493    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-hxgpj" event={"ID":"9b6376b1-fe4f-4e8f-b2aa-18b7d640d685","Type":"ContainerStarted","Data":"34fb9159cdaecc85465a84c96bc4981bb3b8bff87ede05b60a319962d66dfe44"}
Sep 05 07:49:18.069355 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:18.069328    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79bddd854d-9vlsf" event={"ID":"27817b6a-02ee-4fde-856f-221f422e231f","Type":"ContainerStarted","Data":"6b1be8a5f2907210f8047cc63ef7e2cfafd9721aad5d9d8676730b3d9e3c7e77"}
Sep 05 07:49:18.069484 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:18.069368    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79bddd854d-9vlsf" event={"ID":"27817b6a-02ee-4fde-856f-221f422e231f","Type":"ContainerStarted","Data":"610e9ab40f52f64112cb0196304ab4cc1048b1a15faf6195453de75bc33bbcee"}
Sep 05 07:49:18.069484 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:18.069466    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-79bddd854d-9vlsf"
Sep 05 07:49:18.089103 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:18.089041    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-79bddd854d-9vlsf" podStartSLOduration=17.089023784 podStartE2EDuration="17.089023784s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:49:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:18.08798844 +0000 UTC m=+60.858130797" watchObservedRunningTime="2026-09-05 07:49:18.089023784 +0000 UTC m=+60.859166122"
Sep 05 07:49:18.298810 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:18.298724    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:18.500439 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:18.500408    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:18.601486 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:18.601434    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:18.701074 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:18.701039    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:18.817701 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:18.817663    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:18.917442 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:18.917411    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:19.074866 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:19.074802    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-hxgpj" event={"ID":"9b6376b1-fe4f-4e8f-b2aa-18b7d640d685","Type":"ContainerStarted","Data":"f7cb98d07b5f4b4b8144bb687153dbaef2f544830450011f9b71bec5566e72b1"}
Sep 05 07:49:19.075197 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:19.074872    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-hxgpj" event={"ID":"9b6376b1-fe4f-4e8f-b2aa-18b7d640d685","Type":"ContainerStarted","Data":"94deb66274abeb3ab08ec6a0be65826090730e77985e2e8479c372e9b6744dda"}
Sep 05 07:49:19.089382 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:19.089337    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-hxgpj" podStartSLOduration=17.414187967 podStartE2EDuration="19.089319977s" totalImagesPullingTime="1.67513201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:49:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:19.089229015 +0000 UTC m=+61.859371354" watchObservedRunningTime="2026-09-05 07:49:19.089319977 +0000 UTC m=+61.859462316"
Sep 05 07:49:19.096544 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:19.096524    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:20.093257 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:20.093227    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:20.293681 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:20.293645    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:20.697635 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:20.697599    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:20.901258 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:20.901223    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:20.961485 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:20.961405    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:21.499429 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:21.499397    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:21.902380 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:21.902351    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:22.091573 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:22.091545    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:22.296634 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:22.296541    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:22.397388 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:22.397357    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:22.497456 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:22.497417    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:22.552474 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:22.552401    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:22.664645 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:22.664608    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:22.838517 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:22.838491    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:22.847552 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:22.847529    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3387beaa-e6d7-4f0a-bc56-0c06ef410d80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mtq5b\" (UID: \"3387beaa-e6d7-4f0a-bc56-0c06ef410d80\") " pod="openshift-insights/insights-runtime-extractor-mtq5b"
Sep 05 07:49:22.849952 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:22.849928    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3387beaa-e6d7-4f0a-bc56-0c06ef410d80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mtq5b\" (UID: \"3387beaa-e6d7-4f0a-bc56-0c06ef410d80\") " pod="openshift-insights/insights-runtime-extractor-mtq5b"
Sep 05 07:49:22.919258 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:22.919226    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gx2xl\""
Sep 05 07:49:22.927800 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:22.927774    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mtq5b"
Sep 05 07:49:23.213160 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:23.213083    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mtq5b"]
Sep 05 07:49:23.215998 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:49:23.215973    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3387beaa_e6d7_4f0a_bc56_0c06ef410d80.slice/crio-b59ff428588b463849b47fd881dd42cdf9e79f30a0486826b435cbf029f056b9 WatchSource:0}: Error finding container b59ff428588b463849b47fd881dd42cdf9e79f30a0486826b435cbf029f056b9: Status 404 returned error can't find the container with id b59ff428588b463849b47fd881dd42cdf9e79f30a0486826b435cbf029f056b9
Sep 05 07:49:23.425344 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:23.425311    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:23.452602 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:23.452578    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e352b34-8143-4a94-86e7-11a9460d2946-original-pull-secret\") pod \"global-pull-secret-syncer-lx5db\" (UID: \"7e352b34-8143-4a94-86e7-11a9460d2946\") " pod="kube-system/global-pull-secret-syncer-lx5db"
Sep 05 07:49:23.452694 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:23.452620    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/404c81cf-16aa-49f0-9059-fb97a5791ed7-metrics-certs\") pod \"network-metrics-daemon-l9lgq\" (UID: \"404c81cf-16aa-49f0-9059-fb97a5791ed7\") " pod="openshift-multus/network-metrics-daemon-l9lgq"
Sep 05 07:49:23.455211 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:23.455190    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 05 07:49:23.455289 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:23.455262    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 05 07:49:23.465401 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:23.465356    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/404c81cf-16aa-49f0-9059-fb97a5791ed7-metrics-certs\") pod \"network-metrics-daemon-l9lgq\" (UID: \"404c81cf-16aa-49f0-9059-fb97a5791ed7\") " pod="openshift-multus/network-metrics-daemon-l9lgq"
Sep 05 07:49:23.465401 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:23.465359    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e352b34-8143-4a94-86e7-11a9460d2946-original-pull-secret\") pod \"global-pull-secret-syncer-lx5db\" (UID: \"7e352b34-8143-4a94-86e7-11a9460d2946\") " pod="kube-system/global-pull-secret-syncer-lx5db"
Sep 05 07:49:23.524109 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:23.524078    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:23.553466 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:23.553441    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wt7mt\" (UniqueName: \"kubernetes.io/projected/b1e5c942-8397-439a-96b6-13149a974b18-kube-api-access-wt7mt\") pod \"network-check-target-rbtht\" (UID: \"b1e5c942-8397-439a-96b6-13149a974b18\") " pod="openshift-network-diagnostics/network-check-target-rbtht"
Sep 05 07:49:23.556032 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:23.556009    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 05 07:49:23.566146 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:23.566130    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 05 07:49:23.577093 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:23.577076    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wt7mt\" (UniqueName: \"kubernetes.io/projected/b1e5c942-8397-439a-96b6-13149a974b18-kube-api-access-wt7mt\") pod \"network-check-target-rbtht\" (UID: \"b1e5c942-8397-439a-96b6-13149a974b18\") " pod="openshift-network-diagnostics/network-check-target-rbtht"
Sep 05 07:49:23.635234 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:23.635214    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-g9xp6\""
Sep 05 07:49:23.637754 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:23.637739    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lx5db"
Sep 05 07:49:23.643856 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:23.643824    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rbtht"
Sep 05 07:49:23.741074 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:23.740996    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hp5x9\""
Sep 05 07:49:23.748887 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:23.748796    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9lgq"
Sep 05 07:49:23.851144 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:23.851118    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:23.910499 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:23.910434    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:23.932034 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:23.932006    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lx5db"]
Sep 05 07:49:23.968667 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:23.968633    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rbtht"]
Sep 05 07:49:24.032815 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:49:24.032727    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e352b34_8143_4a94_86e7_11a9460d2946.slice/crio-8f59cb816067c88116afe25ec71a834d67693ff2fcaf0840fb1d0db618fc5f49 WatchSource:0}: Error finding container 8f59cb816067c88116afe25ec71a834d67693ff2fcaf0840fb1d0db618fc5f49: Status 404 returned error can't find the container with id 8f59cb816067c88116afe25ec71a834d67693ff2fcaf0840fb1d0db618fc5f49
Sep 05 07:49:24.033698 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:49:24.033671    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1e5c942_8397_439a_96b6_13149a974b18.slice/crio-a3b7cf1487644dad02823275edf950ebe754a5d20bbb9b1e37e346cb789faa04 WatchSource:0}: Error finding container a3b7cf1487644dad02823275edf950ebe754a5d20bbb9b1e37e346cb789faa04: Status 404 returned error can't find the container with id a3b7cf1487644dad02823275edf950ebe754a5d20bbb9b1e37e346cb789faa04
Sep 05 07:49:24.034080 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:24.034056    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:24.039826 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:24.039801    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-l9lgq"]
Sep 05 07:49:24.042884 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:49:24.042859    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod404c81cf_16aa_49f0_9059_fb97a5791ed7.slice/crio-dc41366728dfe4218968cd92be48b6ec2293bf276cf434cf4944c613c7cecb69 WatchSource:0}: Error finding container dc41366728dfe4218968cd92be48b6ec2293bf276cf434cf4944c613c7cecb69: Status 404 returned error can't find the container with id dc41366728dfe4218968cd92be48b6ec2293bf276cf434cf4944c613c7cecb69
Sep 05 07:49:24.090729 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:24.090695    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rbtht" event={"ID":"b1e5c942-8397-439a-96b6-13149a974b18","Type":"ContainerStarted","Data":"a3b7cf1487644dad02823275edf950ebe754a5d20bbb9b1e37e346cb789faa04"}
Sep 05 07:49:24.092222 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:24.092199    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mtq5b" event={"ID":"3387beaa-e6d7-4f0a-bc56-0c06ef410d80","Type":"ContainerStarted","Data":"0a2ba490a063ae636a817dfe9ffce2e933ac87276284d4803a7ac22290d86b74"}
Sep 05 07:49:24.092339 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:24.092249    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mtq5b" event={"ID":"3387beaa-e6d7-4f0a-bc56-0c06ef410d80","Type":"ContainerStarted","Data":"b59ff428588b463849b47fd881dd42cdf9e79f30a0486826b435cbf029f056b9"}
Sep 05 07:49:24.093354 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:24.093332    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l9lgq" event={"ID":"404c81cf-16aa-49f0-9059-fb97a5791ed7","Type":"ContainerStarted","Data":"dc41366728dfe4218968cd92be48b6ec2293bf276cf434cf4944c613c7cecb69"}
Sep 05 07:49:24.094387 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:24.094369    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lx5db" event={"ID":"7e352b34-8143-4a94-86e7-11a9460d2946","Type":"ContainerStarted","Data":"8f59cb816067c88116afe25ec71a834d67693ff2fcaf0840fb1d0db618fc5f49"}
Sep 05 07:49:24.135495 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:24.135474    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:24.462756 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:24.462714    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2de9d5a3-384d-4f3e-866f-19f1b5d4126d-metrics-tls\") pod \"dns-default-9wmv2\" (UID: \"2de9d5a3-384d-4f3e-866f-19f1b5d4126d\") " pod="openshift-dns/dns-default-9wmv2"
Sep 05 07:49:24.465253 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:24.465230    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2de9d5a3-384d-4f3e-866f-19f1b5d4126d-metrics-tls\") pod \"dns-default-9wmv2\" (UID: \"2de9d5a3-384d-4f3e-866f-19f1b5d4126d\") " pod="openshift-dns/dns-default-9wmv2"
Sep 05 07:49:24.734407 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:24.734331    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-t6wxv\""
Sep 05 07:49:24.742745 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:24.742711    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9wmv2"
Sep 05 07:49:25.029446 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:25.029368    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9wmv2"]
Sep 05 07:49:25.041388 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:49:25.041225    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2de9d5a3_384d_4f3e_866f_19f1b5d4126d.slice/crio-d79d99eb012313588bf9f23f30de419ba3a1f18f852d45d4f457c5be6d978397 WatchSource:0}: Error finding container d79d99eb012313588bf9f23f30de419ba3a1f18f852d45d4f457c5be6d978397: Status 404 returned error can't find the container with id d79d99eb012313588bf9f23f30de419ba3a1f18f852d45d4f457c5be6d978397
Sep 05 07:49:25.104122 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:25.104088    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mtq5b" event={"ID":"3387beaa-e6d7-4f0a-bc56-0c06ef410d80","Type":"ContainerStarted","Data":"45906cab07aa5e1d7c54c4e61753aa973cbd56d8be06ba659b4bb6831e5e88b1"}
Sep 05 07:49:25.109129 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:25.109089    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9wmv2" event={"ID":"2de9d5a3-384d-4f3e-866f-19f1b5d4126d","Type":"ContainerStarted","Data":"d79d99eb012313588bf9f23f30de419ba3a1f18f852d45d4f457c5be6d978397"}
Sep 05 07:49:25.868709 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:25.868649    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:25.925767 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:25.925737    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:27.121917 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:27.121881    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l9lgq" event={"ID":"404c81cf-16aa-49f0-9059-fb97a5791ed7","Type":"ContainerStarted","Data":"0d81d25ba8d1c307eeb3d04625dc736459023a2cdc09079109115c1f251bc45b"}
Sep 05 07:49:27.122390 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:27.121947    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l9lgq" event={"ID":"404c81cf-16aa-49f0-9059-fb97a5791ed7","Type":"ContainerStarted","Data":"112958a2771b55889494b959a8ee7e394c2f3c42dc0d54f1afd3a8b1e636d5b6"}
Sep 05 07:49:27.124239 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:27.124202    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mtq5b" event={"ID":"3387beaa-e6d7-4f0a-bc56-0c06ef410d80","Type":"ContainerStarted","Data":"7dad491e8a649b0da413c70f99b5a55cc62b3fc1359386988823a0ef61d4468b"}
Sep 05 07:49:27.138588 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:27.138543    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-l9lgq" podStartSLOduration=67.136545138 podStartE2EDuration="1m9.138527625s" totalImagesPullingTime="2.001982487s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:48:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:27.136892598 +0000 UTC m=+69.907034970" watchObservedRunningTime="2026-09-05 07:49:27.138527625 +0000 UTC m=+69.908669961"
Sep 05 07:49:27.159730 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:27.159669    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mtq5b" podStartSLOduration=18.499469594 podStartE2EDuration="21.159653195s" totalImagesPullingTime="2.660183601s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 07:49:06 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:27.15872025 +0000 UTC m=+69.928862601" watchObservedRunningTime="2026-09-05 07:49:27.159653195 +0000 UTC m=+69.929795531"
Sep 05 07:49:30.137547 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:30.137511    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rbtht" event={"ID":"b1e5c942-8397-439a-96b6-13149a974b18","Type":"ContainerStarted","Data":"55ebb511da25bae905e63a9b5397093f5cbae33e6797177c32879c3d0451064d"}
Sep 05 07:49:30.138020 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:30.137623    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-rbtht"
Sep 05 07:49:30.139328 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:30.139303    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9wmv2" event={"ID":"2de9d5a3-384d-4f3e-866f-19f1b5d4126d","Type":"ContainerStarted","Data":"ee0cc61942447efd40ff3d5c58f9c17d0326ba3a8899525f8846164b39f1e9d1"}
Sep 05 07:49:30.139451 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:30.139347    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9wmv2" event={"ID":"2de9d5a3-384d-4f3e-866f-19f1b5d4126d","Type":"ContainerStarted","Data":"fa6574fd511a9228fec7ca35c199c4261bda7d09415ddb9b7e57fbe0d4c37fce"}
Sep 05 07:49:30.139451 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:30.139375    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9wmv2"
Sep 05 07:49:30.140826 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:30.140806    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lx5db" event={"ID":"7e352b34-8143-4a94-86e7-11a9460d2946","Type":"ContainerStarted","Data":"0e211906c8979bf94636d0295a1b842eb07b5a612b2cb881308e1fb8c05c3c9c"}
Sep 05 07:49:30.152764 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:30.152717    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-rbtht" podStartSLOduration=66.83476994 podStartE2EDuration="1m12.152703956s" totalImagesPullingTime="5.317934016s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:48:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:30.151859145 +0000 UTC m=+72.922001484" watchObservedRunningTime="2026-09-05 07:49:30.152703956 +0000 UTC m=+72.922846293"
Sep 05 07:49:30.169199 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:30.169151    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9wmv2" podStartSLOduration=33.858192195 podStartE2EDuration="38.169135882s" totalImagesPullingTime="4.310943687s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:48:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:30.168071763 +0000 UTC m=+72.938214120" watchObservedRunningTime="2026-09-05 07:49:30.169135882 +0000 UTC m=+72.939278233"
Sep 05 07:49:30.182371 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:30.182326    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-lx5db" podStartSLOduration=66.846356219 podStartE2EDuration="1m12.182308955s" totalImagesPullingTime="5.335952736s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 07:48:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:30.182070276 +0000 UTC m=+72.952212624" watchObservedRunningTime="2026-09-05 07:49:30.182308955 +0000 UTC m=+72.952451293"
Sep 05 07:49:32.285565 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.285531    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dc786958d-hhrvc"]
Sep 05 07:49:32.288587 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.288567    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dc786958d-hhrvc"
Sep 05 07:49:32.290986 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.290963    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-fbx7z\""
Sep 05 07:49:32.291891 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.291867    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 05 07:49:32.292004 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.291874    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 05 07:49:32.292004 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.291875    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 05 07:49:32.292004 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.291904    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 05 07:49:32.297033 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.297013    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-68944df8f6-zgn62"]
Sep 05 07:49:32.299742 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.299726    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68944df8f6-zgn62"
Sep 05 07:49:32.302144 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.302121    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dc786958d-hhrvc"]
Sep 05 07:49:32.302287 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.302266    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 05 07:49:32.317179 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.317156    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-79bddd854d-9vlsf"]
Sep 05 07:49:32.318662 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.318543    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-68944df8f6-zgn62"]
Sep 05 07:49:32.322554 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.322527    2563 patch_prober.go:28] interesting pod/image-registry-79bddd854d-9vlsf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 05 07:49:32.322650 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.322589    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-79bddd854d-9vlsf" podUID="27817b6a-02ee-4fde-856f-221f422e231f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 05 07:49:32.381599 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.381546    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:32.403262 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.403237    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-8p5k6"]
Sep 05 07:49:32.406138 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.406122    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-8p5k6"
Sep 05 07:49:32.408586 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.408562    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-722hd\""
Sep 05 07:49:32.408918 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.408902    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 05 07:49:32.409656 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.409643    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 05 07:49:32.413911 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.413883    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-768c75965-mrkgv"]
Sep 05 07:49:32.416592 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.416577    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-768c75965-mrkgv"
Sep 05 07:49:32.428505 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.428479    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-8p5k6"]
Sep 05 07:49:32.431080 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.431038    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dccca92d-bd8e-4fbc-b1f4-1dc28952eb99-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5dc786958d-hhrvc\" (UID: \"dccca92d-bd8e-4fbc-b1f4-1dc28952eb99\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dc786958d-hhrvc"
Sep 05 07:49:32.431275 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.431257    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e63927ae-347a-4b72-9f1f-74d0e5e2cad5-tmp\") pod \"klusterlet-addon-workmgr-68944df8f6-zgn62\" (UID: \"e63927ae-347a-4b72-9f1f-74d0e5e2cad5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68944df8f6-zgn62"
Sep 05 07:49:32.431517 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.431468    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e63927ae-347a-4b72-9f1f-74d0e5e2cad5-klusterlet-config\") pod \"klusterlet-addon-workmgr-68944df8f6-zgn62\" (UID: \"e63927ae-347a-4b72-9f1f-74d0e5e2cad5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68944df8f6-zgn62"
Sep 05 07:49:32.431643 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.431537    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jxh4\" (UniqueName: \"kubernetes.io/projected/e63927ae-347a-4b72-9f1f-74d0e5e2cad5-kube-api-access-2jxh4\") pod \"klusterlet-addon-workmgr-68944df8f6-zgn62\" (UID: \"e63927ae-347a-4b72-9f1f-74d0e5e2cad5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68944df8f6-zgn62"
Sep 05 07:49:32.431643 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.431595    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbtzn\" (UniqueName: \"kubernetes.io/projected/dccca92d-bd8e-4fbc-b1f4-1dc28952eb99-kube-api-access-vbtzn\") pod \"managed-serviceaccount-addon-agent-5dc786958d-hhrvc\" (UID: \"dccca92d-bd8e-4fbc-b1f4-1dc28952eb99\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dc786958d-hhrvc"
Sep 05 07:49:32.432604 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.432583    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-768c75965-mrkgv"]
Sep 05 07:49:32.531921 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.531890    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jxh4\" (UniqueName: \"kubernetes.io/projected/e63927ae-347a-4b72-9f1f-74d0e5e2cad5-kube-api-access-2jxh4\") pod \"klusterlet-addon-workmgr-68944df8f6-zgn62\" (UID: \"e63927ae-347a-4b72-9f1f-74d0e5e2cad5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68944df8f6-zgn62"
Sep 05 07:49:32.531921 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.531926    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5cfee253-a143-4b62-af4a-54a94e8d2c04-image-registry-private-configuration\") pod \"image-registry-768c75965-mrkgv\" (UID: \"5cfee253-a143-4b62-af4a-54a94e8d2c04\") " pod="openshift-image-registry/image-registry-768c75965-mrkgv"
Sep 05 07:49:32.532118 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.531957    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jp7hm\" (UniqueName: \"kubernetes.io/projected/c7f6c7f2-bb81-4da8-8ff6-2be8dae1554d-kube-api-access-jp7hm\") pod \"downloads-85d98c998f-8p5k6\" (UID: \"c7f6c7f2-bb81-4da8-8ff6-2be8dae1554d\") " pod="openshift-console/downloads-85d98c998f-8p5k6"
Sep 05 07:49:32.532118 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.532014    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbtzn\" (UniqueName: \"kubernetes.io/projected/dccca92d-bd8e-4fbc-b1f4-1dc28952eb99-kube-api-access-vbtzn\") pod \"managed-serviceaccount-addon-agent-5dc786958d-hhrvc\" (UID: \"dccca92d-bd8e-4fbc-b1f4-1dc28952eb99\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dc786958d-hhrvc"
Sep 05 07:49:32.532118 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.532084    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5cfee253-a143-4b62-af4a-54a94e8d2c04-registry-certificates\") pod \"image-registry-768c75965-mrkgv\" (UID: \"5cfee253-a143-4b62-af4a-54a94e8d2c04\") " pod="openshift-image-registry/image-registry-768c75965-mrkgv"
Sep 05 07:49:32.532118 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.532108    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5cfee253-a143-4b62-af4a-54a94e8d2c04-installation-pull-secrets\") pod \"image-registry-768c75965-mrkgv\" (UID: \"5cfee253-a143-4b62-af4a-54a94e8d2c04\") " pod="openshift-image-registry/image-registry-768c75965-mrkgv"
Sep 05 07:49:32.532262 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.532155    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dccca92d-bd8e-4fbc-b1f4-1dc28952eb99-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5dc786958d-hhrvc\" (UID: \"dccca92d-bd8e-4fbc-b1f4-1dc28952eb99\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dc786958d-hhrvc"
Sep 05 07:49:32.532262 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.532194    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e63927ae-347a-4b72-9f1f-74d0e5e2cad5-tmp\") pod \"klusterlet-addon-workmgr-68944df8f6-zgn62\" (UID: \"e63927ae-347a-4b72-9f1f-74d0e5e2cad5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68944df8f6-zgn62"
Sep 05 07:49:32.532262 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.532223    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5cfee253-a143-4b62-af4a-54a94e8d2c04-registry-tls\") pod \"image-registry-768c75965-mrkgv\" (UID: \"5cfee253-a143-4b62-af4a-54a94e8d2c04\") " pod="openshift-image-registry/image-registry-768c75965-mrkgv"
Sep 05 07:49:32.532262 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.532250    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5cfee253-a143-4b62-af4a-54a94e8d2c04-ca-trust-extracted\") pod \"image-registry-768c75965-mrkgv\" (UID: \"5cfee253-a143-4b62-af4a-54a94e8d2c04\") " pod="openshift-image-registry/image-registry-768c75965-mrkgv"
Sep 05 07:49:32.532440 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.532267    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5cfee253-a143-4b62-af4a-54a94e8d2c04-bound-sa-token\") pod \"image-registry-768c75965-mrkgv\" (UID: \"5cfee253-a143-4b62-af4a-54a94e8d2c04\") " pod="openshift-image-registry/image-registry-768c75965-mrkgv"
Sep 05 07:49:32.532440 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.532291    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5cfee253-a143-4b62-af4a-54a94e8d2c04-trusted-ca\") pod \"image-registry-768c75965-mrkgv\" (UID: \"5cfee253-a143-4b62-af4a-54a94e8d2c04\") " pod="openshift-image-registry/image-registry-768c75965-mrkgv"
Sep 05 07:49:32.532440 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.532308    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e63927ae-347a-4b72-9f1f-74d0e5e2cad5-klusterlet-config\") pod \"klusterlet-addon-workmgr-68944df8f6-zgn62\" (UID: \"e63927ae-347a-4b72-9f1f-74d0e5e2cad5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68944df8f6-zgn62"
Sep 05 07:49:32.532440 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.532354    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtct7\" (UniqueName: \"kubernetes.io/projected/5cfee253-a143-4b62-af4a-54a94e8d2c04-kube-api-access-vtct7\") pod \"image-registry-768c75965-mrkgv\" (UID: \"5cfee253-a143-4b62-af4a-54a94e8d2c04\") " pod="openshift-image-registry/image-registry-768c75965-mrkgv"
Sep 05 07:49:32.532622 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.532592    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e63927ae-347a-4b72-9f1f-74d0e5e2cad5-tmp\") pod \"klusterlet-addon-workmgr-68944df8f6-zgn62\" (UID: \"e63927ae-347a-4b72-9f1f-74d0e5e2cad5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68944df8f6-zgn62"
Sep 05 07:49:32.534765 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.534743    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dccca92d-bd8e-4fbc-b1f4-1dc28952eb99-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5dc786958d-hhrvc\" (UID: \"dccca92d-bd8e-4fbc-b1f4-1dc28952eb99\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dc786958d-hhrvc"
Sep 05 07:49:32.534878 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.534768    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e63927ae-347a-4b72-9f1f-74d0e5e2cad5-klusterlet-config\") pod \"klusterlet-addon-workmgr-68944df8f6-zgn62\" (UID: \"e63927ae-347a-4b72-9f1f-74d0e5e2cad5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68944df8f6-zgn62"
Sep 05 07:49:32.540502 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.540447    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jxh4\" (UniqueName: \"kubernetes.io/projected/e63927ae-347a-4b72-9f1f-74d0e5e2cad5-kube-api-access-2jxh4\") pod \"klusterlet-addon-workmgr-68944df8f6-zgn62\" (UID: \"e63927ae-347a-4b72-9f1f-74d0e5e2cad5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68944df8f6-zgn62"
Sep 05 07:49:32.540933 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.540916    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbtzn\" (UniqueName: \"kubernetes.io/projected/dccca92d-bd8e-4fbc-b1f4-1dc28952eb99-kube-api-access-vbtzn\") pod \"managed-serviceaccount-addon-agent-5dc786958d-hhrvc\" (UID: \"dccca92d-bd8e-4fbc-b1f4-1dc28952eb99\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dc786958d-hhrvc"
Sep 05 07:49:32.566297 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.566274    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:32.605656 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.605636    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dc786958d-hhrvc"
Sep 05 07:49:32.612620 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.612601    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68944df8f6-zgn62"
Sep 05 07:49:32.633825 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.633795    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtct7\" (UniqueName: \"kubernetes.io/projected/5cfee253-a143-4b62-af4a-54a94e8d2c04-kube-api-access-vtct7\") pod \"image-registry-768c75965-mrkgv\" (UID: \"5cfee253-a143-4b62-af4a-54a94e8d2c04\") " pod="openshift-image-registry/image-registry-768c75965-mrkgv"
Sep 05 07:49:32.633962 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.633864    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5cfee253-a143-4b62-af4a-54a94e8d2c04-image-registry-private-configuration\") pod \"image-registry-768c75965-mrkgv\" (UID: \"5cfee253-a143-4b62-af4a-54a94e8d2c04\") " pod="openshift-image-registry/image-registry-768c75965-mrkgv"
Sep 05 07:49:32.633962 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.633905    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jp7hm\" (UniqueName: \"kubernetes.io/projected/c7f6c7f2-bb81-4da8-8ff6-2be8dae1554d-kube-api-access-jp7hm\") pod \"downloads-85d98c998f-8p5k6\" (UID: \"c7f6c7f2-bb81-4da8-8ff6-2be8dae1554d\") " pod="openshift-console/downloads-85d98c998f-8p5k6"
Sep 05 07:49:32.634070 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.633973    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5cfee253-a143-4b62-af4a-54a94e8d2c04-registry-certificates\") pod \"image-registry-768c75965-mrkgv\" (UID: \"5cfee253-a143-4b62-af4a-54a94e8d2c04\") " pod="openshift-image-registry/image-registry-768c75965-mrkgv"
Sep 05 07:49:32.634070 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.633997    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5cfee253-a143-4b62-af4a-54a94e8d2c04-installation-pull-secrets\") pod \"image-registry-768c75965-mrkgv\" (UID: \"5cfee253-a143-4b62-af4a-54a94e8d2c04\") " pod="openshift-image-registry/image-registry-768c75965-mrkgv"
Sep 05 07:49:32.634070 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.634064    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5cfee253-a143-4b62-af4a-54a94e8d2c04-registry-tls\") pod \"image-registry-768c75965-mrkgv\" (UID: \"5cfee253-a143-4b62-af4a-54a94e8d2c04\") " pod="openshift-image-registry/image-registry-768c75965-mrkgv"
Sep 05 07:49:32.634202 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.634098    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5cfee253-a143-4b62-af4a-54a94e8d2c04-ca-trust-extracted\") pod \"image-registry-768c75965-mrkgv\" (UID: \"5cfee253-a143-4b62-af4a-54a94e8d2c04\") " pod="openshift-image-registry/image-registry-768c75965-mrkgv"
Sep 05 07:49:32.634202 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.634120    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5cfee253-a143-4b62-af4a-54a94e8d2c04-bound-sa-token\") pod \"image-registry-768c75965-mrkgv\" (UID: \"5cfee253-a143-4b62-af4a-54a94e8d2c04\") " pod="openshift-image-registry/image-registry-768c75965-mrkgv"
Sep 05 07:49:32.634202 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.634155    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5cfee253-a143-4b62-af4a-54a94e8d2c04-trusted-ca\") pod \"image-registry-768c75965-mrkgv\" (UID: \"5cfee253-a143-4b62-af4a-54a94e8d2c04\") " pod="openshift-image-registry/image-registry-768c75965-mrkgv"
Sep 05 07:49:32.635770 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.635218    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5cfee253-a143-4b62-af4a-54a94e8d2c04-ca-trust-extracted\") pod \"image-registry-768c75965-mrkgv\" (UID: \"5cfee253-a143-4b62-af4a-54a94e8d2c04\") " pod="openshift-image-registry/image-registry-768c75965-mrkgv"
Sep 05 07:49:32.635770 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.635721    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5cfee253-a143-4b62-af4a-54a94e8d2c04-registry-certificates\") pod \"image-registry-768c75965-mrkgv\" (UID: \"5cfee253-a143-4b62-af4a-54a94e8d2c04\") " pod="openshift-image-registry/image-registry-768c75965-mrkgv"
Sep 05 07:49:32.635959 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.635887    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5cfee253-a143-4b62-af4a-54a94e8d2c04-trusted-ca\") pod \"image-registry-768c75965-mrkgv\" (UID: \"5cfee253-a143-4b62-af4a-54a94e8d2c04\") " pod="openshift-image-registry/image-registry-768c75965-mrkgv"
Sep 05 07:49:32.636824 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.636786    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5cfee253-a143-4b62-af4a-54a94e8d2c04-installation-pull-secrets\") pod \"image-registry-768c75965-mrkgv\" (UID: \"5cfee253-a143-4b62-af4a-54a94e8d2c04\") " pod="openshift-image-registry/image-registry-768c75965-mrkgv"
Sep 05 07:49:32.636947 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.636884    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5cfee253-a143-4b62-af4a-54a94e8d2c04-image-registry-private-configuration\") pod \"image-registry-768c75965-mrkgv\" (UID: \"5cfee253-a143-4b62-af4a-54a94e8d2c04\") " pod="openshift-image-registry/image-registry-768c75965-mrkgv"
Sep 05 07:49:32.637091 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.637071    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5cfee253-a143-4b62-af4a-54a94e8d2c04-registry-tls\") pod \"image-registry-768c75965-mrkgv\" (UID: \"5cfee253-a143-4b62-af4a-54a94e8d2c04\") " pod="openshift-image-registry/image-registry-768c75965-mrkgv"
Sep 05 07:49:32.643887 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.643812    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtct7\" (UniqueName: \"kubernetes.io/projected/5cfee253-a143-4b62-af4a-54a94e8d2c04-kube-api-access-vtct7\") pod \"image-registry-768c75965-mrkgv\" (UID: \"5cfee253-a143-4b62-af4a-54a94e8d2c04\") " pod="openshift-image-registry/image-registry-768c75965-mrkgv"
Sep 05 07:49:32.645196 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.645167    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5cfee253-a143-4b62-af4a-54a94e8d2c04-bound-sa-token\") pod \"image-registry-768c75965-mrkgv\" (UID: \"5cfee253-a143-4b62-af4a-54a94e8d2c04\") " pod="openshift-image-registry/image-registry-768c75965-mrkgv"
Sep 05 07:49:32.646124 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.646090    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jp7hm\" (UniqueName: \"kubernetes.io/projected/c7f6c7f2-bb81-4da8-8ff6-2be8dae1554d-kube-api-access-jp7hm\") pod \"downloads-85d98c998f-8p5k6\" (UID: \"c7f6c7f2-bb81-4da8-8ff6-2be8dae1554d\") " pod="openshift-console/downloads-85d98c998f-8p5k6"
Sep 05 07:49:32.665484 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.665400    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:32.715452 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.715426    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-8p5k6"
Sep 05 07:49:32.724340 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.724324    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-768c75965-mrkgv"
Sep 05 07:49:32.878362 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.878330    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dc786958d-hhrvc"]
Sep 05 07:49:32.881279 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:49:32.881250    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddccca92d_bd8e_4fbc_b1f4_1dc28952eb99.slice/crio-48778c29fce75bf61c857ec028b353c6ae52dc5a150d4bb139a7516c06f342e8 WatchSource:0}: Error finding container 48778c29fce75bf61c857ec028b353c6ae52dc5a150d4bb139a7516c06f342e8: Status 404 returned error can't find the container with id 48778c29fce75bf61c857ec028b353c6ae52dc5a150d4bb139a7516c06f342e8
Sep 05 07:49:32.895379 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.895359    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-68944df8f6-zgn62"]
Sep 05 07:49:32.898073 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:49:32.898052    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode63927ae_347a_4b72_9f1f_74d0e5e2cad5.slice/crio-1082ec4d0481102c3792d4d00b764baccd34f5254c6ba42f70703ac6c3de8c79 WatchSource:0}: Error finding container 1082ec4d0481102c3792d4d00b764baccd34f5254c6ba42f70703ac6c3de8c79: Status 404 returned error can't find the container with id 1082ec4d0481102c3792d4d00b764baccd34f5254c6ba42f70703ac6c3de8c79
Sep 05 07:49:32.936755 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.936721    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9163aed9-c681-4b4d-a7f9-63438c02656f-service-ca-bundle\") pod \"router-default-8557f77dd-lw4qb\" (UID: \"9163aed9-c681-4b4d-a7f9-63438c02656f\") " pod="openshift-ingress/router-default-8557f77dd-lw4qb"
Sep 05 07:49:32.937305 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:32.937286    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9163aed9-c681-4b4d-a7f9-63438c02656f-service-ca-bundle\") pod \"router-default-8557f77dd-lw4qb\" (UID: \"9163aed9-c681-4b4d-a7f9-63438c02656f\") " pod="openshift-ingress/router-default-8557f77dd-lw4qb"
Sep 05 07:49:33.002967 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:33.002768    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-8p5k6"]
Sep 05 07:49:33.005338 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:49:33.005307    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7f6c7f2_bb81_4da8_8ff6_2be8dae1554d.slice/crio-b6b69f9ceb18952c12ca1363d79b3f1d2f12437839104f26b7c1a1b5da5686cc WatchSource:0}: Error finding container b6b69f9ceb18952c12ca1363d79b3f1d2f12437839104f26b7c1a1b5da5686cc: Status 404 returned error can't find the container with id b6b69f9ceb18952c12ca1363d79b3f1d2f12437839104f26b7c1a1b5da5686cc
Sep 05 07:49:33.035927 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:33.035902    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-768c75965-mrkgv"]
Sep 05 07:49:33.038545 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:49:33.038514    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cfee253_a143_4b62_af4a_54a94e8d2c04.slice/crio-1297d15e9bf67b0ee4a72521ec271a9728c3a0595b9a70a91e5c832854c16ac5 WatchSource:0}: Error finding container 1297d15e9bf67b0ee4a72521ec271a9728c3a0595b9a70a91e5c832854c16ac5: Status 404 returned error can't find the container with id 1297d15e9bf67b0ee4a72521ec271a9728c3a0595b9a70a91e5c832854c16ac5
Sep 05 07:49:33.138175 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:33.138104    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cafd9630-0d63-4c63-9829-bb8f215e97ac-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-g27lz\" (UID: \"cafd9630-0d63-4c63-9829-bb8f215e97ac\") " pod="openshift-network-console/networking-console-plugin-79977f9844-g27lz"
Sep 05 07:49:33.140444 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:33.140407    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cafd9630-0d63-4c63-9829-bb8f215e97ac-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-g27lz\" (UID: \"cafd9630-0d63-4c63-9829-bb8f215e97ac\") " pod="openshift-network-console/networking-console-plugin-79977f9844-g27lz"
Sep 05 07:49:33.144306 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:33.144288    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-ml7q9\""
Sep 05 07:49:33.153000 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:33.152972    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-8557f77dd-lw4qb"
Sep 05 07:49:33.154475 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:33.154441    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-8p5k6" event={"ID":"c7f6c7f2-bb81-4da8-8ff6-2be8dae1554d","Type":"ContainerStarted","Data":"b6b69f9ceb18952c12ca1363d79b3f1d2f12437839104f26b7c1a1b5da5686cc"}
Sep 05 07:49:33.155782 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:33.155745    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68944df8f6-zgn62" event={"ID":"e63927ae-347a-4b72-9f1f-74d0e5e2cad5","Type":"ContainerStarted","Data":"1082ec4d0481102c3792d4d00b764baccd34f5254c6ba42f70703ac6c3de8c79"}
Sep 05 07:49:33.157097 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:33.157068    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dc786958d-hhrvc" event={"ID":"dccca92d-bd8e-4fbc-b1f4-1dc28952eb99","Type":"ContainerStarted","Data":"48778c29fce75bf61c857ec028b353c6ae52dc5a150d4bb139a7516c06f342e8"}
Sep 05 07:49:33.158686 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:33.158663    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-768c75965-mrkgv" event={"ID":"5cfee253-a143-4b62-af4a-54a94e8d2c04","Type":"ContainerStarted","Data":"ea06b5553459e3e798f26d0ab7ea694d7719a9f5e028430aab04317ea5cc9fe1"}
Sep 05 07:49:33.158787 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:33.158705    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-768c75965-mrkgv" event={"ID":"5cfee253-a143-4b62-af4a-54a94e8d2c04","Type":"ContainerStarted","Data":"1297d15e9bf67b0ee4a72521ec271a9728c3a0595b9a70a91e5c832854c16ac5"}
Sep 05 07:49:33.158787 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:33.158742    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-768c75965-mrkgv"
Sep 05 07:49:33.180990 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:33.180940    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-768c75965-mrkgv" podStartSLOduration=1.1809231279999999 podStartE2EDuration="1.180923128s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:49:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:33.179717516 +0000 UTC m=+75.949859854" watchObservedRunningTime="2026-09-05 07:49:33.180923128 +0000 UTC m=+75.951065464"
Sep 05 07:49:33.185782 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:33.185750    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:33.286176 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:33.286140    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:33.313379 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:33.313354    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-676t7\""
Sep 05 07:49:33.322324 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:33.322296    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-g27lz"
Sep 05 07:49:33.436947 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:33.436009    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-8557f77dd-lw4qb"]
Sep 05 07:49:33.442099 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:49:33.442069    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9163aed9_c681_4b4d_a7f9_63438c02656f.slice/crio-cf7c7b989f4b47fdba7100cb9dde092ca209288e460bc0d9174a0f200e05a0ca WatchSource:0}: Error finding container cf7c7b989f4b47fdba7100cb9dde092ca209288e460bc0d9174a0f200e05a0ca: Status 404 returned error can't find the container with id cf7c7b989f4b47fdba7100cb9dde092ca209288e460bc0d9174a0f200e05a0ca
Sep 05 07:49:33.635227 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:33.635146    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-g27lz"]
Sep 05 07:49:33.640495 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:49:33.640461    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcafd9630_0d63_4c63_9829_bb8f215e97ac.slice/crio-1d8a0e2cb467c2d897b2f823ad3aef24b05d88ece0720d46b080ba4394754488 WatchSource:0}: Error finding container 1d8a0e2cb467c2d897b2f823ad3aef24b05d88ece0720d46b080ba4394754488: Status 404 returned error can't find the container with id 1d8a0e2cb467c2d897b2f823ad3aef24b05d88ece0720d46b080ba4394754488
Sep 05 07:49:34.167042 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:34.166960    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-g27lz" event={"ID":"cafd9630-0d63-4c63-9829-bb8f215e97ac","Type":"ContainerStarted","Data":"1d8a0e2cb467c2d897b2f823ad3aef24b05d88ece0720d46b080ba4394754488"}
Sep 05 07:49:34.170539 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:34.170404    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-8557f77dd-lw4qb" event={"ID":"9163aed9-c681-4b4d-a7f9-63438c02656f","Type":"ContainerStarted","Data":"1e68ddd94b1dbf96cbb25c671b437576fac5dfb39165db338f52ccba9fc41721"}
Sep 05 07:49:34.170539 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:34.170445    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-8557f77dd-lw4qb" event={"ID":"9163aed9-c681-4b4d-a7f9-63438c02656f","Type":"ContainerStarted","Data":"cf7c7b989f4b47fdba7100cb9dde092ca209288e460bc0d9174a0f200e05a0ca"}
Sep 05 07:49:34.190712 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:34.190632    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-8557f77dd-lw4qb" podStartSLOduration=34.19061201 podStartE2EDuration="34.19061201s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:49:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:34.189046568 +0000 UTC m=+76.959188928" watchObservedRunningTime="2026-09-05 07:49:34.19061201 +0000 UTC m=+76.960754349"
Sep 05 07:49:34.870500 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:34.870450    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:34.928267 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:34.928235    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:35.153360 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:35.153282    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-8557f77dd-lw4qb"
Sep 05 07:49:35.156762 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:35.156507    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-8557f77dd-lw4qb"
Sep 05 07:49:35.176934 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:35.176887    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-8557f77dd-lw4qb"
Sep 05 07:49:35.178904 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:35.178683    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-8557f77dd-lw4qb"
Sep 05 07:49:35.869476 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:35.869440    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:35.970433 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:35.970393    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:36.869761 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:36.869729    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:37.541423 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:37.541368    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:37.640030 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:37.639947    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:37.731762 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:37.731731    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:37.811044 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:37.811012    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:38.069420 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:38.069331    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:38.126678 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:38.126607    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:38.435592 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:38.435506    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:38.501238 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:38.501196    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:38.858653 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:38.858588    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:38.959298 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:38.959208    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:39.196553 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:39.196510    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68944df8f6-zgn62" event={"ID":"e63927ae-347a-4b72-9f1f-74d0e5e2cad5","Type":"ContainerStarted","Data":"54b4bfb39b6359658313d5f067cb9ca22e2057c9f1b1a0fcb9309937ccaba861"}
Sep 05 07:49:39.196736 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:39.196671    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68944df8f6-zgn62"
Sep 05 07:49:39.198801 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:39.198775    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dc786958d-hhrvc" event={"ID":"dccca92d-bd8e-4fbc-b1f4-1dc28952eb99","Type":"ContainerStarted","Data":"8784df1d36688a34b977ba687a42225d70b0b6ac5c85a8f4d5283cdfae37e605"}
Sep 05 07:49:39.198938 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:39.198873    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68944df8f6-zgn62"
Sep 05 07:49:39.200727 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:39.200683    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-g27lz" event={"ID":"cafd9630-0d63-4c63-9829-bb8f215e97ac","Type":"ContainerStarted","Data":"3be622c182f5fc34a67cbc26a5893f1bb943d278d1d9f53dbe82932c9b01d547"}
Sep 05 07:49:39.215414 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:39.215336    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68944df8f6-zgn62" podStartSLOduration=1.429894032 podStartE2EDuration="7.215322556s" totalImagesPullingTime="5.785428524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:49:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:39.214123483 +0000 UTC m=+81.984265845" watchObservedRunningTime="2026-09-05 07:49:39.215322556 +0000 UTC m=+81.985464887"
Sep 05 07:49:39.229970 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:39.229916    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-g27lz" podStartSLOduration=33.212643571 podStartE2EDuration="38.229898881s" totalImagesPullingTime="5.01725531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:49:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:39.229249552 +0000 UTC m=+81.999391889" watchObservedRunningTime="2026-09-05 07:49:39.229898881 +0000 UTC m=+82.000041214"
Sep 05 07:49:39.251607 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:39.251555    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dc786958d-hhrvc" podStartSLOduration=1.464402534 podStartE2EDuration="7.25152801s" totalImagesPullingTime="5.787125476s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:49:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:39.251447859 +0000 UTC m=+82.021590196" watchObservedRunningTime="2026-09-05 07:49:39.25152801 +0000 UTC m=+82.021670348"
Sep 05 07:49:39.280584 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:39.280552    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:39.380648 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:39.380617    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:39.446656 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:39.446628    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:40.086050 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:40.086014    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:40.147200 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:40.147172    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9wmv2"
Sep 05 07:49:40.151550 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:40.151517    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:41.748327 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:41.748292    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:41.819178 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:41.819147    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:41.979237 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:41.979207    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:42.289784 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:42.289750    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:42.322120 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:42.322085    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-79bddd854d-9vlsf"
Sep 05 07:49:42.390947 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:42.390914    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:44.576176 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:44.576145    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:44.976148 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:44.976079    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:45.499039 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:45.499002    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:45.577516 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:45.577481    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:45.656424 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:45.656392    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:46.469406 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:46.469374    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:46.570009 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:46.569963    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:46.933938 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:46.933904    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:47.044396 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.044363    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:47.172091 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.172058    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:47.290413 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.288979    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-94gwm"]
Sep 05 07:49:47.292344 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.292267    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:47.295267 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.295243    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-94gwm"
Sep 05 07:49:47.298402 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.298355    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-9tz6f\""
Sep 05 07:49:47.302279 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.302255    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 05 07:49:47.302279 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.302273    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 05 07:49:47.302432 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.302273    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 05 07:49:47.302432 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.302321    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 05 07:49:47.302432 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.302333    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 05 07:49:47.302658 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.302641    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 05 07:49:47.406668 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.406638    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:47.470638 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.470604    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c91637a2-daa6-4523-8a9e-b2da9321546f-node-exporter-textfile\") pod \"node-exporter-94gwm\" (UID: \"c91637a2-daa6-4523-8a9e-b2da9321546f\") " pod="openshift-monitoring/node-exporter-94gwm"
Sep 05 07:49:47.470801 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.470645    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s89bx\" (UniqueName: \"kubernetes.io/projected/c91637a2-daa6-4523-8a9e-b2da9321546f-kube-api-access-s89bx\") pod \"node-exporter-94gwm\" (UID: \"c91637a2-daa6-4523-8a9e-b2da9321546f\") " pod="openshift-monitoring/node-exporter-94gwm"
Sep 05 07:49:47.470801 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.470669    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c91637a2-daa6-4523-8a9e-b2da9321546f-sys\") pod \"node-exporter-94gwm\" (UID: \"c91637a2-daa6-4523-8a9e-b2da9321546f\") " pod="openshift-monitoring/node-exporter-94gwm"
Sep 05 07:49:47.470801 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.470762    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c91637a2-daa6-4523-8a9e-b2da9321546f-metrics-client-ca\") pod \"node-exporter-94gwm\" (UID: \"c91637a2-daa6-4523-8a9e-b2da9321546f\") " pod="openshift-monitoring/node-exporter-94gwm"
Sep 05 07:49:47.470801 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.470796    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c91637a2-daa6-4523-8a9e-b2da9321546f-node-exporter-tls\") pod \"node-exporter-94gwm\" (UID: \"c91637a2-daa6-4523-8a9e-b2da9321546f\") " pod="openshift-monitoring/node-exporter-94gwm"
Sep 05 07:49:47.471014 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.470864    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c91637a2-daa6-4523-8a9e-b2da9321546f-node-exporter-accelerators-collector-config\") pod \"node-exporter-94gwm\" (UID: \"c91637a2-daa6-4523-8a9e-b2da9321546f\") " pod="openshift-monitoring/node-exporter-94gwm"
Sep 05 07:49:47.471014 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.470916    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c91637a2-daa6-4523-8a9e-b2da9321546f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-94gwm\" (UID: \"c91637a2-daa6-4523-8a9e-b2da9321546f\") " pod="openshift-monitoring/node-exporter-94gwm"
Sep 05 07:49:47.471014 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.470948    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c91637a2-daa6-4523-8a9e-b2da9321546f-node-exporter-wtmp\") pod \"node-exporter-94gwm\" (UID: \"c91637a2-daa6-4523-8a9e-b2da9321546f\") " pod="openshift-monitoring/node-exporter-94gwm"
Sep 05 07:49:47.471014 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.470982    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c91637a2-daa6-4523-8a9e-b2da9321546f-root\") pod \"node-exporter-94gwm\" (UID: \"c91637a2-daa6-4523-8a9e-b2da9321546f\") " pod="openshift-monitoring/node-exporter-94gwm"
Sep 05 07:49:47.504813 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.504776    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:47.571630 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.571581    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c91637a2-daa6-4523-8a9e-b2da9321546f-node-exporter-textfile\") pod \"node-exporter-94gwm\" (UID: \"c91637a2-daa6-4523-8a9e-b2da9321546f\") " pod="openshift-monitoring/node-exporter-94gwm"
Sep 05 07:49:47.571807 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.571638    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s89bx\" (UniqueName: \"kubernetes.io/projected/c91637a2-daa6-4523-8a9e-b2da9321546f-kube-api-access-s89bx\") pod \"node-exporter-94gwm\" (UID: \"c91637a2-daa6-4523-8a9e-b2da9321546f\") " pod="openshift-monitoring/node-exporter-94gwm"
Sep 05 07:49:47.571807 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.571662    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c91637a2-daa6-4523-8a9e-b2da9321546f-sys\") pod \"node-exporter-94gwm\" (UID: \"c91637a2-daa6-4523-8a9e-b2da9321546f\") " pod="openshift-monitoring/node-exporter-94gwm"
Sep 05 07:49:47.571807 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.571688    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c91637a2-daa6-4523-8a9e-b2da9321546f-metrics-client-ca\") pod \"node-exporter-94gwm\" (UID: \"c91637a2-daa6-4523-8a9e-b2da9321546f\") " pod="openshift-monitoring/node-exporter-94gwm"
Sep 05 07:49:47.571807 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.571709    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c91637a2-daa6-4523-8a9e-b2da9321546f-node-exporter-tls\") pod \"node-exporter-94gwm\" (UID: \"c91637a2-daa6-4523-8a9e-b2da9321546f\") " pod="openshift-monitoring/node-exporter-94gwm"
Sep 05 07:49:47.571807 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.571749    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c91637a2-daa6-4523-8a9e-b2da9321546f-node-exporter-accelerators-collector-config\") pod \"node-exporter-94gwm\" (UID: \"c91637a2-daa6-4523-8a9e-b2da9321546f\") " pod="openshift-monitoring/node-exporter-94gwm"
Sep 05 07:49:47.571807 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.571772    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c91637a2-daa6-4523-8a9e-b2da9321546f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-94gwm\" (UID: \"c91637a2-daa6-4523-8a9e-b2da9321546f\") " pod="openshift-monitoring/node-exporter-94gwm"
Sep 05 07:49:47.571807 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.571780    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c91637a2-daa6-4523-8a9e-b2da9321546f-sys\") pod \"node-exporter-94gwm\" (UID: \"c91637a2-daa6-4523-8a9e-b2da9321546f\") " pod="openshift-monitoring/node-exporter-94gwm"
Sep 05 07:49:47.572118 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.571792    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c91637a2-daa6-4523-8a9e-b2da9321546f-node-exporter-wtmp\") pod \"node-exporter-94gwm\" (UID: \"c91637a2-daa6-4523-8a9e-b2da9321546f\") " pod="openshift-monitoring/node-exporter-94gwm"
Sep 05 07:49:47.572118 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.571880    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c91637a2-daa6-4523-8a9e-b2da9321546f-root\") pod \"node-exporter-94gwm\" (UID: \"c91637a2-daa6-4523-8a9e-b2da9321546f\") " pod="openshift-monitoring/node-exporter-94gwm"
Sep 05 07:49:47.572118 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.571910    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c91637a2-daa6-4523-8a9e-b2da9321546f-node-exporter-wtmp\") pod \"node-exporter-94gwm\" (UID: \"c91637a2-daa6-4523-8a9e-b2da9321546f\") " pod="openshift-monitoring/node-exporter-94gwm"
Sep 05 07:49:47.572118 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.571975    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c91637a2-daa6-4523-8a9e-b2da9321546f-node-exporter-textfile\") pod \"node-exporter-94gwm\" (UID: \"c91637a2-daa6-4523-8a9e-b2da9321546f\") " pod="openshift-monitoring/node-exporter-94gwm"
Sep 05 07:49:47.572118 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.572043    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c91637a2-daa6-4523-8a9e-b2da9321546f-root\") pod \"node-exporter-94gwm\" (UID: \"c91637a2-daa6-4523-8a9e-b2da9321546f\") " pod="openshift-monitoring/node-exporter-94gwm"
Sep 05 07:49:47.572359 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.572277    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c91637a2-daa6-4523-8a9e-b2da9321546f-metrics-client-ca\") pod \"node-exporter-94gwm\" (UID: \"c91637a2-daa6-4523-8a9e-b2da9321546f\") " pod="openshift-monitoring/node-exporter-94gwm"
Sep 05 07:49:47.572550 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.572531    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c91637a2-daa6-4523-8a9e-b2da9321546f-node-exporter-accelerators-collector-config\") pod \"node-exporter-94gwm\" (UID: \"c91637a2-daa6-4523-8a9e-b2da9321546f\") " pod="openshift-monitoring/node-exporter-94gwm"
Sep 05 07:49:47.574614 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.574590    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c91637a2-daa6-4523-8a9e-b2da9321546f-node-exporter-tls\") pod \"node-exporter-94gwm\" (UID: \"c91637a2-daa6-4523-8a9e-b2da9321546f\") " pod="openshift-monitoring/node-exporter-94gwm"
Sep 05 07:49:47.574737 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.574705    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c91637a2-daa6-4523-8a9e-b2da9321546f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-94gwm\" (UID: \"c91637a2-daa6-4523-8a9e-b2da9321546f\") " pod="openshift-monitoring/node-exporter-94gwm"
Sep 05 07:49:47.580196 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.580173    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s89bx\" (UniqueName: \"kubernetes.io/projected/c91637a2-daa6-4523-8a9e-b2da9321546f-kube-api-access-s89bx\") pod \"node-exporter-94gwm\" (UID: \"c91637a2-daa6-4523-8a9e-b2da9321546f\") " pod="openshift-monitoring/node-exporter-94gwm"
Sep 05 07:49:47.601178 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.601143    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:47.606804 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.606780    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-94gwm"
Sep 05 07:49:47.703828 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.703781    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:47.926363 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:47.926282    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:48.183011 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:48.182936    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:48.266891 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:48.266862    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:48.580098 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:48.580067    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:49.095002 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:49.094970    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:49.195664 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:49.195613    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:49.627192 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:49.627159    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:49.727861 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:49.727812    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:50.140759 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:50.140723    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:50.234601 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:50.234553    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:50.325021 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:50.324987    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:50.474963 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:49:50.474929    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc91637a2_daa6_4523_8a9e_b2da9321546f.slice/crio-f60fd4198acd39af1389ef03b9a3cf63a4835a2959b29e944c22ead8c75c7309 WatchSource:0}: Error finding container f60fd4198acd39af1389ef03b9a3cf63a4835a2959b29e944c22ead8c75c7309: Status 404 returned error can't find the container with id f60fd4198acd39af1389ef03b9a3cf63a4835a2959b29e944c22ead8c75c7309
Sep 05 07:49:50.712139 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:50.712109    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:51.252032 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:51.251986    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-8p5k6" event={"ID":"c7f6c7f2-bb81-4da8-8ff6-2be8dae1554d","Type":"ContainerStarted","Data":"e714106e693f5accd488c09a72da91533d0dfb9cdc7f6ca7517175b0325b495a"}
Sep 05 07:49:51.253201 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:51.253176    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-8p5k6"
Sep 05 07:49:51.256303 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:51.256245    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-94gwm" event={"ID":"c91637a2-daa6-4523-8a9e-b2da9321546f","Type":"ContainerStarted","Data":"f60fd4198acd39af1389ef03b9a3cf63a4835a2959b29e944c22ead8c75c7309"}
Sep 05 07:49:51.264939 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:51.264881    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-8p5k6"
Sep 05 07:49:51.289697 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:51.289627    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-8p5k6" podStartSLOduration=1.571562192 podStartE2EDuration="19.289605783s" totalImagesPullingTime="17.718043591s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:49:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:51.271957144 +0000 UTC m=+94.042099501" watchObservedRunningTime="2026-09-05 07:49:51.289605783 +0000 UTC m=+94.059748123"
Sep 05 07:49:51.376667 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:51.376496    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:51.476427 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:51.476381    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:51.978681 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:51.978645    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:52.083567 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:52.083532    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:52.262581 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:52.262487    2563 generic.go:356] "Generic (PLEG): container finished" podID="c91637a2-daa6-4523-8a9e-b2da9321546f" containerID="922fe9dcc316fce75655a6cfe2d29d1d1a4d8e65b5185de233b7785c0a96b46c" exitCode=0
Sep 05 07:49:52.263057 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:52.262580    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-94gwm" event={"ID":"c91637a2-daa6-4523-8a9e-b2da9321546f","Type":"ContainerDied","Data":"922fe9dcc316fce75655a6cfe2d29d1d1a4d8e65b5185de233b7785c0a96b46c"}
Sep 05 07:49:52.411887 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:52.411853    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:53.271373 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:53.271334    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-94gwm" event={"ID":"c91637a2-daa6-4523-8a9e-b2da9321546f","Type":"ContainerStarted","Data":"662c78da3476c802a63863711e184eff062773f389916deaaae4fed85569a60a"}
Sep 05 07:49:53.271812 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:53.271394    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-94gwm" event={"ID":"c91637a2-daa6-4523-8a9e-b2da9321546f","Type":"ContainerStarted","Data":"6c77018b3ac85e41065a42e65269214caa97b4a7b7e5c5ba2f4a42aba7ebad5b"}
Sep 05 07:49:53.275790 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:53.275754    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:53.312358 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:53.312294    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-94gwm" podStartSLOduration=5.236400146 podStartE2EDuration="6.312270373s" totalImagesPullingTime="993.261831ms" totalInitContainerRuntime="82.608396ms" isStatefulPod=true podCreationTimestamp="2026-09-05 07:49:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:53.311039791 +0000 UTC m=+96.081182155" watchObservedRunningTime="2026-09-05 07:49:53.312270373 +0000 UTC m=+96.082412711"
Sep 05 07:49:53.398506 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:53.398477    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:53.516001 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:53.515964    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:54.174817 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:54.174786    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-768c75965-mrkgv"
Sep 05 07:49:54.318700 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:54.318663    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:54.420734 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:54.420702    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:54.519268 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:54.519183    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:55.496592 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:55.496547    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:55.598126 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:55.598093    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:57.337571 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:57.337526    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-79bddd854d-9vlsf" podUID="27817b6a-02ee-4fde-856f-221f422e231f" containerName="registry" containerID="cri-o://6b1be8a5f2907210f8047cc63ef7e2cfafd9721aad5d9d8676730b3d9e3c7e77" gracePeriod=30
Sep 05 07:49:57.381485 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:57.381391    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:57.446175 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:57.446144    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:57.643137 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:57.643102    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79bddd854d-9vlsf"
Sep 05 07:49:57.664878 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:57.663160    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/27817b6a-02ee-4fde-856f-221f422e231f-registry-tls\") pod \"27817b6a-02ee-4fde-856f-221f422e231f\" (UID: \"27817b6a-02ee-4fde-856f-221f422e231f\") "
Sep 05 07:49:57.664878 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:57.663205    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/27817b6a-02ee-4fde-856f-221f422e231f-image-registry-private-configuration\") pod \"27817b6a-02ee-4fde-856f-221f422e231f\" (UID: \"27817b6a-02ee-4fde-856f-221f422e231f\") "
Sep 05 07:49:57.664878 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:57.663248    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tb54t\" (UniqueName: \"kubernetes.io/projected/27817b6a-02ee-4fde-856f-221f422e231f-kube-api-access-tb54t\") pod \"27817b6a-02ee-4fde-856f-221f422e231f\" (UID: \"27817b6a-02ee-4fde-856f-221f422e231f\") "
Sep 05 07:49:57.664878 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:57.663276    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/27817b6a-02ee-4fde-856f-221f422e231f-ca-trust-extracted\") pod \"27817b6a-02ee-4fde-856f-221f422e231f\" (UID: \"27817b6a-02ee-4fde-856f-221f422e231f\") "
Sep 05 07:49:57.664878 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:57.663303    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/27817b6a-02ee-4fde-856f-221f422e231f-registry-certificates\") pod \"27817b6a-02ee-4fde-856f-221f422e231f\" (UID: \"27817b6a-02ee-4fde-856f-221f422e231f\") "
Sep 05 07:49:57.664878 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:57.663360    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/27817b6a-02ee-4fde-856f-221f422e231f-installation-pull-secrets\") pod \"27817b6a-02ee-4fde-856f-221f422e231f\" (UID: \"27817b6a-02ee-4fde-856f-221f422e231f\") "
Sep 05 07:49:57.664878 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:57.663393    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/27817b6a-02ee-4fde-856f-221f422e231f-bound-sa-token\") pod \"27817b6a-02ee-4fde-856f-221f422e231f\" (UID: \"27817b6a-02ee-4fde-856f-221f422e231f\") "
Sep 05 07:49:57.664878 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:57.663436    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/27817b6a-02ee-4fde-856f-221f422e231f-trusted-ca\") pod \"27817b6a-02ee-4fde-856f-221f422e231f\" (UID: \"27817b6a-02ee-4fde-856f-221f422e231f\") "
Sep 05 07:49:57.664878 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:57.664122    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27817b6a-02ee-4fde-856f-221f422e231f-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "27817b6a-02ee-4fde-856f-221f422e231f" (UID: "27817b6a-02ee-4fde-856f-221f422e231f"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 07:49:57.666533 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:57.666508    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27817b6a-02ee-4fde-856f-221f422e231f-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "27817b6a-02ee-4fde-856f-221f422e231f" (UID: "27817b6a-02ee-4fde-856f-221f422e231f"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 07:49:57.666646 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:57.666581    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27817b6a-02ee-4fde-856f-221f422e231f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "27817b6a-02ee-4fde-856f-221f422e231f" (UID: "27817b6a-02ee-4fde-856f-221f422e231f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 07:49:57.667314 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:57.667286    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27817b6a-02ee-4fde-856f-221f422e231f-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "27817b6a-02ee-4fde-856f-221f422e231f" (UID: "27817b6a-02ee-4fde-856f-221f422e231f"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 07:49:57.668239 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:57.668203    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27817b6a-02ee-4fde-856f-221f422e231f-kube-api-access-tb54t" (OuterVolumeSpecName: "kube-api-access-tb54t") pod "27817b6a-02ee-4fde-856f-221f422e231f" (UID: "27817b6a-02ee-4fde-856f-221f422e231f"). InnerVolumeSpecName "kube-api-access-tb54t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 07:49:57.669529 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:57.669493    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27817b6a-02ee-4fde-856f-221f422e231f-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "27817b6a-02ee-4fde-856f-221f422e231f" (UID: "27817b6a-02ee-4fde-856f-221f422e231f"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 07:49:57.670363 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:57.670328    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27817b6a-02ee-4fde-856f-221f422e231f-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "27817b6a-02ee-4fde-856f-221f422e231f" (UID: "27817b6a-02ee-4fde-856f-221f422e231f"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 07:49:57.678158 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:57.678128    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27817b6a-02ee-4fde-856f-221f422e231f-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "27817b6a-02ee-4fde-856f-221f422e231f" (UID: "27817b6a-02ee-4fde-856f-221f422e231f"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 07:49:57.764130 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:57.764093    2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/27817b6a-02ee-4fde-856f-221f422e231f-installation-pull-secrets\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 07:49:57.764130 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:57.764127    2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/27817b6a-02ee-4fde-856f-221f422e231f-bound-sa-token\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 07:49:57.764130 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:57.764136    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/27817b6a-02ee-4fde-856f-221f422e231f-trusted-ca\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 07:49:57.764343 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:57.764144    2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/27817b6a-02ee-4fde-856f-221f422e231f-registry-tls\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 07:49:57.764343 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:57.764153    2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/27817b6a-02ee-4fde-856f-221f422e231f-image-registry-private-configuration\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 07:49:57.764343 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:57.764163    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tb54t\" (UniqueName: \"kubernetes.io/projected/27817b6a-02ee-4fde-856f-221f422e231f-kube-api-access-tb54t\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 07:49:57.764343 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:57.764171    2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/27817b6a-02ee-4fde-856f-221f422e231f-ca-trust-extracted\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 07:49:57.764343 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:57.764182    2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/27817b6a-02ee-4fde-856f-221f422e231f-registry-certificates\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 07:49:57.951683 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:57.951598    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:58.009864 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:58.009812    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:49:58.293937 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:58.293832    2563 generic.go:356] "Generic (PLEG): container finished" podID="27817b6a-02ee-4fde-856f-221f422e231f" containerID="6b1be8a5f2907210f8047cc63ef7e2cfafd9721aad5d9d8676730b3d9e3c7e77" exitCode=0
Sep 05 07:49:58.293937 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:58.293883    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79bddd854d-9vlsf" event={"ID":"27817b6a-02ee-4fde-856f-221f422e231f","Type":"ContainerDied","Data":"6b1be8a5f2907210f8047cc63ef7e2cfafd9721aad5d9d8676730b3d9e3c7e77"}
Sep 05 07:49:58.293937 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:58.293920    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79bddd854d-9vlsf"
Sep 05 07:49:58.294264 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:58.293973    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79bddd854d-9vlsf" event={"ID":"27817b6a-02ee-4fde-856f-221f422e231f","Type":"ContainerDied","Data":"610e9ab40f52f64112cb0196304ab4cc1048b1a15faf6195453de75bc33bbcee"}
Sep 05 07:49:58.294264 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:58.294003    2563 scope.go:117] "RemoveContainer" containerID="6b1be8a5f2907210f8047cc63ef7e2cfafd9721aad5d9d8676730b3d9e3c7e77"
Sep 05 07:49:58.303579 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:58.303553    2563 scope.go:117] "RemoveContainer" containerID="6b1be8a5f2907210f8047cc63ef7e2cfafd9721aad5d9d8676730b3d9e3c7e77"
Sep 05 07:49:58.303914 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:49:58.303890    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b1be8a5f2907210f8047cc63ef7e2cfafd9721aad5d9d8676730b3d9e3c7e77\": container with ID starting with 6b1be8a5f2907210f8047cc63ef7e2cfafd9721aad5d9d8676730b3d9e3c7e77 not found: ID does not exist" containerID="6b1be8a5f2907210f8047cc63ef7e2cfafd9721aad5d9d8676730b3d9e3c7e77"
Sep 05 07:49:58.304009 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:58.303920    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b1be8a5f2907210f8047cc63ef7e2cfafd9721aad5d9d8676730b3d9e3c7e77"} err="failed to get container status \"6b1be8a5f2907210f8047cc63ef7e2cfafd9721aad5d9d8676730b3d9e3c7e77\": rpc error: code = NotFound desc = could not find container \"6b1be8a5f2907210f8047cc63ef7e2cfafd9721aad5d9d8676730b3d9e3c7e77\": container with ID starting with 6b1be8a5f2907210f8047cc63ef7e2cfafd9721aad5d9d8676730b3d9e3c7e77 not found: ID does not exist"
Sep 05 07:49:58.312153 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:58.312125    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-79bddd854d-9vlsf"]
Sep 05 07:49:58.317812 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:58.317784    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-79bddd854d-9vlsf"]
Sep 05 07:49:59.729489 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:49:59.729446    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27817b6a-02ee-4fde-856f-221f422e231f" path="/var/lib/kubelet/pods/27817b6a-02ee-4fde-856f-221f422e231f/volumes"
Sep 05 07:50:01.146784 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:50:01.146759    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-rbtht"
Sep 05 07:50:05.128106 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:50:05.128073    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:50:05.236198 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:50:05.236164    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:50:08.194900 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:50:08.194831    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:50:08.256754 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:50:08.256722    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:50:08.327562 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:50:08.327527    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:50:08.436190 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:50:08.436160    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:50:08.499680 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:50:08.499614    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:50:11.340281 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:50:11.340246    2563 generic.go:356] "Generic (PLEG): container finished" podID="143aee12-5c44-440c-ac06-1a5146e2d401" containerID="f6946f6074b8e080daae5deaee1088c62c29b1c2a996904c77f2661801c809ca" exitCode=0
Sep 05 07:50:11.340766 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:50:11.340290    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-rzj45" event={"ID":"143aee12-5c44-440c-ac06-1a5146e2d401","Type":"ContainerDied","Data":"f6946f6074b8e080daae5deaee1088c62c29b1c2a996904c77f2661801c809ca"}
Sep 05 07:50:11.340766 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:50:11.340606    2563 scope.go:117] "RemoveContainer" containerID="f6946f6074b8e080daae5deaee1088c62c29b1c2a996904c77f2661801c809ca"
Sep 05 07:50:12.345693 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:50:12.345658    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-rzj45" event={"ID":"143aee12-5c44-440c-ac06-1a5146e2d401","Type":"ContainerStarted","Data":"8f80411a5e7e863b7df3bf6a9b201487ed8357c62c304d6d43f8ac9a3340cf33"}
Sep 05 07:50:16.360041 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:50:16.360005    2563 generic.go:356] "Generic (PLEG): container finished" podID="999f5178-4c4b-4888-a348-c0d434b38967" containerID="bc822850751b2f130084d7cb31a3f7a57007a0ffe2e36e10f0753d96c98bdd56" exitCode=0
Sep 05 07:50:16.360413 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:50:16.360056    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-dcbfv" event={"ID":"999f5178-4c4b-4888-a348-c0d434b38967","Type":"ContainerDied","Data":"bc822850751b2f130084d7cb31a3f7a57007a0ffe2e36e10f0753d96c98bdd56"}
Sep 05 07:50:16.360475 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:50:16.360457    2563 scope.go:117] "RemoveContainer" containerID="bc822850751b2f130084d7cb31a3f7a57007a0ffe2e36e10f0753d96c98bdd56"
Sep 05 07:50:17.369826 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:50:17.369793    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-dcbfv" event={"ID":"999f5178-4c4b-4888-a348-c0d434b38967","Type":"ContainerStarted","Data":"4088b06ca6d94458c580924096ddff4bc195df9cbdd4d22897afbf5794db38d8"}
Sep 05 07:50:22.064233 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:50:22.064203    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:50:22.122469 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:50:22.122430    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:50:23.428551 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:50:23.428499    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:50:23.483642 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:50:23.483594    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:50:30.833608 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:50:30.833571    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:51:03.315069 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:51:03.315036    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:51:03.371068 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:51:03.371039    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:51:12.153747 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:51:12.153674    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:51:15.235348 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:51:15.235311    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:51:15.303804 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:51:15.303776    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:51:22.455456 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:51:22.455418    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:51:22.515011 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:51:22.514978    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:51:23.433982 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:51:23.433949    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:51:23.536060 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:51:23.536030    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:51:33.552216 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:51:33.552181    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:51:33.653248 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:51:33.653214    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:51:33.717752 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:51:33.717718    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:51:47.244591 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:51:47.244556    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:51:47.296810 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:51:47.296777    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:52:02.379946 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:02.379909    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:52:02.438756 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:02.438716    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:52:12.600013 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:12.599984    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:52:12.699826 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:12.699797    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:52:34.321721 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:34.321686    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:52:35.137397 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:35.137366    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:52:36.286633 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:36.286569    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8x9w8"]
Sep 05 07:52:36.288632 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:36.288079    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27817b6a-02ee-4fde-856f-221f422e231f" containerName="registry"
Sep 05 07:52:36.288632 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:36.288105    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="27817b6a-02ee-4fde-856f-221f422e231f" containerName="registry"
Sep 05 07:52:36.288632 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:36.288244    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="27817b6a-02ee-4fde-856f-221f422e231f" containerName="registry"
Sep 05 07:52:36.290889 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:36.290871    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8x9w8"
Sep 05 07:52:36.293099 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:36.293079    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 05 07:52:36.293197 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:36.293108    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 05 07:52:36.293197 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:36.293183    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-d8vxh\""
Sep 05 07:52:36.302274 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:36.302253    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8x9w8"]
Sep 05 07:52:36.449881 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:36.449836    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3be1b438-4c54-48fe-a54e-5015b6812e2c-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-8x9w8\" (UID: \"3be1b438-4c54-48fe-a54e-5015b6812e2c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8x9w8"
Sep 05 07:52:36.450045 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:36.449894    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hm7nv\" (UniqueName: \"kubernetes.io/projected/3be1b438-4c54-48fe-a54e-5015b6812e2c-kube-api-access-hm7nv\") pod \"cert-manager-operator-controller-manager-55c444bf86-8x9w8\" (UID: \"3be1b438-4c54-48fe-a54e-5015b6812e2c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8x9w8"
Sep 05 07:52:36.550666 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:36.550588    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3be1b438-4c54-48fe-a54e-5015b6812e2c-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-8x9w8\" (UID: \"3be1b438-4c54-48fe-a54e-5015b6812e2c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8x9w8"
Sep 05 07:52:36.550666 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:36.550638    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hm7nv\" (UniqueName: \"kubernetes.io/projected/3be1b438-4c54-48fe-a54e-5015b6812e2c-kube-api-access-hm7nv\") pod \"cert-manager-operator-controller-manager-55c444bf86-8x9w8\" (UID: \"3be1b438-4c54-48fe-a54e-5015b6812e2c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8x9w8"
Sep 05 07:52:36.550984 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:36.550965    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3be1b438-4c54-48fe-a54e-5015b6812e2c-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-8x9w8\" (UID: \"3be1b438-4c54-48fe-a54e-5015b6812e2c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8x9w8"
Sep 05 07:52:36.562039 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:36.562017    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hm7nv\" (UniqueName: \"kubernetes.io/projected/3be1b438-4c54-48fe-a54e-5015b6812e2c-kube-api-access-hm7nv\") pod \"cert-manager-operator-controller-manager-55c444bf86-8x9w8\" (UID: \"3be1b438-4c54-48fe-a54e-5015b6812e2c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8x9w8"
Sep 05 07:52:36.599946 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:36.599925    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8x9w8"
Sep 05 07:52:36.880007 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:36.879981    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8x9w8"]
Sep 05 07:52:36.882781 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:52:36.882757    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3be1b438_4c54_48fe_a54e_5015b6812e2c.slice/crio-7e23c4875b5071c84cc5f1b52b18040bc99f65819ce6f747a2769219e85149b3 WatchSource:0}: Error finding container 7e23c4875b5071c84cc5f1b52b18040bc99f65819ce6f747a2769219e85149b3: Status 404 returned error can't find the container with id 7e23c4875b5071c84cc5f1b52b18040bc99f65819ce6f747a2769219e85149b3
Sep 05 07:52:37.861476 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:37.861403    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8x9w8" event={"ID":"3be1b438-4c54-48fe-a54e-5015b6812e2c","Type":"ContainerStarted","Data":"7e23c4875b5071c84cc5f1b52b18040bc99f65819ce6f747a2769219e85149b3"}
Sep 05 07:52:37.948742 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:37.948704    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:52:38.048908 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:38.048877    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:52:39.871670 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:39.871633    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8x9w8" event={"ID":"3be1b438-4c54-48fe-a54e-5015b6812e2c","Type":"ContainerStarted","Data":"1b0c5fc687f7ac355742d781a26827a380c6f22f3f816703a0b1607866b1f47f"}
Sep 05 07:52:39.904448 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:39.904404    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8x9w8" podStartSLOduration=1.462644146 podStartE2EDuration="3.904383614s" totalImagesPullingTime="2.441739468s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:52:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:52:39.903632605 +0000 UTC m=+262.673774957" watchObservedRunningTime="2026-09-05 07:52:39.904383614 +0000 UTC m=+262.674525954"
Sep 05 07:52:40.190577 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:40.190508    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:52:40.293062 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:40.293031    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:52:40.415226 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:40.415191    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:52:42.588976 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:42.588948    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:52:42.775304 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:42.775268    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:52:44.930860 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:44.930820    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:52:44.996289 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:44.996259    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:52:49.776913 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:49.776882    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:52:51.417271 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:51.417234    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:52:51.475038 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:51.475011    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:52:52.632886 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:52.632837    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:52:52.733675 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:52.733647    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:52:54.453746 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:54.453705    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-c6dbv"]
Sep 05 07:52:54.457728 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:54.457703    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-c6dbv"
Sep 05 07:52:54.459808 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:54.459790    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 05 07:52:54.460373 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:54.460357    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 05 07:52:54.460502 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:54.460379    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-n7x6k\""
Sep 05 07:52:54.464790 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:54.464766    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-c6dbv"]
Sep 05 07:52:54.479893 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:54.479833    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/509d9d1e-d7cf-4477-9eb1-02281b7fd194-tmp\") pod \"openshift-lws-operator-fd8ccff4c-c6dbv\" (UID: \"509d9d1e-d7cf-4477-9eb1-02281b7fd194\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-c6dbv"
Sep 05 07:52:54.480021 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:54.479943    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crrj5\" (UniqueName: \"kubernetes.io/projected/509d9d1e-d7cf-4477-9eb1-02281b7fd194-kube-api-access-crrj5\") pod \"openshift-lws-operator-fd8ccff4c-c6dbv\" (UID: \"509d9d1e-d7cf-4477-9eb1-02281b7fd194\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-c6dbv"
Sep 05 07:52:54.581135 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:54.581098    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crrj5\" (UniqueName: \"kubernetes.io/projected/509d9d1e-d7cf-4477-9eb1-02281b7fd194-kube-api-access-crrj5\") pod \"openshift-lws-operator-fd8ccff4c-c6dbv\" (UID: \"509d9d1e-d7cf-4477-9eb1-02281b7fd194\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-c6dbv"
Sep 05 07:52:54.581307 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:54.581144    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/509d9d1e-d7cf-4477-9eb1-02281b7fd194-tmp\") pod \"openshift-lws-operator-fd8ccff4c-c6dbv\" (UID: \"509d9d1e-d7cf-4477-9eb1-02281b7fd194\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-c6dbv"
Sep 05 07:52:54.581497 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:54.581482    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/509d9d1e-d7cf-4477-9eb1-02281b7fd194-tmp\") pod \"openshift-lws-operator-fd8ccff4c-c6dbv\" (UID: \"509d9d1e-d7cf-4477-9eb1-02281b7fd194\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-c6dbv"
Sep 05 07:52:54.589102 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:54.589073    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crrj5\" (UniqueName: \"kubernetes.io/projected/509d9d1e-d7cf-4477-9eb1-02281b7fd194-kube-api-access-crrj5\") pod \"openshift-lws-operator-fd8ccff4c-c6dbv\" (UID: \"509d9d1e-d7cf-4477-9eb1-02281b7fd194\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-c6dbv"
Sep 05 07:52:54.766889 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:54.766791    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-c6dbv"
Sep 05 07:52:55.034157 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:55.034083    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-c6dbv"]
Sep 05 07:52:55.037280 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:52:55.037253    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod509d9d1e_d7cf_4477_9eb1_02281b7fd194.slice/crio-e029ca7e8418086374c87904a951a192a8d3cb0f59baaac089d0d945c7d23c6c WatchSource:0}: Error finding container e029ca7e8418086374c87904a951a192a8d3cb0f59baaac089d0d945c7d23c6c: Status 404 returned error can't find the container with id e029ca7e8418086374c87904a951a192a8d3cb0f59baaac089d0d945c7d23c6c
Sep 05 07:52:55.934321 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:55.934276    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-c6dbv" event={"ID":"509d9d1e-d7cf-4477-9eb1-02281b7fd194","Type":"ContainerStarted","Data":"e029ca7e8418086374c87904a951a192a8d3cb0f59baaac089d0d945c7d23c6c"}
Sep 05 07:52:57.945280 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:57.945245    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-c6dbv" event={"ID":"509d9d1e-d7cf-4477-9eb1-02281b7fd194","Type":"ContainerStarted","Data":"2c2c785f38b4ba4eadf4649433937b9e443d6ac48d302ef6ab9d6d2c6233931a"}
Sep 05 07:52:57.962909 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:57.962862    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-c6dbv" podStartSLOduration=1.459232313 podStartE2EDuration="3.962834627s" totalImagesPullingTime="2.503602314s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:52:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:52:57.962468697 +0000 UTC m=+280.732611057" watchObservedRunningTime="2026-09-05 07:52:57.962834627 +0000 UTC m=+280.732976963"
Sep 05 07:52:58.661991 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:58.661954    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:52:58.763171 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:52:58.763143    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:53:00.489393 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:00.489360    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:53:16.588902 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:16.588868    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:53:16.703750 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:16.703714    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:53:17.395780 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:17.395750    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:53:17.645234 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:17.645192    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2bzp8_569cffae-1c5c-4b02-af57-9ca53c455e4c/ovn-acl-logging/0.log"
Sep 05 07:53:17.649588 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:17.649530    2563 kubelet.go:1648] "Image garbage collection succeeded"
Sep 05 07:53:17.649877 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:17.649862    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2bzp8_569cffae-1c5c-4b02-af57-9ca53c455e4c/ovn-acl-logging/0.log"
Sep 05 07:53:17.670754 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:17.670723    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:53:17.749984 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:17.749804    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:53:17.828586 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:17.828561    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-lqkkk"]
Sep 05 07:53:17.831727 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:17.831711    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-lqkkk"
Sep 05 07:53:17.833975 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:17.833950    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 05 07:53:17.833975 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:17.833955    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 05 07:53:17.834387 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:17.834374    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-mb5bz\""
Sep 05 07:53:17.834633 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:17.834619    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 05 07:53:17.834683 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:17.834674    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 05 07:53:17.864570 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:17.864543    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-lqkkk"]
Sep 05 07:53:17.958752 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:17.958670    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/56de4a10-282d-48e5-b922-9e35da8fb5e2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9fc4f56d7-lqkkk\" (UID: \"56de4a10-282d-48e5-b922-9e35da8fb5e2\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-lqkkk"
Sep 05 07:53:17.958752 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:17.958719    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/56de4a10-282d-48e5-b922-9e35da8fb5e2-tmp\") pod \"opendatahub-operator-controller-manager-9fc4f56d7-lqkkk\" (UID: \"56de4a10-282d-48e5-b922-9e35da8fb5e2\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-lqkkk"
Sep 05 07:53:17.958968 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:17.958786    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/56de4a10-282d-48e5-b922-9e35da8fb5e2-manifests\") pod \"opendatahub-operator-controller-manager-9fc4f56d7-lqkkk\" (UID: \"56de4a10-282d-48e5-b922-9e35da8fb5e2\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-lqkkk"
Sep 05 07:53:17.958968 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:17.958830    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/56de4a10-282d-48e5-b922-9e35da8fb5e2-webhook-cert\") pod \"opendatahub-operator-controller-manager-9fc4f56d7-lqkkk\" (UID: \"56de4a10-282d-48e5-b922-9e35da8fb5e2\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-lqkkk"
Sep 05 07:53:17.958968 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:17.958891    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blcs8\" (UniqueName: \"kubernetes.io/projected/56de4a10-282d-48e5-b922-9e35da8fb5e2-kube-api-access-blcs8\") pod \"opendatahub-operator-controller-manager-9fc4f56d7-lqkkk\" (UID: \"56de4a10-282d-48e5-b922-9e35da8fb5e2\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-lqkkk"
Sep 05 07:53:17.964397 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:17.964373    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:53:18.060134 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:18.060104    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/56de4a10-282d-48e5-b922-9e35da8fb5e2-manifests\") pod \"opendatahub-operator-controller-manager-9fc4f56d7-lqkkk\" (UID: \"56de4a10-282d-48e5-b922-9e35da8fb5e2\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-lqkkk"
Sep 05 07:53:18.060274 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:18.060148    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/56de4a10-282d-48e5-b922-9e35da8fb5e2-webhook-cert\") pod \"opendatahub-operator-controller-manager-9fc4f56d7-lqkkk\" (UID: \"56de4a10-282d-48e5-b922-9e35da8fb5e2\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-lqkkk"
Sep 05 07:53:18.060274 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:18.060196    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blcs8\" (UniqueName: \"kubernetes.io/projected/56de4a10-282d-48e5-b922-9e35da8fb5e2-kube-api-access-blcs8\") pod \"opendatahub-operator-controller-manager-9fc4f56d7-lqkkk\" (UID: \"56de4a10-282d-48e5-b922-9e35da8fb5e2\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-lqkkk"
Sep 05 07:53:18.060274 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:18.060219    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/56de4a10-282d-48e5-b922-9e35da8fb5e2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9fc4f56d7-lqkkk\" (UID: \"56de4a10-282d-48e5-b922-9e35da8fb5e2\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-lqkkk"
Sep 05 07:53:18.060274 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:18.060254    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/56de4a10-282d-48e5-b922-9e35da8fb5e2-tmp\") pod \"opendatahub-operator-controller-manager-9fc4f56d7-lqkkk\" (UID: \"56de4a10-282d-48e5-b922-9e35da8fb5e2\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-lqkkk"
Sep 05 07:53:18.060510 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:18.060449    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/56de4a10-282d-48e5-b922-9e35da8fb5e2-manifests\") pod \"opendatahub-operator-controller-manager-9fc4f56d7-lqkkk\" (UID: \"56de4a10-282d-48e5-b922-9e35da8fb5e2\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-lqkkk"
Sep 05 07:53:18.062486 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:18.062457    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/56de4a10-282d-48e5-b922-9e35da8fb5e2-tmp\") pod \"opendatahub-operator-controller-manager-9fc4f56d7-lqkkk\" (UID: \"56de4a10-282d-48e5-b922-9e35da8fb5e2\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-lqkkk"
Sep 05 07:53:18.063181 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:18.063163    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/56de4a10-282d-48e5-b922-9e35da8fb5e2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9fc4f56d7-lqkkk\" (UID: \"56de4a10-282d-48e5-b922-9e35da8fb5e2\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-lqkkk"
Sep 05 07:53:18.063239 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:18.063222    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/56de4a10-282d-48e5-b922-9e35da8fb5e2-webhook-cert\") pod \"opendatahub-operator-controller-manager-9fc4f56d7-lqkkk\" (UID: \"56de4a10-282d-48e5-b922-9e35da8fb5e2\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-lqkkk"
Sep 05 07:53:18.065799 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:18.065779    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:53:18.067982 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:18.067964    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blcs8\" (UniqueName: \"kubernetes.io/projected/56de4a10-282d-48e5-b922-9e35da8fb5e2-kube-api-access-blcs8\") pod \"opendatahub-operator-controller-manager-9fc4f56d7-lqkkk\" (UID: \"56de4a10-282d-48e5-b922-9e35da8fb5e2\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-lqkkk"
Sep 05 07:53:18.142157 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:18.142129    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-lqkkk"
Sep 05 07:53:18.164918 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:18.164892    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:53:18.272558 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:18.272468    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:53:18.378615 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:18.378582    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:53:18.432745 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:18.432713    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-lqkkk"]
Sep 05 07:53:18.436209 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:53:18.436184    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56de4a10_282d_48e5_b922_9e35da8fb5e2.slice/crio-d738e46cfce71ecc73dd91c8777e74f8083dc341ee3a63ebde442b2c6c3f8b70 WatchSource:0}: Error finding container d738e46cfce71ecc73dd91c8777e74f8083dc341ee3a63ebde442b2c6c3f8b70: Status 404 returned error can't find the container with id d738e46cfce71ecc73dd91c8777e74f8083dc341ee3a63ebde442b2c6c3f8b70
Sep 05 07:53:19.031285 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:19.031253    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-lqkkk" event={"ID":"56de4a10-282d-48e5-b922-9e35da8fb5e2","Type":"ContainerStarted","Data":"d738e46cfce71ecc73dd91c8777e74f8083dc341ee3a63ebde442b2c6c3f8b70"}
Sep 05 07:53:22.047293 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:22.047256    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-lqkkk" event={"ID":"56de4a10-282d-48e5-b922-9e35da8fb5e2","Type":"ContainerStarted","Data":"01e49d8410b7bc107f4ea5ab089df076ab93d9f020235b28465c4159ffb45114"}
Sep 05 07:53:23.054967 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:23.054938    2563 generic.go:356] "Generic (PLEG): container finished" podID="56de4a10-282d-48e5-b922-9e35da8fb5e2" containerID="01e49d8410b7bc107f4ea5ab089df076ab93d9f020235b28465c4159ffb45114" exitCode=0
Sep 05 07:53:23.055317 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:23.055019    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-lqkkk" event={"ID":"56de4a10-282d-48e5-b922-9e35da8fb5e2","Type":"ContainerDied","Data":"01e49d8410b7bc107f4ea5ab089df076ab93d9f020235b28465c4159ffb45114"}
Sep 05 07:53:23.055488 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:23.055474    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 05 07:53:24.061549 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:24.061508    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-lqkkk" event={"ID":"56de4a10-282d-48e5-b922-9e35da8fb5e2","Type":"ContainerStarted","Data":"3c6bf6f8518f2de4105a903e5d7cde4d9062b40e0ba6b7b1fe523f850c03e0cd"}
Sep 05 07:53:24.061921 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:24.061642    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-lqkkk"
Sep 05 07:53:24.082710 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:24.082661    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-lqkkk" podStartSLOduration=2.660796208 podStartE2EDuration="7.082645038s" totalImagesPullingTime="3.351536922s" totalInitContainerRuntime="1.070311908s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:53:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:53:24.081269293 +0000 UTC m=+306.851411652" watchObservedRunningTime="2026-09-05 07:53:24.082645038 +0000 UTC m=+306.852787450"
Sep 05 07:53:24.733995 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:24.733962    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:53:26.160721 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:26.160687    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:53:29.990739 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:29.990694    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:53:30.097937 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:30.097902    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:53:35.068597 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:35.068564    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-lqkkk"
Sep 05 07:53:37.997385 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:37.997357    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:53:38.160143 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:38.160111    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:53:38.411192 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:38.411160    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:53:38.465862 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:38.465809    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-74c76cbcf8-f2jsh"]
Sep 05 07:53:38.470124 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:38.470094    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-74c76cbcf8-f2jsh"
Sep 05 07:53:38.472164 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:38.472142    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 05 07:53:38.473057 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:38.473038    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 05 07:53:38.473145 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:38.473048    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-7jsrr\""
Sep 05 07:53:38.485707 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:38.485685    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-74c76cbcf8-f2jsh"]
Sep 05 07:53:38.529185 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:38.529139    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:53:38.629021 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:38.628987    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6wqp\" (UniqueName: \"kubernetes.io/projected/f7d4a4c9-3912-4cff-b5e2-a06d853c81ca-kube-api-access-j6wqp\") pod \"kube-auth-proxy-74c76cbcf8-f2jsh\" (UID: \"f7d4a4c9-3912-4cff-b5e2-a06d853c81ca\") " pod="openshift-ingress/kube-auth-proxy-74c76cbcf8-f2jsh"
Sep 05 07:53:38.629021 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:38.629025    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7d4a4c9-3912-4cff-b5e2-a06d853c81ca-tls-certs\") pod \"kube-auth-proxy-74c76cbcf8-f2jsh\" (UID: \"f7d4a4c9-3912-4cff-b5e2-a06d853c81ca\") " pod="openshift-ingress/kube-auth-proxy-74c76cbcf8-f2jsh"
Sep 05 07:53:38.629233 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:38.629055    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f7d4a4c9-3912-4cff-b5e2-a06d853c81ca-tmp\") pod \"kube-auth-proxy-74c76cbcf8-f2jsh\" (UID: \"f7d4a4c9-3912-4cff-b5e2-a06d853c81ca\") " pod="openshift-ingress/kube-auth-proxy-74c76cbcf8-f2jsh"
Sep 05 07:53:38.729641 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:38.729551    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f7d4a4c9-3912-4cff-b5e2-a06d853c81ca-tmp\") pod \"kube-auth-proxy-74c76cbcf8-f2jsh\" (UID: \"f7d4a4c9-3912-4cff-b5e2-a06d853c81ca\") " pod="openshift-ingress/kube-auth-proxy-74c76cbcf8-f2jsh"
Sep 05 07:53:38.729641 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:38.729624    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6wqp\" (UniqueName: \"kubernetes.io/projected/f7d4a4c9-3912-4cff-b5e2-a06d853c81ca-kube-api-access-j6wqp\") pod \"kube-auth-proxy-74c76cbcf8-f2jsh\" (UID: \"f7d4a4c9-3912-4cff-b5e2-a06d853c81ca\") " pod="openshift-ingress/kube-auth-proxy-74c76cbcf8-f2jsh"
Sep 05 07:53:38.729887 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:38.729718    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7d4a4c9-3912-4cff-b5e2-a06d853c81ca-tls-certs\") pod \"kube-auth-proxy-74c76cbcf8-f2jsh\" (UID: \"f7d4a4c9-3912-4cff-b5e2-a06d853c81ca\") " pod="openshift-ingress/kube-auth-proxy-74c76cbcf8-f2jsh"
Sep 05 07:53:38.731883 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:38.731834    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f7d4a4c9-3912-4cff-b5e2-a06d853c81ca-tmp\") pod \"kube-auth-proxy-74c76cbcf8-f2jsh\" (UID: \"f7d4a4c9-3912-4cff-b5e2-a06d853c81ca\") " pod="openshift-ingress/kube-auth-proxy-74c76cbcf8-f2jsh"
Sep 05 07:53:38.732141 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:38.732122    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7d4a4c9-3912-4cff-b5e2-a06d853c81ca-tls-certs\") pod \"kube-auth-proxy-74c76cbcf8-f2jsh\" (UID: \"f7d4a4c9-3912-4cff-b5e2-a06d853c81ca\") " pod="openshift-ingress/kube-auth-proxy-74c76cbcf8-f2jsh"
Sep 05 07:53:38.784740 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:38.784705    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6wqp\" (UniqueName: \"kubernetes.io/projected/f7d4a4c9-3912-4cff-b5e2-a06d853c81ca-kube-api-access-j6wqp\") pod \"kube-auth-proxy-74c76cbcf8-f2jsh\" (UID: \"f7d4a4c9-3912-4cff-b5e2-a06d853c81ca\") " pod="openshift-ingress/kube-auth-proxy-74c76cbcf8-f2jsh"
Sep 05 07:53:39.080468 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:39.080432    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-74c76cbcf8-f2jsh"
Sep 05 07:53:39.373804 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:39.373724    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-74c76cbcf8-f2jsh"]
Sep 05 07:53:39.376563 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:53:39.376524    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7d4a4c9_3912_4cff_b5e2_a06d853c81ca.slice/crio-95f60b2598c78251495512e4ae67c725ae037304f9a1e27eec5229a274332703 WatchSource:0}: Error finding container 95f60b2598c78251495512e4ae67c725ae037304f9a1e27eec5229a274332703: Status 404 returned error can't find the container with id 95f60b2598c78251495512e4ae67c725ae037304f9a1e27eec5229a274332703
Sep 05 07:53:39.835040 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:39.835012    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:53:39.934344 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:39.934285    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:53:40.128560 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:40.128451    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-74c76cbcf8-f2jsh" event={"ID":"f7d4a4c9-3912-4cff-b5e2-a06d853c81ca","Type":"ContainerStarted","Data":"95f60b2598c78251495512e4ae67c725ae037304f9a1e27eec5229a274332703"}
Sep 05 07:53:40.757978 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:40.757888    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:53:42.138013 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:42.137979    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-74c76cbcf8-f2jsh" event={"ID":"f7d4a4c9-3912-4cff-b5e2-a06d853c81ca","Type":"ContainerStarted","Data":"1126c776b89048a2d749beb4761867bc1e3e6dea99a4c60bf6ba7dd7d7f2024d"}
Sep 05 07:53:42.153424 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:42.153382    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-74c76cbcf8-f2jsh" podStartSLOduration=1.566646902 podStartE2EDuration="4.153366871s" totalImagesPullingTime="2.586719969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:53:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:53:42.152295647 +0000 UTC m=+324.922437998" watchObservedRunningTime="2026-09-05 07:53:42.153366871 +0000 UTC m=+324.923509208"
Sep 05 07:53:45.189534 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:45.189475    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:53:45.290705 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:45.290670    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:53:45.850644 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:45.850610    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw"]
Sep 05 07:53:45.854043 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:45.854014    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw"
Sep 05 07:53:45.856111 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:45.856087    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 05 07:53:45.856250 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:45.856130    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-969j7\""
Sep 05 07:53:45.865612 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:45.865593    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw"]
Sep 05 07:53:45.881960 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:45.881934    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:53:45.993137 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:45.993102    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/baa0f853-9766-4839-abc2-6da72b4ea0f0-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw\" (UID: \"baa0f853-9766-4839-abc2-6da72b4ea0f0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw"
Sep 05 07:53:45.993297 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:45.993152    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/baa0f853-9766-4839-abc2-6da72b4ea0f0-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw\" (UID: \"baa0f853-9766-4839-abc2-6da72b4ea0f0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw"
Sep 05 07:53:45.993297 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:45.993210    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwnhd\" (UniqueName: \"kubernetes.io/projected/baa0f853-9766-4839-abc2-6da72b4ea0f0-kube-api-access-mwnhd\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw\" (UID: \"baa0f853-9766-4839-abc2-6da72b4ea0f0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw"
Sep 05 07:53:45.993297 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:45.993237    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/baa0f853-9766-4839-abc2-6da72b4ea0f0-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw\" (UID: \"baa0f853-9766-4839-abc2-6da72b4ea0f0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw"
Sep 05 07:53:45.993297 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:45.993263    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/baa0f853-9766-4839-abc2-6da72b4ea0f0-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw\" (UID: \"baa0f853-9766-4839-abc2-6da72b4ea0f0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw"
Sep 05 07:53:45.993297 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:45.993285    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/baa0f853-9766-4839-abc2-6da72b4ea0f0-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw\" (UID: \"baa0f853-9766-4839-abc2-6da72b4ea0f0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw"
Sep 05 07:53:45.993472 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:45.993309    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/baa0f853-9766-4839-abc2-6da72b4ea0f0-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw\" (UID: \"baa0f853-9766-4839-abc2-6da72b4ea0f0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw"
Sep 05 07:53:45.993472 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:45.993396    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/baa0f853-9766-4839-abc2-6da72b4ea0f0-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw\" (UID: \"baa0f853-9766-4839-abc2-6da72b4ea0f0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw"
Sep 05 07:53:45.993472 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:45.993431    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/baa0f853-9766-4839-abc2-6da72b4ea0f0-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw\" (UID: \"baa0f853-9766-4839-abc2-6da72b4ea0f0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw"
Sep 05 07:53:46.094510 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:46.094476    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/baa0f853-9766-4839-abc2-6da72b4ea0f0-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw\" (UID: \"baa0f853-9766-4839-abc2-6da72b4ea0f0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw"
Sep 05 07:53:46.094667 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:46.094522    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/baa0f853-9766-4839-abc2-6da72b4ea0f0-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw\" (UID: \"baa0f853-9766-4839-abc2-6da72b4ea0f0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw"
Sep 05 07:53:46.094667 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:46.094637    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/baa0f853-9766-4839-abc2-6da72b4ea0f0-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw\" (UID: \"baa0f853-9766-4839-abc2-6da72b4ea0f0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw"
Sep 05 07:53:46.094738 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:46.094693    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/baa0f853-9766-4839-abc2-6da72b4ea0f0-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw\" (UID: \"baa0f853-9766-4839-abc2-6da72b4ea0f0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw"
Sep 05 07:53:46.094738 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:46.094732    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/baa0f853-9766-4839-abc2-6da72b4ea0f0-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw\" (UID: \"baa0f853-9766-4839-abc2-6da72b4ea0f0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw"
Sep 05 07:53:46.094813 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:46.094763    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwnhd\" (UniqueName: \"kubernetes.io/projected/baa0f853-9766-4839-abc2-6da72b4ea0f0-kube-api-access-mwnhd\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw\" (UID: \"baa0f853-9766-4839-abc2-6da72b4ea0f0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw"
Sep 05 07:53:46.094813 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:46.094800    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/baa0f853-9766-4839-abc2-6da72b4ea0f0-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw\" (UID: \"baa0f853-9766-4839-abc2-6da72b4ea0f0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw"
Sep 05 07:53:46.094944 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:46.094825    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/baa0f853-9766-4839-abc2-6da72b4ea0f0-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw\" (UID: \"baa0f853-9766-4839-abc2-6da72b4ea0f0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw"
Sep 05 07:53:46.094944 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:46.094879    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/baa0f853-9766-4839-abc2-6da72b4ea0f0-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw\" (UID: \"baa0f853-9766-4839-abc2-6da72b4ea0f0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw"
Sep 05 07:53:46.094944 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:46.094919    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/baa0f853-9766-4839-abc2-6da72b4ea0f0-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw\" (UID: \"baa0f853-9766-4839-abc2-6da72b4ea0f0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw"
Sep 05 07:53:46.095089 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:46.094973    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/baa0f853-9766-4839-abc2-6da72b4ea0f0-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw\" (UID: \"baa0f853-9766-4839-abc2-6da72b4ea0f0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw"
Sep 05 07:53:46.095146 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:46.095092    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/baa0f853-9766-4839-abc2-6da72b4ea0f0-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw\" (UID: \"baa0f853-9766-4839-abc2-6da72b4ea0f0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw"
Sep 05 07:53:46.095271 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:46.095249    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/baa0f853-9766-4839-abc2-6da72b4ea0f0-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw\" (UID: \"baa0f853-9766-4839-abc2-6da72b4ea0f0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw"
Sep 05 07:53:46.095545 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:46.095523    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/baa0f853-9766-4839-abc2-6da72b4ea0f0-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw\" (UID: \"baa0f853-9766-4839-abc2-6da72b4ea0f0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw"
Sep 05 07:53:46.097331 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:46.097311    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/baa0f853-9766-4839-abc2-6da72b4ea0f0-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw\" (UID: \"baa0f853-9766-4839-abc2-6da72b4ea0f0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw"
Sep 05 07:53:46.097416 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:46.097309    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/baa0f853-9766-4839-abc2-6da72b4ea0f0-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw\" (UID: \"baa0f853-9766-4839-abc2-6da72b4ea0f0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw"
Sep 05 07:53:46.107806 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:46.107749    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:53:46.110575 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:46.110551    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/baa0f853-9766-4839-abc2-6da72b4ea0f0-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw\" (UID: \"baa0f853-9766-4839-abc2-6da72b4ea0f0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw"
Sep 05 07:53:46.110575 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:46.110571    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwnhd\" (UniqueName: \"kubernetes.io/projected/baa0f853-9766-4839-abc2-6da72b4ea0f0-kube-api-access-mwnhd\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw\" (UID: \"baa0f853-9766-4839-abc2-6da72b4ea0f0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw"
Sep 05 07:53:46.164163 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:46.164134    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw"
Sep 05 07:53:46.200210 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:46.200179    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:53:46.439084 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:46.439061    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw"]
Sep 05 07:53:46.441319 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:53:46.441290    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaa0f853_9766_4839_abc2_6da72b4ea0f0.slice/crio-af51a5d88ae36f7a679472630c93357ab2b7576ee839a218fd5f4b196f1818d2 WatchSource:0}: Error finding container af51a5d88ae36f7a679472630c93357ab2b7576ee839a218fd5f4b196f1818d2: Status 404 returned error can't find the container with id af51a5d88ae36f7a679472630c93357ab2b7576ee839a218fd5f4b196f1818d2
Sep 05 07:53:47.160076 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:47.160044    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw" event={"ID":"baa0f853-9766-4839-abc2-6da72b4ea0f0","Type":"ContainerStarted","Data":"af51a5d88ae36f7a679472630c93357ab2b7576ee839a218fd5f4b196f1818d2"}
Sep 05 07:53:49.149137 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:49.149103    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-bdp2h"]
Sep 05 07:53:49.159167 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:49.159145    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-bdp2h"
Sep 05 07:53:49.161459 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:49.161437    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 05 07:53:49.161592 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:49.161484    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-vm4g4\""
Sep 05 07:53:49.173369 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:49.173340    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-bdp2h"]
Sep 05 07:53:49.194859 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:49.194811    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:53:49.218351 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:49.218316    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6td5\" (UniqueName: \"kubernetes.io/projected/ced931cd-0339-489d-b218-01ac7471b5f1-kube-api-access-k6td5\") pod \"odh-model-controller-7998c67dc4-bdp2h\" (UID: \"ced931cd-0339-489d-b218-01ac7471b5f1\") " pod="opendatahub/odh-model-controller-7998c67dc4-bdp2h"
Sep 05 07:53:49.218493 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:49.218421    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ced931cd-0339-489d-b218-01ac7471b5f1-cert\") pod \"odh-model-controller-7998c67dc4-bdp2h\" (UID: \"ced931cd-0339-489d-b218-01ac7471b5f1\") " pod="opendatahub/odh-model-controller-7998c67dc4-bdp2h"
Sep 05 07:53:49.290437 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:49.290406    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:53:49.317135 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:49.317099    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 05 07:53:49.317234 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:49.317168    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 05 07:53:49.317234 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:49.317198    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 05 07:53:49.318948 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:49.318915    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6td5\" (UniqueName: \"kubernetes.io/projected/ced931cd-0339-489d-b218-01ac7471b5f1-kube-api-access-k6td5\") pod \"odh-model-controller-7998c67dc4-bdp2h\" (UID: \"ced931cd-0339-489d-b218-01ac7471b5f1\") " pod="opendatahub/odh-model-controller-7998c67dc4-bdp2h"
Sep 05 07:53:49.319071 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:49.318967    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ced931cd-0339-489d-b218-01ac7471b5f1-cert\") pod \"odh-model-controller-7998c67dc4-bdp2h\" (UID: \"ced931cd-0339-489d-b218-01ac7471b5f1\") " pod="opendatahub/odh-model-controller-7998c67dc4-bdp2h"
Sep 05 07:53:49.321409 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:49.321384    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ced931cd-0339-489d-b218-01ac7471b5f1-cert\") pod \"odh-model-controller-7998c67dc4-bdp2h\" (UID: \"ced931cd-0339-489d-b218-01ac7471b5f1\") " pod="opendatahub/odh-model-controller-7998c67dc4-bdp2h"
Sep 05 07:53:49.329451 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:49.329428    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6td5\" (UniqueName: \"kubernetes.io/projected/ced931cd-0339-489d-b218-01ac7471b5f1-kube-api-access-k6td5\") pod \"odh-model-controller-7998c67dc4-bdp2h\" (UID: \"ced931cd-0339-489d-b218-01ac7471b5f1\") " pod="opendatahub/odh-model-controller-7998c67dc4-bdp2h"
Sep 05 07:53:49.391345 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:49.391321    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:53:49.471830 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:49.471758    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-bdp2h"
Sep 05 07:53:49.688061 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:49.688033    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-mq4pt"]
Sep 05 07:53:49.692354 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:49.692336    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-mq4pt"
Sep 05 07:53:49.695562 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:49.695532    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 05 07:53:49.703185 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:49.703154    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-mq4pt"]
Sep 05 07:53:49.703453 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:49.703428    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:53:49.721780 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:49.721756    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6g68\" (UniqueName: \"kubernetes.io/projected/9b4d9b90-5a4e-491b-ab28-884452aac755-kube-api-access-n6g68\") pod \"model-serving-api-54d89bd5fc-mq4pt\" (UID: \"9b4d9b90-5a4e-491b-ab28-884452aac755\") " pod="opendatahub/model-serving-api-54d89bd5fc-mq4pt"
Sep 05 07:53:49.722016 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:49.721799    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b4d9b90-5a4e-491b-ab28-884452aac755-tls-certs\") pod \"model-serving-api-54d89bd5fc-mq4pt\" (UID: \"9b4d9b90-5a4e-491b-ab28-884452aac755\") " pod="opendatahub/model-serving-api-54d89bd5fc-mq4pt"
Sep 05 07:53:49.753462 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:49.753425    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-bdp2h"]
Sep 05 07:53:49.756234 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:53:49.756205    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podced931cd_0339_489d_b218_01ac7471b5f1.slice/crio-88cac3db1a98937506622746d5ba84115a734e883d6a825c55065a23744eac56 WatchSource:0}: Error finding container 88cac3db1a98937506622746d5ba84115a734e883d6a825c55065a23744eac56: Status 404 returned error can't find the container with id 88cac3db1a98937506622746d5ba84115a734e883d6a825c55065a23744eac56
Sep 05 07:53:49.804098 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:49.804049    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:53:49.822346 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:49.822315    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6g68\" (UniqueName: \"kubernetes.io/projected/9b4d9b90-5a4e-491b-ab28-884452aac755-kube-api-access-n6g68\") pod \"model-serving-api-54d89bd5fc-mq4pt\" (UID: \"9b4d9b90-5a4e-491b-ab28-884452aac755\") " pod="opendatahub/model-serving-api-54d89bd5fc-mq4pt"
Sep 05 07:53:49.822490 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:49.822469    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b4d9b90-5a4e-491b-ab28-884452aac755-tls-certs\") pod \"model-serving-api-54d89bd5fc-mq4pt\" (UID: \"9b4d9b90-5a4e-491b-ab28-884452aac755\") " pod="opendatahub/model-serving-api-54d89bd5fc-mq4pt"
Sep 05 07:53:49.824991 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:49.824975    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b4d9b90-5a4e-491b-ab28-884452aac755-tls-certs\") pod \"model-serving-api-54d89bd5fc-mq4pt\" (UID: \"9b4d9b90-5a4e-491b-ab28-884452aac755\") " pod="opendatahub/model-serving-api-54d89bd5fc-mq4pt"
Sep 05 07:53:49.830218 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:49.830188    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6g68\" (UniqueName: \"kubernetes.io/projected/9b4d9b90-5a4e-491b-ab28-884452aac755-kube-api-access-n6g68\") pod \"model-serving-api-54d89bd5fc-mq4pt\" (UID: \"9b4d9b90-5a4e-491b-ab28-884452aac755\") " pod="opendatahub/model-serving-api-54d89bd5fc-mq4pt"
Sep 05 07:53:50.003220 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:50.003132    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-mq4pt"
Sep 05 07:53:50.175862 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:50.175807    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw" event={"ID":"baa0f853-9766-4839-abc2-6da72b4ea0f0","Type":"ContainerStarted","Data":"8b6fb88e2fef61ca223ac5ad4a631382af5c8ce09766816320138905751e8696"}
Sep 05 07:53:50.177178 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:50.177154    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-bdp2h" event={"ID":"ced931cd-0339-489d-b218-01ac7471b5f1","Type":"ContainerStarted","Data":"88cac3db1a98937506622746d5ba84115a734e883d6a825c55065a23744eac56"}
Sep 05 07:53:50.194063 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:50.194009    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw" podStartSLOduration=2.320523413 podStartE2EDuration="5.193988193s" totalImagesPullingTime="2.87346478s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:53:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:53:50.192906203 +0000 UTC m=+332.963048557" watchObservedRunningTime="2026-09-05 07:53:50.193988193 +0000 UTC m=+332.964130530"
Sep 05 07:53:50.272161 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:50.272086    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-mq4pt"]
Sep 05 07:53:50.276735 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:53:50.276677    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b4d9b90_5a4e_491b_ab28_884452aac755.slice/crio-e89c3a1254c0ca943852ce610ab28f2a197f0ac120d4fbb129784b6b330b9222 WatchSource:0}: Error finding container e89c3a1254c0ca943852ce610ab28f2a197f0ac120d4fbb129784b6b330b9222: Status 404 returned error can't find the container with id e89c3a1254c0ca943852ce610ab28f2a197f0ac120d4fbb129784b6b330b9222
Sep 05 07:53:51.165078 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:51.165039    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw"
Sep 05 07:53:51.170663 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:51.170634    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw"
Sep 05 07:53:51.183452 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:51.183414    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-mq4pt" event={"ID":"9b4d9b90-5a4e-491b-ab28-884452aac755","Type":"ContainerStarted","Data":"e89c3a1254c0ca943852ce610ab28f2a197f0ac120d4fbb129784b6b330b9222"}
Sep 05 07:53:51.183886 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:51.183765    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw"
Sep 05 07:53:51.184930 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:51.184907    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw"
Sep 05 07:53:52.876291 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:52.876205    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:53:53.196021 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:53.195923    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-bdp2h" event={"ID":"ced931cd-0339-489d-b218-01ac7471b5f1","Type":"ContainerStarted","Data":"75fd742c6737904b02f08855de94c4b57cbde8a326cd8adc2408687e10aaa81a"}
Sep 05 07:53:53.196021 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:53.196018    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-bdp2h"
Sep 05 07:53:53.197654 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:53.197629    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-mq4pt" event={"ID":"9b4d9b90-5a4e-491b-ab28-884452aac755","Type":"ContainerStarted","Data":"200f3685278b62c5aafd2d173af19453e5466524f4a56f44287c15626c1cdef5"}
Sep 05 07:53:53.197900 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:53.197873    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-mq4pt"
Sep 05 07:53:53.211333 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:53.211281    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-bdp2h" podStartSLOduration=1.341315055 podStartE2EDuration="4.211265189s" totalImagesPullingTime="2.869950134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:53:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:53:53.210721899 +0000 UTC m=+335.980864255" watchObservedRunningTime="2026-09-05 07:53:53.211265189 +0000 UTC m=+335.981407527"
Sep 05 07:53:53.230086 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:53.230038    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-mq4pt" podStartSLOduration=1.895288825 podStartE2EDuration="4.230022483s" totalImagesPullingTime="2.334733658s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:53:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:53:53.228493103 +0000 UTC m=+335.998635440" watchObservedRunningTime="2026-09-05 07:53:53.230022483 +0000 UTC m=+336.000164822"
Sep 05 07:53:54.740712 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:54.740677    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:53:56.085340 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:56.085305    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-bd655598-g899n"]
Sep 05 07:53:56.094738 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:56.094706    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-bd655598-g899n"
Sep 05 07:53:56.097197 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:56.097175    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 05 07:53:56.097801 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:56.097782    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 05 07:53:56.097913 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:56.097822    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 05 07:53:56.097913 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:56.097822    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-7gtqx\""
Sep 05 07:53:56.110109 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:56.110089    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-bd655598-g899n"]
Sep 05 07:53:56.174957 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:56.174926    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d0690683-7305-4e76-ab7c-183a766d282e-manager-config\") pod \"lws-controller-manager-bd655598-g899n\" (UID: \"d0690683-7305-4e76-ab7c-183a766d282e\") " pod="openshift-lws-operator/lws-controller-manager-bd655598-g899n"
Sep 05 07:53:56.174957 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:56.174961    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rljrr\" (UniqueName: \"kubernetes.io/projected/d0690683-7305-4e76-ab7c-183a766d282e-kube-api-access-rljrr\") pod \"lws-controller-manager-bd655598-g899n\" (UID: \"d0690683-7305-4e76-ab7c-183a766d282e\") " pod="openshift-lws-operator/lws-controller-manager-bd655598-g899n"
Sep 05 07:53:56.175152 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:56.174999    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d0690683-7305-4e76-ab7c-183a766d282e-cert\") pod \"lws-controller-manager-bd655598-g899n\" (UID: \"d0690683-7305-4e76-ab7c-183a766d282e\") " pod="openshift-lws-operator/lws-controller-manager-bd655598-g899n"
Sep 05 07:53:56.175152 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:56.175052    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d0690683-7305-4e76-ab7c-183a766d282e-metrics-cert\") pod \"lws-controller-manager-bd655598-g899n\" (UID: \"d0690683-7305-4e76-ab7c-183a766d282e\") " pod="openshift-lws-operator/lws-controller-manager-bd655598-g899n"
Sep 05 07:53:56.275446 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:56.275402    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d0690683-7305-4e76-ab7c-183a766d282e-manager-config\") pod \"lws-controller-manager-bd655598-g899n\" (UID: \"d0690683-7305-4e76-ab7c-183a766d282e\") " pod="openshift-lws-operator/lws-controller-manager-bd655598-g899n"
Sep 05 07:53:56.275446 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:56.275440    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rljrr\" (UniqueName: \"kubernetes.io/projected/d0690683-7305-4e76-ab7c-183a766d282e-kube-api-access-rljrr\") pod \"lws-controller-manager-bd655598-g899n\" (UID: \"d0690683-7305-4e76-ab7c-183a766d282e\") " pod="openshift-lws-operator/lws-controller-manager-bd655598-g899n"
Sep 05 07:53:56.275691 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:56.275488    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d0690683-7305-4e76-ab7c-183a766d282e-cert\") pod \"lws-controller-manager-bd655598-g899n\" (UID: \"d0690683-7305-4e76-ab7c-183a766d282e\") " pod="openshift-lws-operator/lws-controller-manager-bd655598-g899n"
Sep 05 07:53:56.275691 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:56.275528    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d0690683-7305-4e76-ab7c-183a766d282e-metrics-cert\") pod \"lws-controller-manager-bd655598-g899n\" (UID: \"d0690683-7305-4e76-ab7c-183a766d282e\") " pod="openshift-lws-operator/lws-controller-manager-bd655598-g899n"
Sep 05 07:53:56.276102 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:56.276080    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d0690683-7305-4e76-ab7c-183a766d282e-manager-config\") pod \"lws-controller-manager-bd655598-g899n\" (UID: \"d0690683-7305-4e76-ab7c-183a766d282e\") " pod="openshift-lws-operator/lws-controller-manager-bd655598-g899n"
Sep 05 07:53:56.278135 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:56.278107    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d0690683-7305-4e76-ab7c-183a766d282e-metrics-cert\") pod \"lws-controller-manager-bd655598-g899n\" (UID: \"d0690683-7305-4e76-ab7c-183a766d282e\") " pod="openshift-lws-operator/lws-controller-manager-bd655598-g899n"
Sep 05 07:53:56.278248 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:56.278137    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d0690683-7305-4e76-ab7c-183a766d282e-cert\") pod \"lws-controller-manager-bd655598-g899n\" (UID: \"d0690683-7305-4e76-ab7c-183a766d282e\") " pod="openshift-lws-operator/lws-controller-manager-bd655598-g899n"
Sep 05 07:53:56.283095 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:56.283075    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rljrr\" (UniqueName: \"kubernetes.io/projected/d0690683-7305-4e76-ab7c-183a766d282e-kube-api-access-rljrr\") pod \"lws-controller-manager-bd655598-g899n\" (UID: \"d0690683-7305-4e76-ab7c-183a766d282e\") " pod="openshift-lws-operator/lws-controller-manager-bd655598-g899n"
Sep 05 07:53:56.404051 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:56.403977    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-bd655598-g899n"
Sep 05 07:53:56.705937 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:56.705818    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-bd655598-g899n"]
Sep 05 07:53:56.708724 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:53:56.708699    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0690683_7305_4e76_ab7c_183a766d282e.slice/crio-e000c50f81aabf92e7a669ecba47154b986f85d67d1eda1c7b64e68c4a3c0179 WatchSource:0}: Error finding container e000c50f81aabf92e7a669ecba47154b986f85d67d1eda1c7b64e68c4a3c0179: Status 404 returned error can't find the container with id e000c50f81aabf92e7a669ecba47154b986f85d67d1eda1c7b64e68c4a3c0179
Sep 05 07:53:57.214833 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:57.214796    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-bd655598-g899n" event={"ID":"d0690683-7305-4e76-ab7c-183a766d282e","Type":"ContainerStarted","Data":"e000c50f81aabf92e7a669ecba47154b986f85d67d1eda1c7b64e68c4a3c0179"}
Sep 05 07:53:58.221587 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:58.221552    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-bd655598-g899n" event={"ID":"d0690683-7305-4e76-ab7c-183a766d282e","Type":"ContainerStarted","Data":"ca597721428cb2ffafed570e304942d73c8c790aa4019b26e7beeaa4dfb37104"}
Sep 05 07:53:58.222047 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:58.221684    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-bd655598-g899n"
Sep 05 07:53:58.248210 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:58.248112    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-bd655598-g899n" podStartSLOduration=0.960787665 podStartE2EDuration="2.248095829s" totalImagesPullingTime="1.287308164s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:53:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:53:58.247226413 +0000 UTC m=+341.017368781" watchObservedRunningTime="2026-09-05 07:53:58.248095829 +0000 UTC m=+341.018238165"
Sep 05 07:53:58.781415 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:58.781371    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:53:58.885559 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:58.885512    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:53:58.998504 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:53:58.998469    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:54:00.086494 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:54:00.086458    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:54:00.179929 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:54:00.179899    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:54:00.299349 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:54:00.299299    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:54:04.204005 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:54:04.203973    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-bdp2h"
Sep 05 07:54:04.205648 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:54:04.205626    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-mq4pt"
Sep 05 07:54:09.228650 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:54:09.228569    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-bd655598-g899n"
Sep 05 07:54:36.581136 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:54:36.581100    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:54:36.719583 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:54:36.719549    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:54:37.319054 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:54:37.319018    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:54:37.420801 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:54:37.420772    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:54:47.450612 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:54:47.450582    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:54:47.506491 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:54:47.506462    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:54:47.568320 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:54:47.568287    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:54:50.278349 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:54:50.278313    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:54:50.372200 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:54:50.372162    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:55:03.390610 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:55:03.390573    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:55:04.158335 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:55:04.158303    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:55:07.372609 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:55:07.372571    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:55:07.637636 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:55:07.637536    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:55:09.028267 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:55:09.028229    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:55:09.633400 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:55:09.633367    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:55:09.969310 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:55:09.969230    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:55:10.376799 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:55:10.376763    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:55:10.693136 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:55:10.693048    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:55:10.809420 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:55:10.809390    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:55:13.772248 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:55:13.772219    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:55:15.302677 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:55:15.302632    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:55:16.771580 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:55:16.771544    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:55:16.873517 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:55:16.873485    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:55:17.225481 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:55:17.225448    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:55:17.318032 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:55:17.317998    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:55:20.484662 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:55:20.484624    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:55:20.776248 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:55:20.776166    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:55:20.842271 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:55:20.842236    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:55:20.914259 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:55:20.914229    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:55:22.368320 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:55:22.368278    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:55:26.365723 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:55:26.365687    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:55:26.465122 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:55:26.465094    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:55:26.563184 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:55:26.563148    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:55:33.401491 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:55:33.401432    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:55:33.509312 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:55:33.509280    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:55:37.192548 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:55:37.192488    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:55:37.263749 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:55:37.263720    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:55:37.776306 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:55:37.776279    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:55:37.876860 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:55:37.876814    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:55:40.190575 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:55:40.190484    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:55:40.249385 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:55:40.249356    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:55:46.237652 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:55:46.237612    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:55:46.337772 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:55:46.337739    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:55:46.437225 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:55:46.437194    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:55:53.281187 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:55:53.281145    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:55:53.380731 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:55:53.380701    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:56:04.597637 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:04.597595    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:56:04.776332 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:04.776300    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:56:19.415160 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:19.415123    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:56:19.515921 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:19.515889    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:56:19.616256 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:19.616219    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:56:20.266775 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:20.266739    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b54df94b4-jwgfs"]
Sep 05 07:56:20.270081 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:20.270066    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-jwgfs"
Sep 05 07:56:20.272459 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:20.272419    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 05 07:56:20.272967 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:20.272947    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-c2j9p\""
Sep 05 07:56:20.273070 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:20.273024    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 05 07:56:20.278108 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:20.278087    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-jwgfs"]
Sep 05 07:56:20.305428 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:20.305405    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:56:20.369532 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:20.369498    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49c5f\" (UniqueName: \"kubernetes.io/projected/222f61a2-36f1-4d5e-83fc-3e06834fcf48-kube-api-access-49c5f\") pod \"authorino-b54df94b4-jwgfs\" (UID: \"222f61a2-36f1-4d5e-83fc-3e06834fcf48\") " pod="kuadrant-system/authorino-b54df94b4-jwgfs"
Sep 05 07:56:20.389610 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:20.389585    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:56:20.470040 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:20.470017    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49c5f\" (UniqueName: \"kubernetes.io/projected/222f61a2-36f1-4d5e-83fc-3e06834fcf48-kube-api-access-49c5f\") pod \"authorino-b54df94b4-jwgfs\" (UID: \"222f61a2-36f1-4d5e-83fc-3e06834fcf48\") " pod="kuadrant-system/authorino-b54df94b4-jwgfs"
Sep 05 07:56:20.477914 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:20.477891    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49c5f\" (UniqueName: \"kubernetes.io/projected/222f61a2-36f1-4d5e-83fc-3e06834fcf48-kube-api-access-49c5f\") pod \"authorino-b54df94b4-jwgfs\" (UID: \"222f61a2-36f1-4d5e-83fc-3e06834fcf48\") " pod="kuadrant-system/authorino-b54df94b4-jwgfs"
Sep 05 07:56:20.579269 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:20.579239    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-jwgfs"
Sep 05 07:56:20.850329 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:20.850301    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-jwgfs"]
Sep 05 07:56:20.852763 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:56:20.852732    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod222f61a2_36f1_4d5e_83fc_3e06834fcf48.slice/crio-39653852639e5a3e6fe36a17e60020af83b311c5b2a7e38ca93920d5e4bbc16d WatchSource:0}: Error finding container 39653852639e5a3e6fe36a17e60020af83b311c5b2a7e38ca93920d5e4bbc16d: Status 404 returned error can't find the container with id 39653852639e5a3e6fe36a17e60020af83b311c5b2a7e38ca93920d5e4bbc16d
Sep 05 07:56:21.846058 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:21.846018    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-jwgfs" event={"ID":"222f61a2-36f1-4d5e-83fc-3e06834fcf48","Type":"ContainerStarted","Data":"39653852639e5a3e6fe36a17e60020af83b311c5b2a7e38ca93920d5e4bbc16d"}
Sep 05 07:56:23.856856 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:23.856819    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-jwgfs" event={"ID":"222f61a2-36f1-4d5e-83fc-3e06834fcf48","Type":"ContainerStarted","Data":"9362daf882689109da2d78ab7b4897c1d16303073cf9ca06143478ba67e0a445"}
Sep 05 07:56:23.861120 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:23.861095    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-jwgfs"]
Sep 05 07:56:23.872096 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:23.872054    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b54df94b4-jwgfs" podStartSLOduration=1.319349379 podStartE2EDuration="3.87203947s" totalImagesPullingTime="2.552690091s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:56:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:56:23.871036549 +0000 UTC m=+486.641178887" watchObservedRunningTime="2026-09-05 07:56:23.87203947 +0000 UTC m=+486.642181807"
Sep 05 07:56:24.149274 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:24.149185    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:56:24.217152 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:24.217115    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:56:25.868023 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:25.867983    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-b54df94b4-jwgfs" podUID="222f61a2-36f1-4d5e-83fc-3e06834fcf48" containerName="authorino" containerID="cri-o://9362daf882689109da2d78ab7b4897c1d16303073cf9ca06143478ba67e0a445" gracePeriod=30
Sep 05 07:56:26.111760 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:26.111736    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-jwgfs"
Sep 05 07:56:26.222754 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:26.222663    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49c5f\" (UniqueName: \"kubernetes.io/projected/222f61a2-36f1-4d5e-83fc-3e06834fcf48-kube-api-access-49c5f\") pod \"222f61a2-36f1-4d5e-83fc-3e06834fcf48\" (UID: \"222f61a2-36f1-4d5e-83fc-3e06834fcf48\") "
Sep 05 07:56:26.224985 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:26.224950    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/222f61a2-36f1-4d5e-83fc-3e06834fcf48-kube-api-access-49c5f" (OuterVolumeSpecName: "kube-api-access-49c5f") pod "222f61a2-36f1-4d5e-83fc-3e06834fcf48" (UID: "222f61a2-36f1-4d5e-83fc-3e06834fcf48"). InnerVolumeSpecName "kube-api-access-49c5f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 07:56:26.323589 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:26.323557    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49c5f\" (UniqueName: \"kubernetes.io/projected/222f61a2-36f1-4d5e-83fc-3e06834fcf48-kube-api-access-49c5f\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 07:56:26.873170 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:26.873135    2563 generic.go:356] "Generic (PLEG): container finished" podID="222f61a2-36f1-4d5e-83fc-3e06834fcf48" containerID="9362daf882689109da2d78ab7b4897c1d16303073cf9ca06143478ba67e0a445" exitCode=0
Sep 05 07:56:26.873596 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:26.873179    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-jwgfs"
Sep 05 07:56:26.873596 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:26.873182    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-jwgfs" event={"ID":"222f61a2-36f1-4d5e-83fc-3e06834fcf48","Type":"ContainerDied","Data":"9362daf882689109da2d78ab7b4897c1d16303073cf9ca06143478ba67e0a445"}
Sep 05 07:56:26.873596 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:26.873298    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-jwgfs" event={"ID":"222f61a2-36f1-4d5e-83fc-3e06834fcf48","Type":"ContainerDied","Data":"39653852639e5a3e6fe36a17e60020af83b311c5b2a7e38ca93920d5e4bbc16d"}
Sep 05 07:56:26.873596 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:26.873322    2563 scope.go:117] "RemoveContainer" containerID="9362daf882689109da2d78ab7b4897c1d16303073cf9ca06143478ba67e0a445"
Sep 05 07:56:26.881615 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:26.881598    2563 scope.go:117] "RemoveContainer" containerID="9362daf882689109da2d78ab7b4897c1d16303073cf9ca06143478ba67e0a445"
Sep 05 07:56:26.881888 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:56:26.881865    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9362daf882689109da2d78ab7b4897c1d16303073cf9ca06143478ba67e0a445\": container with ID starting with 9362daf882689109da2d78ab7b4897c1d16303073cf9ca06143478ba67e0a445 not found: ID does not exist" containerID="9362daf882689109da2d78ab7b4897c1d16303073cf9ca06143478ba67e0a445"
Sep 05 07:56:26.881963 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:26.881894    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9362daf882689109da2d78ab7b4897c1d16303073cf9ca06143478ba67e0a445"} err="failed to get container status \"9362daf882689109da2d78ab7b4897c1d16303073cf9ca06143478ba67e0a445\": rpc error: code = NotFound desc = could not find container \"9362daf882689109da2d78ab7b4897c1d16303073cf9ca06143478ba67e0a445\": container with ID starting with 9362daf882689109da2d78ab7b4897c1d16303073cf9ca06143478ba67e0a445 not found: ID does not exist"
Sep 05 07:56:26.892067 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:26.892044    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-jwgfs"]
Sep 05 07:56:26.895262 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:26.895244    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b54df94b4-jwgfs"]
Sep 05 07:56:27.729539 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:27.729498    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="222f61a2-36f1-4d5e-83fc-3e06834fcf48" path="/var/lib/kubelet/pods/222f61a2-36f1-4d5e-83fc-3e06834fcf48/volumes"
Sep 05 07:56:36.230422 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:36.230381    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:56:36.331679 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:36.331653    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:56:36.743036 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:36.743004    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:56:49.295516 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:49.295477    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:56:49.558433 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:49.558401    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-5wbr5"]
Sep 05 07:56:49.558742 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:49.558730    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="222f61a2-36f1-4d5e-83fc-3e06834fcf48" containerName="authorino"
Sep 05 07:56:49.558783 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:49.558745    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="222f61a2-36f1-4d5e-83fc-3e06834fcf48" containerName="authorino"
Sep 05 07:56:49.558820 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:49.558801    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="222f61a2-36f1-4d5e-83fc-3e06834fcf48" containerName="authorino"
Sep 05 07:56:49.561591 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:49.561576    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-5wbr5"
Sep 05 07:56:49.563800 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:49.563783    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 05 07:56:49.563800 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:49.563788    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-c2j9p\""
Sep 05 07:56:49.564400 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:49.564383    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 05 07:56:49.567012 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:49.566991    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-5wbr5"]
Sep 05 07:56:49.578560 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:49.578534    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:56:49.618174 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:49.618145    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77fkn\" (UniqueName: \"kubernetes.io/projected/e0bdaa7a-a627-4998-a3d7-81db68195ebc-kube-api-access-77fkn\") pod \"authorino-5dc7ccb5f-5wbr5\" (UID: \"e0bdaa7a-a627-4998-a3d7-81db68195ebc\") " pod="kuadrant-system/authorino-5dc7ccb5f-5wbr5"
Sep 05 07:56:49.674837 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:49.674806    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-5wbr5"]
Sep 05 07:56:49.675079 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:56:49.675061    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-77fkn], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5dc7ccb5f-5wbr5" podUID="e0bdaa7a-a627-4998-a3d7-81db68195ebc"
Sep 05 07:56:49.702619 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:49.702592    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-687bd45585-wlbpr"]
Sep 05 07:56:49.705652 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:49.705638    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687bd45585-wlbpr"
Sep 05 07:56:49.707823 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:49.707809    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 05 07:56:49.714093 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:49.714070    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-687bd45585-wlbpr"]
Sep 05 07:56:49.718894 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:49.718871    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77fkn\" (UniqueName: \"kubernetes.io/projected/e0bdaa7a-a627-4998-a3d7-81db68195ebc-kube-api-access-77fkn\") pod \"authorino-5dc7ccb5f-5wbr5\" (UID: \"e0bdaa7a-a627-4998-a3d7-81db68195ebc\") " pod="kuadrant-system/authorino-5dc7ccb5f-5wbr5"
Sep 05 07:56:49.729869 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:49.729829    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77fkn\" (UniqueName: \"kubernetes.io/projected/e0bdaa7a-a627-4998-a3d7-81db68195ebc-kube-api-access-77fkn\") pod \"authorino-5dc7ccb5f-5wbr5\" (UID: \"e0bdaa7a-a627-4998-a3d7-81db68195ebc\") " pod="kuadrant-system/authorino-5dc7ccb5f-5wbr5"
Sep 05 07:56:49.801268 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:49.801232    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-687bd45585-wlbpr"]
Sep 05 07:56:49.801491 ip-10-0-133-14 kubenswrapper[2563]: E0905 07:56:49.801471    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-mmmgh tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-687bd45585-wlbpr" podUID="6ee22a53-3660-4cbf-bd00-d0687097f4f2"
Sep 05 07:56:49.820120 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:49.820048    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6ee22a53-3660-4cbf-bd00-d0687097f4f2-tls-cert\") pod \"authorino-687bd45585-wlbpr\" (UID: \"6ee22a53-3660-4cbf-bd00-d0687097f4f2\") " pod="kuadrant-system/authorino-687bd45585-wlbpr"
Sep 05 07:56:49.820120 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:49.820088    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmmgh\" (UniqueName: \"kubernetes.io/projected/6ee22a53-3660-4cbf-bd00-d0687097f4f2-kube-api-access-mmmgh\") pod \"authorino-687bd45585-wlbpr\" (UID: \"6ee22a53-3660-4cbf-bd00-d0687097f4f2\") " pod="kuadrant-system/authorino-687bd45585-wlbpr"
Sep 05 07:56:49.921295 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:49.921259    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6ee22a53-3660-4cbf-bd00-d0687097f4f2-tls-cert\") pod \"authorino-687bd45585-wlbpr\" (UID: \"6ee22a53-3660-4cbf-bd00-d0687097f4f2\") " pod="kuadrant-system/authorino-687bd45585-wlbpr"
Sep 05 07:56:49.921446 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:49.921304    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmmgh\" (UniqueName: \"kubernetes.io/projected/6ee22a53-3660-4cbf-bd00-d0687097f4f2-kube-api-access-mmmgh\") pod \"authorino-687bd45585-wlbpr\" (UID: \"6ee22a53-3660-4cbf-bd00-d0687097f4f2\") " pod="kuadrant-system/authorino-687bd45585-wlbpr"
Sep 05 07:56:49.923823 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:49.923797    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6ee22a53-3660-4cbf-bd00-d0687097f4f2-tls-cert\") pod \"authorino-687bd45585-wlbpr\" (UID: \"6ee22a53-3660-4cbf-bd00-d0687097f4f2\") " pod="kuadrant-system/authorino-687bd45585-wlbpr"
Sep 05 07:56:49.929982 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:49.929958    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmmgh\" (UniqueName: \"kubernetes.io/projected/6ee22a53-3660-4cbf-bd00-d0687097f4f2-kube-api-access-mmmgh\") pod \"authorino-687bd45585-wlbpr\" (UID: \"6ee22a53-3660-4cbf-bd00-d0687097f4f2\") " pod="kuadrant-system/authorino-687bd45585-wlbpr"
Sep 05 07:56:49.974323 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:49.974299    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687bd45585-wlbpr"
Sep 05 07:56:49.974460 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:49.974299    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-5wbr5"
Sep 05 07:56:49.981203 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:49.981165    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687bd45585-wlbpr"
Sep 05 07:56:49.984092 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:49.984064    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-5wbr5"
Sep 05 07:56:50.021739 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:50.021713    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mmmgh\" (UniqueName: \"kubernetes.io/projected/6ee22a53-3660-4cbf-bd00-d0687097f4f2-kube-api-access-mmmgh\") pod \"6ee22a53-3660-4cbf-bd00-d0687097f4f2\" (UID: \"6ee22a53-3660-4cbf-bd00-d0687097f4f2\") "
Sep 05 07:56:50.021940 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:50.021753    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6ee22a53-3660-4cbf-bd00-d0687097f4f2-tls-cert\") pod \"6ee22a53-3660-4cbf-bd00-d0687097f4f2\" (UID: \"6ee22a53-3660-4cbf-bd00-d0687097f4f2\") "
Sep 05 07:56:50.021940 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:50.021803    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-77fkn\" (UniqueName: \"kubernetes.io/projected/e0bdaa7a-a627-4998-a3d7-81db68195ebc-kube-api-access-77fkn\") pod \"e0bdaa7a-a627-4998-a3d7-81db68195ebc\" (UID: \"e0bdaa7a-a627-4998-a3d7-81db68195ebc\") "
Sep 05 07:56:50.024095 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:50.024062    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ee22a53-3660-4cbf-bd00-d0687097f4f2-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "6ee22a53-3660-4cbf-bd00-d0687097f4f2" (UID: "6ee22a53-3660-4cbf-bd00-d0687097f4f2"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 07:56:50.024189 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:50.024111    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0bdaa7a-a627-4998-a3d7-81db68195ebc-kube-api-access-77fkn" (OuterVolumeSpecName: "kube-api-access-77fkn") pod "e0bdaa7a-a627-4998-a3d7-81db68195ebc" (UID: "e0bdaa7a-a627-4998-a3d7-81db68195ebc"). InnerVolumeSpecName "kube-api-access-77fkn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 07:56:50.024189 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:50.024154    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ee22a53-3660-4cbf-bd00-d0687097f4f2-kube-api-access-mmmgh" (OuterVolumeSpecName: "kube-api-access-mmmgh") pod "6ee22a53-3660-4cbf-bd00-d0687097f4f2" (UID: "6ee22a53-3660-4cbf-bd00-d0687097f4f2"). InnerVolumeSpecName "kube-api-access-mmmgh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 07:56:50.122665 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:50.122593    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmmgh\" (UniqueName: \"kubernetes.io/projected/6ee22a53-3660-4cbf-bd00-d0687097f4f2-kube-api-access-mmmgh\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 07:56:50.122665 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:50.122619    2563 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6ee22a53-3660-4cbf-bd00-d0687097f4f2-tls-cert\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 07:56:50.122665 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:50.122628    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-77fkn\" (UniqueName: \"kubernetes.io/projected/e0bdaa7a-a627-4998-a3d7-81db68195ebc-kube-api-access-77fkn\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 07:56:50.978590 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:50.978558    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-5wbr5"
Sep 05 07:56:50.979056 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:50.978558    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687bd45585-wlbpr"
Sep 05 07:56:51.011288 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:51.011255    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-5wbr5"]
Sep 05 07:56:51.017079 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:51.017053    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-5wbr5"]
Sep 05 07:56:51.040404 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:51.040379    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-687bd45585-wlbpr"]
Sep 05 07:56:51.047006 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:51.046985    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-687bd45585-wlbpr"]
Sep 05 07:56:51.729886 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:51.729857    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ee22a53-3660-4cbf-bd00-d0687097f4f2" path="/var/lib/kubelet/pods/6ee22a53-3660-4cbf-bd00-d0687097f4f2/volumes"
Sep 05 07:56:51.730102 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:51.730090    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0bdaa7a-a627-4998-a3d7-81db68195ebc" path="/var/lib/kubelet/pods/e0bdaa7a-a627-4998-a3d7-81db68195ebc/volumes"
Sep 05 07:56:57.876292 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:57.876251    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:56:57.976236 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:57.976204    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:56:58.076515 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:58.076479    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:56:58.174226 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:58.174142    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:56:58.275048 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:56:58.275015    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:57:01.634312 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:57:01.634279    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:57:01.746617 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:57:01.746589    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:57:08.226544 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:57:08.226513    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:57:34.041931 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:57:34.041885    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:57:34.110605 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:57:34.110571    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:57:35.500608 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:57:35.500575    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:57:35.566664 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:57:35.566632    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:57:35.672622 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:57:35.672587    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:57:35.932111 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:57:35.932076    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:57:36.031196 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:57:36.031168    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:57:36.117124 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:57:36.117088    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:57:46.114156 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:57:46.114124    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:57:46.166438 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:57:46.166408    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:57:47.379236 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:57:47.379204    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:57:47.503218 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:57:47.503186    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:57:56.031970 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:57:56.031940    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:57:58.959693 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:57:58.959653    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:58:07.095750 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:07.095712    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-nxh4m"]
Sep 05 07:58:07.099038 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:07.099018    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxh4m"
Sep 05 07:58:07.101168 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:07.101148    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 05 07:58:07.101934 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:07.101911    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-7nfpk\""
Sep 05 07:58:07.101934 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:07.101920    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 05 07:58:07.102088 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:07.101947    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 05 07:58:07.107612 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:07.107591    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-nxh4m"]
Sep 05 07:58:07.115190 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:07.115170    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:58:07.177231 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:07.177199    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4712b98c-7254-4736-b545-55f5ca9eae13-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxh4m\" (UID: \"4712b98c-7254-4736-b545-55f5ca9eae13\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxh4m"
Sep 05 07:58:07.177383 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:07.177247    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4712b98c-7254-4736-b545-55f5ca9eae13-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxh4m\" (UID: \"4712b98c-7254-4736-b545-55f5ca9eae13\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxh4m"
Sep 05 07:58:07.177383 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:07.177296    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qh9t\" (UniqueName: \"kubernetes.io/projected/4712b98c-7254-4736-b545-55f5ca9eae13-kube-api-access-4qh9t\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxh4m\" (UID: \"4712b98c-7254-4736-b545-55f5ca9eae13\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxh4m"
Sep 05 07:58:07.177383 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:07.177313    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4712b98c-7254-4736-b545-55f5ca9eae13-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxh4m\" (UID: \"4712b98c-7254-4736-b545-55f5ca9eae13\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxh4m"
Sep 05 07:58:07.177383 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:07.177327    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4712b98c-7254-4736-b545-55f5ca9eae13-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxh4m\" (UID: \"4712b98c-7254-4736-b545-55f5ca9eae13\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxh4m"
Sep 05 07:58:07.177383 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:07.177342    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4712b98c-7254-4736-b545-55f5ca9eae13-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxh4m\" (UID: \"4712b98c-7254-4736-b545-55f5ca9eae13\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxh4m"
Sep 05 07:58:07.277723 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:07.277696    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4712b98c-7254-4736-b545-55f5ca9eae13-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxh4m\" (UID: \"4712b98c-7254-4736-b545-55f5ca9eae13\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxh4m"
Sep 05 07:58:07.277916 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:07.277739    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qh9t\" (UniqueName: \"kubernetes.io/projected/4712b98c-7254-4736-b545-55f5ca9eae13-kube-api-access-4qh9t\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxh4m\" (UID: \"4712b98c-7254-4736-b545-55f5ca9eae13\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxh4m"
Sep 05 07:58:07.277916 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:07.277758    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4712b98c-7254-4736-b545-55f5ca9eae13-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxh4m\" (UID: \"4712b98c-7254-4736-b545-55f5ca9eae13\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxh4m"
Sep 05 07:58:07.277916 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:07.277893    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4712b98c-7254-4736-b545-55f5ca9eae13-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxh4m\" (UID: \"4712b98c-7254-4736-b545-55f5ca9eae13\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxh4m"
Sep 05 07:58:07.278068 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:07.277931    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4712b98c-7254-4736-b545-55f5ca9eae13-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxh4m\" (UID: \"4712b98c-7254-4736-b545-55f5ca9eae13\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxh4m"
Sep 05 07:58:07.278068 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:07.277998    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4712b98c-7254-4736-b545-55f5ca9eae13-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxh4m\" (UID: \"4712b98c-7254-4736-b545-55f5ca9eae13\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxh4m"
Sep 05 07:58:07.278237 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:07.278213    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4712b98c-7254-4736-b545-55f5ca9eae13-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxh4m\" (UID: \"4712b98c-7254-4736-b545-55f5ca9eae13\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxh4m"
Sep 05 07:58:07.278372 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:07.278288    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4712b98c-7254-4736-b545-55f5ca9eae13-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxh4m\" (UID: \"4712b98c-7254-4736-b545-55f5ca9eae13\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxh4m"
Sep 05 07:58:07.278372 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:07.278315    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4712b98c-7254-4736-b545-55f5ca9eae13-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxh4m\" (UID: \"4712b98c-7254-4736-b545-55f5ca9eae13\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxh4m"
Sep 05 07:58:07.280302 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:07.280286    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4712b98c-7254-4736-b545-55f5ca9eae13-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxh4m\" (UID: \"4712b98c-7254-4736-b545-55f5ca9eae13\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxh4m"
Sep 05 07:58:07.280416 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:07.280389    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4712b98c-7254-4736-b545-55f5ca9eae13-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxh4m\" (UID: \"4712b98c-7254-4736-b545-55f5ca9eae13\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxh4m"
Sep 05 07:58:07.286019 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:07.285997    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qh9t\" (UniqueName: \"kubernetes.io/projected/4712b98c-7254-4736-b545-55f5ca9eae13-kube-api-access-4qh9t\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxh4m\" (UID: \"4712b98c-7254-4736-b545-55f5ca9eae13\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxh4m"
Sep 05 07:58:07.409941 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:07.409862    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxh4m"
Sep 05 07:58:07.679989 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:07.679958    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-nxh4m"]
Sep 05 07:58:07.682570 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:58:07.682546    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4712b98c_7254_4736_b545_55f5ca9eae13.slice/crio-2f8ec27b1963ad09a7d56a9339d24d12ea32a53a950bb005b5ab0f4af7b65e2f WatchSource:0}: Error finding container 2f8ec27b1963ad09a7d56a9339d24d12ea32a53a950bb005b5ab0f4af7b65e2f: Status 404 returned error can't find the container with id 2f8ec27b1963ad09a7d56a9339d24d12ea32a53a950bb005b5ab0f4af7b65e2f
Sep 05 07:58:08.305421 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:08.305387    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxh4m" event={"ID":"4712b98c-7254-4736-b545-55f5ca9eae13","Type":"ContainerStarted","Data":"2f8ec27b1963ad09a7d56a9339d24d12ea32a53a950bb005b5ab0f4af7b65e2f"}
Sep 05 07:58:09.311383 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:09.311339    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxh4m" event={"ID":"4712b98c-7254-4736-b545-55f5ca9eae13","Type":"ContainerStarted","Data":"edc998d6ccff50e354f9c0445a5ef832a9ffbba05d2aee022f8fbf302dc7d8fa"}
Sep 05 07:58:09.328071 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:09.328028    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxh4m" podStartSLOduration=1.380210182 podStartE2EDuration="2.328012705s" totalImagesPullingTime="947.802523ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:58:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:58:09.326270132 +0000 UTC m=+592.096412462" watchObservedRunningTime="2026-09-05 07:58:09.328012705 +0000 UTC m=+592.098155041"
Sep 05 07:58:13.925633 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:13.925589    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:58:17.410686 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:17.410648    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxh4m"
Sep 05 07:58:17.411156 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:17.411018    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxh4m"
Sep 05 07:58:17.423205 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:17.423182    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxh4m"
Sep 05 07:58:17.671194 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:17.671121    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2bzp8_569cffae-1c5c-4b02-af57-9ca53c455e4c/ovn-acl-logging/0.log"
Sep 05 07:58:17.672838 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:17.672814    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2bzp8_569cffae-1c5c-4b02-af57-9ca53c455e4c/ovn-acl-logging/0.log"
Sep 05 07:58:18.360211 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:18.360182    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxh4m"
Sep 05 07:58:21.920512 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:21.920467    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:58:22.313152 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:22.313114    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:58:33.298576 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:33.298542    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv"]
Sep 05 07:58:33.302001 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:33.301980    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv"
Sep 05 07:58:33.304206 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:33.304187    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 05 07:58:33.313374 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:33.313347    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv"]
Sep 05 07:58:33.334481 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:33.334450    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:58:33.405220 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:33.400451    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d93ca88-04d7-45d1-ab86-7b1e7e20c9f3-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv\" (UID: \"1d93ca88-04d7-45d1-ab86-7b1e7e20c9f3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv"
Sep 05 07:58:33.405220 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:33.400521    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d93ca88-04d7-45d1-ab86-7b1e7e20c9f3-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv\" (UID: \"1d93ca88-04d7-45d1-ab86-7b1e7e20c9f3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv"
Sep 05 07:58:33.405220 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:33.400597    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d93ca88-04d7-45d1-ab86-7b1e7e20c9f3-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv\" (UID: \"1d93ca88-04d7-45d1-ab86-7b1e7e20c9f3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv"
Sep 05 07:58:33.405220 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:33.400632    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssfpx\" (UniqueName: \"kubernetes.io/projected/1d93ca88-04d7-45d1-ab86-7b1e7e20c9f3-kube-api-access-ssfpx\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv\" (UID: \"1d93ca88-04d7-45d1-ab86-7b1e7e20c9f3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv"
Sep 05 07:58:33.405220 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:33.400669    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d93ca88-04d7-45d1-ab86-7b1e7e20c9f3-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv\" (UID: \"1d93ca88-04d7-45d1-ab86-7b1e7e20c9f3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv"
Sep 05 07:58:33.405220 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:33.400716    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d93ca88-04d7-45d1-ab86-7b1e7e20c9f3-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv\" (UID: \"1d93ca88-04d7-45d1-ab86-7b1e7e20c9f3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv"
Sep 05 07:58:33.501169 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:33.501128    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d93ca88-04d7-45d1-ab86-7b1e7e20c9f3-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv\" (UID: \"1d93ca88-04d7-45d1-ab86-7b1e7e20c9f3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv"
Sep 05 07:58:33.501361 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:33.501176    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssfpx\" (UniqueName: \"kubernetes.io/projected/1d93ca88-04d7-45d1-ab86-7b1e7e20c9f3-kube-api-access-ssfpx\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv\" (UID: \"1d93ca88-04d7-45d1-ab86-7b1e7e20c9f3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv"
Sep 05 07:58:33.501361 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:33.501206    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d93ca88-04d7-45d1-ab86-7b1e7e20c9f3-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv\" (UID: \"1d93ca88-04d7-45d1-ab86-7b1e7e20c9f3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv"
Sep 05 07:58:33.501361 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:33.501239    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d93ca88-04d7-45d1-ab86-7b1e7e20c9f3-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv\" (UID: \"1d93ca88-04d7-45d1-ab86-7b1e7e20c9f3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv"
Sep 05 07:58:33.501361 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:33.501280    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d93ca88-04d7-45d1-ab86-7b1e7e20c9f3-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv\" (UID: \"1d93ca88-04d7-45d1-ab86-7b1e7e20c9f3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv"
Sep 05 07:58:33.501361 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:33.501311    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d93ca88-04d7-45d1-ab86-7b1e7e20c9f3-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv\" (UID: \"1d93ca88-04d7-45d1-ab86-7b1e7e20c9f3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv"
Sep 05 07:58:33.501694 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:33.501663    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d93ca88-04d7-45d1-ab86-7b1e7e20c9f3-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv\" (UID: \"1d93ca88-04d7-45d1-ab86-7b1e7e20c9f3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv"
Sep 05 07:58:33.501864 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:33.501712    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d93ca88-04d7-45d1-ab86-7b1e7e20c9f3-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv\" (UID: \"1d93ca88-04d7-45d1-ab86-7b1e7e20c9f3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv"
Sep 05 07:58:33.501864 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:33.501823    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d93ca88-04d7-45d1-ab86-7b1e7e20c9f3-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv\" (UID: \"1d93ca88-04d7-45d1-ab86-7b1e7e20c9f3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv"
Sep 05 07:58:33.503670 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:33.503651    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d93ca88-04d7-45d1-ab86-7b1e7e20c9f3-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv\" (UID: \"1d93ca88-04d7-45d1-ab86-7b1e7e20c9f3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv"
Sep 05 07:58:33.503831 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:33.503814    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d93ca88-04d7-45d1-ab86-7b1e7e20c9f3-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv\" (UID: \"1d93ca88-04d7-45d1-ab86-7b1e7e20c9f3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv"
Sep 05 07:58:33.509239 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:33.509214    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssfpx\" (UniqueName: \"kubernetes.io/projected/1d93ca88-04d7-45d1-ab86-7b1e7e20c9f3-kube-api-access-ssfpx\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv\" (UID: \"1d93ca88-04d7-45d1-ab86-7b1e7e20c9f3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv"
Sep 05 07:58:33.612540 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:33.612506    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv"
Sep 05 07:58:33.885884 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:33.885837    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv"]
Sep 05 07:58:33.887685 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:58:33.887654    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d93ca88_04d7_45d1_ab86_7b1e7e20c9f3.slice/crio-2f3588e5d4818cb95dd7e3fb837ae376bfc08286e6f7ecb6d7a7b210f72ea73e WatchSource:0}: Error finding container 2f3588e5d4818cb95dd7e3fb837ae376bfc08286e6f7ecb6d7a7b210f72ea73e: Status 404 returned error can't find the container with id 2f3588e5d4818cb95dd7e3fb837ae376bfc08286e6f7ecb6d7a7b210f72ea73e
Sep 05 07:58:33.889360 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:33.889345    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 05 07:58:34.419618 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:34.419581    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv" event={"ID":"1d93ca88-04d7-45d1-ab86-7b1e7e20c9f3","Type":"ContainerStarted","Data":"e40c0fbec56de3b1e63a641897974ad0a00b6dcd198158cd78a6a550e1de2372"}
Sep 05 07:58:34.420112 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:34.419639    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv" event={"ID":"1d93ca88-04d7-45d1-ab86-7b1e7e20c9f3","Type":"ContainerStarted","Data":"2f3588e5d4818cb95dd7e3fb837ae376bfc08286e6f7ecb6d7a7b210f72ea73e"}
Sep 05 07:58:34.435954 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:34.435903    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv" podStartSLOduration=1.266998286 podStartE2EDuration="1.43583188s" totalImagesPullingTime="168.833594ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:58:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:58:34.434892211 +0000 UTC m=+617.205034552" watchObservedRunningTime="2026-09-05 07:58:34.43583188 +0000 UTC m=+617.205974217"
Sep 05 07:58:41.700473 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:41.700378    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-b628l"]
Sep 05 07:58:41.703870 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:41.703820    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-b628l"
Sep 05 07:58:41.706378 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:41.706349    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 05 07:58:41.714130 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:41.714108    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-b628l"]
Sep 05 07:58:41.728144 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:41.728116    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:58:41.875567 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:41.875532    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5b6f\" (UniqueName: \"kubernetes.io/projected/c5b025c7-bccc-417c-bf31-4e93ea8f8d95-kube-api-access-p5b6f\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-b628l\" (UID: \"c5b025c7-bccc-417c-bf31-4e93ea8f8d95\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-b628l"
Sep 05 07:58:41.875750 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:41.875608    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c5b025c7-bccc-417c-bf31-4e93ea8f8d95-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-b628l\" (UID: \"c5b025c7-bccc-417c-bf31-4e93ea8f8d95\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-b628l"
Sep 05 07:58:41.875750 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:41.875631    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5b025c7-bccc-417c-bf31-4e93ea8f8d95-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-b628l\" (UID: \"c5b025c7-bccc-417c-bf31-4e93ea8f8d95\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-b628l"
Sep 05 07:58:41.875750 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:41.875645    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5b025c7-bccc-417c-bf31-4e93ea8f8d95-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-b628l\" (UID: \"c5b025c7-bccc-417c-bf31-4e93ea8f8d95\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-b628l"
Sep 05 07:58:41.875750 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:41.875672    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5b025c7-bccc-417c-bf31-4e93ea8f8d95-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-b628l\" (UID: \"c5b025c7-bccc-417c-bf31-4e93ea8f8d95\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-b628l"
Sep 05 07:58:41.875930 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:41.875761    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5b025c7-bccc-417c-bf31-4e93ea8f8d95-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-b628l\" (UID: \"c5b025c7-bccc-417c-bf31-4e93ea8f8d95\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-b628l"
Sep 05 07:58:41.976428 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:41.976340    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c5b025c7-bccc-417c-bf31-4e93ea8f8d95-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-b628l\" (UID: \"c5b025c7-bccc-417c-bf31-4e93ea8f8d95\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-b628l"
Sep 05 07:58:41.976428 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:41.976374    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5b025c7-bccc-417c-bf31-4e93ea8f8d95-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-b628l\" (UID: \"c5b025c7-bccc-417c-bf31-4e93ea8f8d95\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-b628l"
Sep 05 07:58:41.976428 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:41.976390    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5b025c7-bccc-417c-bf31-4e93ea8f8d95-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-b628l\" (UID: \"c5b025c7-bccc-417c-bf31-4e93ea8f8d95\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-b628l"
Sep 05 07:58:41.976701 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:41.976432    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5b025c7-bccc-417c-bf31-4e93ea8f8d95-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-b628l\" (UID: \"c5b025c7-bccc-417c-bf31-4e93ea8f8d95\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-b628l"
Sep 05 07:58:41.976701 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:41.976459    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5b025c7-bccc-417c-bf31-4e93ea8f8d95-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-b628l\" (UID: \"c5b025c7-bccc-417c-bf31-4e93ea8f8d95\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-b628l"
Sep 05 07:58:41.976701 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:41.976506    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5b6f\" (UniqueName: \"kubernetes.io/projected/c5b025c7-bccc-417c-bf31-4e93ea8f8d95-kube-api-access-p5b6f\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-b628l\" (UID: \"c5b025c7-bccc-417c-bf31-4e93ea8f8d95\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-b628l"
Sep 05 07:58:41.976867 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:41.976740    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c5b025c7-bccc-417c-bf31-4e93ea8f8d95-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-b628l\" (UID: \"c5b025c7-bccc-417c-bf31-4e93ea8f8d95\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-b628l"
Sep 05 07:58:41.976913 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:41.976889    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5b025c7-bccc-417c-bf31-4e93ea8f8d95-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-b628l\" (UID: \"c5b025c7-bccc-417c-bf31-4e93ea8f8d95\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-b628l"
Sep 05 07:58:41.976947 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:41.976922    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5b025c7-bccc-417c-bf31-4e93ea8f8d95-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-b628l\" (UID: \"c5b025c7-bccc-417c-bf31-4e93ea8f8d95\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-b628l"
Sep 05 07:58:41.978870 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:41.978825    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5b025c7-bccc-417c-bf31-4e93ea8f8d95-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-b628l\" (UID: \"c5b025c7-bccc-417c-bf31-4e93ea8f8d95\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-b628l"
Sep 05 07:58:41.978974 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:41.978956    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5b025c7-bccc-417c-bf31-4e93ea8f8d95-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-b628l\" (UID: \"c5b025c7-bccc-417c-bf31-4e93ea8f8d95\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-b628l"
Sep 05 07:58:41.985117 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:41.985096    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5b6f\" (UniqueName: \"kubernetes.io/projected/c5b025c7-bccc-417c-bf31-4e93ea8f8d95-kube-api-access-p5b6f\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-b628l\" (UID: \"c5b025c7-bccc-417c-bf31-4e93ea8f8d95\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-b628l"
Sep 05 07:58:42.014162 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:42.014139    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-b628l"
Sep 05 07:58:42.286690 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:42.286662    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-b628l"]
Sep 05 07:58:42.288717 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:58:42.288687    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5b025c7_bccc_417c_bf31_4e93ea8f8d95.slice/crio-bce051fbd22fd142db0c2870060d3a8ccf05bebf3104e26ca6ba86c63322d0d9 WatchSource:0}: Error finding container bce051fbd22fd142db0c2870060d3a8ccf05bebf3104e26ca6ba86c63322d0d9: Status 404 returned error can't find the container with id bce051fbd22fd142db0c2870060d3a8ccf05bebf3104e26ca6ba86c63322d0d9
Sep 05 07:58:42.455684 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:42.455650    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-b628l" event={"ID":"c5b025c7-bccc-417c-bf31-4e93ea8f8d95","Type":"ContainerStarted","Data":"bce051fbd22fd142db0c2870060d3a8ccf05bebf3104e26ca6ba86c63322d0d9"}
Sep 05 07:58:43.461856 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:43.461809    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-b628l" event={"ID":"c5b025c7-bccc-417c-bf31-4e93ea8f8d95","Type":"ContainerStarted","Data":"27792e5c352158423ab8db35bfc4ae8d463fa1e8829dd480e59c6965516d229d"}
Sep 05 07:58:43.480265 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:43.480217    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-b628l" podStartSLOduration=2.313344255 podStartE2EDuration="2.48020067s" totalImagesPullingTime="166.856415ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:58:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:58:43.47883806 +0000 UTC m=+626.248980398" watchObservedRunningTime="2026-09-05 07:58:43.48020067 +0000 UTC m=+626.250343006"
Sep 05 07:58:43.613208 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:43.613173    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv"
Sep 05 07:58:43.613369 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:43.613238    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv"
Sep 05 07:58:43.625398 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:43.625370    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv"
Sep 05 07:58:44.483226 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:44.483193    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv"
Sep 05 07:58:52.014965 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:52.014908    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-b628l"
Sep 05 07:58:52.015497 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:52.014989    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-b628l"
Sep 05 07:58:52.027918 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:52.027896    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-b628l"
Sep 05 07:58:52.515342 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:52.515310    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-b628l"
Sep 05 07:58:59.292804 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:59.292770    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm"]
Sep 05 07:58:59.296148 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:59.296133    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm"
Sep 05 07:58:59.298165 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:59.298147    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 05 07:58:59.306259 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:59.306235    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm"]
Sep 05 07:58:59.321290 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:59.321260    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b5b1ba7-aad0-4a90-9e8e-8159cd5ce603-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm\" (UID: \"1b5b1ba7-aad0-4a90-9e8e-8159cd5ce603\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm"
Sep 05 07:58:59.321392 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:59.321303    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b5b1ba7-aad0-4a90-9e8e-8159cd5ce603-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm\" (UID: \"1b5b1ba7-aad0-4a90-9e8e-8159cd5ce603\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm"
Sep 05 07:58:59.321392 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:59.321369    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b5b1ba7-aad0-4a90-9e8e-8159cd5ce603-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm\" (UID: \"1b5b1ba7-aad0-4a90-9e8e-8159cd5ce603\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm"
Sep 05 07:58:59.321506 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:59.321403    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b5b1ba7-aad0-4a90-9e8e-8159cd5ce603-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm\" (UID: \"1b5b1ba7-aad0-4a90-9e8e-8159cd5ce603\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm"
Sep 05 07:58:59.321506 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:59.321450    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrz56\" (UniqueName: \"kubernetes.io/projected/1b5b1ba7-aad0-4a90-9e8e-8159cd5ce603-kube-api-access-lrz56\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm\" (UID: \"1b5b1ba7-aad0-4a90-9e8e-8159cd5ce603\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm"
Sep 05 07:58:59.321506 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:59.321493    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b5b1ba7-aad0-4a90-9e8e-8159cd5ce603-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm\" (UID: \"1b5b1ba7-aad0-4a90-9e8e-8159cd5ce603\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm"
Sep 05 07:58:59.330379 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:59.330354    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:58:59.422702 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:59.422675    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrz56\" (UniqueName: \"kubernetes.io/projected/1b5b1ba7-aad0-4a90-9e8e-8159cd5ce603-kube-api-access-lrz56\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm\" (UID: \"1b5b1ba7-aad0-4a90-9e8e-8159cd5ce603\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm"
Sep 05 07:58:59.422881 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:59.422712    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b5b1ba7-aad0-4a90-9e8e-8159cd5ce603-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm\" (UID: \"1b5b1ba7-aad0-4a90-9e8e-8159cd5ce603\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm"
Sep 05 07:58:59.422881 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:59.422764    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b5b1ba7-aad0-4a90-9e8e-8159cd5ce603-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm\" (UID: \"1b5b1ba7-aad0-4a90-9e8e-8159cd5ce603\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm"
Sep 05 07:58:59.422881 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:59.422789    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b5b1ba7-aad0-4a90-9e8e-8159cd5ce603-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm\" (UID: \"1b5b1ba7-aad0-4a90-9e8e-8159cd5ce603\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm"
Sep 05 07:58:59.423062 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:59.422911    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b5b1ba7-aad0-4a90-9e8e-8159cd5ce603-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm\" (UID: \"1b5b1ba7-aad0-4a90-9e8e-8159cd5ce603\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm"
Sep 05 07:58:59.423062 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:59.422934    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b5b1ba7-aad0-4a90-9e8e-8159cd5ce603-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm\" (UID: \"1b5b1ba7-aad0-4a90-9e8e-8159cd5ce603\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm"
Sep 05 07:58:59.423196 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:59.423176    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b5b1ba7-aad0-4a90-9e8e-8159cd5ce603-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm\" (UID: \"1b5b1ba7-aad0-4a90-9e8e-8159cd5ce603\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm"
Sep 05 07:58:59.423252 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:59.423200    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b5b1ba7-aad0-4a90-9e8e-8159cd5ce603-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm\" (UID: \"1b5b1ba7-aad0-4a90-9e8e-8159cd5ce603\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm"
Sep 05 07:58:59.423252 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:59.423218    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b5b1ba7-aad0-4a90-9e8e-8159cd5ce603-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm\" (UID: \"1b5b1ba7-aad0-4a90-9e8e-8159cd5ce603\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm"
Sep 05 07:58:59.425284 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:59.425264    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b5b1ba7-aad0-4a90-9e8e-8159cd5ce603-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm\" (UID: \"1b5b1ba7-aad0-4a90-9e8e-8159cd5ce603\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm"
Sep 05 07:58:59.425493 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:59.425474    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b5b1ba7-aad0-4a90-9e8e-8159cd5ce603-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm\" (UID: \"1b5b1ba7-aad0-4a90-9e8e-8159cd5ce603\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm"
Sep 05 07:58:59.430388 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:59.430368    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrz56\" (UniqueName: \"kubernetes.io/projected/1b5b1ba7-aad0-4a90-9e8e-8159cd5ce603-kube-api-access-lrz56\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm\" (UID: \"1b5b1ba7-aad0-4a90-9e8e-8159cd5ce603\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm"
Sep 05 07:58:59.606010 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:59.605980    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm"
Sep 05 07:58:59.875787 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:59.875760    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm"]
Sep 05 07:58:59.878128 ip-10-0-133-14 kubenswrapper[2563]: W0905 07:58:59.878101    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1b5b1ba7_aad0_4a90_9e8e_8159cd5ce603.slice/crio-111cc4f789ecd871fc93394f147ce623ae30efde06c1e6de33175ed8957e1ba3 WatchSource:0}: Error finding container 111cc4f789ecd871fc93394f147ce623ae30efde06c1e6de33175ed8957e1ba3: Status 404 returned error can't find the container with id 111cc4f789ecd871fc93394f147ce623ae30efde06c1e6de33175ed8957e1ba3
Sep 05 07:58:59.894302 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:59.894279    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:58:59.995237 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:58:59.995205    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 07:59:00.545046 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:59:00.545006    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm" event={"ID":"1b5b1ba7-aad0-4a90-9e8e-8159cd5ce603","Type":"ContainerStarted","Data":"ccf300ac18c9b8fd9e5afce9665ee20197cb72dc8e07bb18fe0518fc5dc1cf2e"}
Sep 05 07:59:00.545484 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:59:00.545057    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm" event={"ID":"1b5b1ba7-aad0-4a90-9e8e-8159cd5ce603","Type":"ContainerStarted","Data":"111cc4f789ecd871fc93394f147ce623ae30efde06c1e6de33175ed8957e1ba3"}
Sep 05 07:59:00.562348 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:59:00.562301    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm" podStartSLOduration=1.397535583 podStartE2EDuration="1.562286103s" totalImagesPullingTime="164.75052ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:58:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:59:00.560715951 +0000 UTC m=+643.330858314" watchObservedRunningTime="2026-09-05 07:59:00.562286103 +0000 UTC m=+643.332428441"
Sep 05 07:59:09.606565 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:59:09.606530    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm"
Sep 05 07:59:09.606565 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:59:09.606573    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm"
Sep 05 07:59:09.618952 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:59:09.618933    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm"
Sep 05 07:59:10.601910 ip-10-0-133-14 kubenswrapper[2563]: I0905 07:59:10.601884    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm"
Sep 05 08:00:09.476767 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:09.476684    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-54d58847c4-4sdn5"]
Sep 05 08:00:09.480078 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:09.480061    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54d58847c4-4sdn5"
Sep 05 08:00:09.482295 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:09.482269    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 05 08:00:09.483209 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:09.483189    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 05 08:00:09.483338 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:09.483236    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-c2j9p\""
Sep 05 08:00:09.483338 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:09.483316    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 05 08:00:09.487747 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:09.487725    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-54d58847c4-4sdn5"]
Sep 05 08:00:09.509559 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:09.509534    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p847v\" (UniqueName: \"kubernetes.io/projected/46f380f3-0a56-427b-85b1-c5463df61a92-kube-api-access-p847v\") pod \"authorino-54d58847c4-4sdn5\" (UID: \"46f380f3-0a56-427b-85b1-c5463df61a92\") " pod="kuadrant-system/authorino-54d58847c4-4sdn5"
Sep 05 08:00:09.509660 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:09.509562    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/46f380f3-0a56-427b-85b1-c5463df61a92-tls-cert\") pod \"authorino-54d58847c4-4sdn5\" (UID: \"46f380f3-0a56-427b-85b1-c5463df61a92\") " pod="kuadrant-system/authorino-54d58847c4-4sdn5"
Sep 05 08:00:09.610702 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:09.610663    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p847v\" (UniqueName: \"kubernetes.io/projected/46f380f3-0a56-427b-85b1-c5463df61a92-kube-api-access-p847v\") pod \"authorino-54d58847c4-4sdn5\" (UID: \"46f380f3-0a56-427b-85b1-c5463df61a92\") " pod="kuadrant-system/authorino-54d58847c4-4sdn5"
Sep 05 08:00:09.610887 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:09.610710    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/46f380f3-0a56-427b-85b1-c5463df61a92-tls-cert\") pod \"authorino-54d58847c4-4sdn5\" (UID: \"46f380f3-0a56-427b-85b1-c5463df61a92\") " pod="kuadrant-system/authorino-54d58847c4-4sdn5"
Sep 05 08:00:09.613350 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:09.613325    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/46f380f3-0a56-427b-85b1-c5463df61a92-tls-cert\") pod \"authorino-54d58847c4-4sdn5\" (UID: \"46f380f3-0a56-427b-85b1-c5463df61a92\") " pod="kuadrant-system/authorino-54d58847c4-4sdn5"
Sep 05 08:00:09.619213 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:09.619188    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p847v\" (UniqueName: \"kubernetes.io/projected/46f380f3-0a56-427b-85b1-c5463df61a92-kube-api-access-p847v\") pod \"authorino-54d58847c4-4sdn5\" (UID: \"46f380f3-0a56-427b-85b1-c5463df61a92\") " pod="kuadrant-system/authorino-54d58847c4-4sdn5"
Sep 05 08:00:09.789728 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:09.789650    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54d58847c4-4sdn5"
Sep 05 08:00:10.064299 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:10.064272    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-54d58847c4-4sdn5"]
Sep 05 08:00:10.066920 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:00:10.066893    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod46f380f3_0a56_427b_85b1_c5463df61a92.slice/crio-a2dd317f6a1064c5548d438aac63bcce78e3e25e55707e0e790f3209f2aade3d WatchSource:0}: Error finding container a2dd317f6a1064c5548d438aac63bcce78e3e25e55707e0e790f3209f2aade3d: Status 404 returned error can't find the container with id a2dd317f6a1064c5548d438aac63bcce78e3e25e55707e0e790f3209f2aade3d
Sep 05 08:00:10.862616 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:10.862585    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54d58847c4-4sdn5" event={"ID":"46f380f3-0a56-427b-85b1-c5463df61a92","Type":"ContainerStarted","Data":"a2dd317f6a1064c5548d438aac63bcce78e3e25e55707e0e790f3209f2aade3d"}
Sep 05 08:00:16.892217 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:16.892184    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54d58847c4-4sdn5" event={"ID":"46f380f3-0a56-427b-85b1-c5463df61a92","Type":"ContainerStarted","Data":"ee3071e742dd343c57c2fbed73c6bae021f9e63efd7391ce46eb27833d2e578b"}
Sep 05 08:00:16.944538 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:16.944472    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-54d58847c4-4sdn5" podStartSLOduration=1.8600130099999999 podStartE2EDuration="7.944451926s" totalImagesPullingTime="6.084438916s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:00:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:00:16.943218501 +0000 UTC m=+719.713360848" watchObservedRunningTime="2026-09-05 08:00:16.944451926 +0000 UTC m=+719.714594265"
Sep 05 08:00:17.743227 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:17.743193    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:17.841972 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:17.841941    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:19.019050 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:19.019011    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:19.115338 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:19.115302    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:39.034569 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:39.034532    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:39.151762 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:39.151722    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:39.582026 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:39.581997    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:40.130093 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:40.130053    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:40.593982 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:40.593948    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:40.692230 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:40.692185    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:41.052503 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:41.052419    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:42.211905 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:42.211817    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:42.297977 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:42.297944    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:43.398727 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:43.398691    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:43.507276 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:43.507237    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:43.656700 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:43.656614    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:44.007133 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:44.007048    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:44.099415 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:44.099374    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:45.937018 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:45.936986    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7859-140262762759472"]
Sep 05 08:00:45.940324 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:45.940308    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7859-140262762759472"
Sep 05 08:00:45.941116 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:45.941091    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wm7gt\" (UniqueName: \"kubernetes.io/projected/07e9fa64-30a4-40a5-9a1a-5b2491af5391-kube-api-access-wm7gt\") pod \"test-curl-7859-140262762759472\" (UID: \"07e9fa64-30a4-40a5-9a1a-5b2491af5391\") " pod="opendatahub/test-curl-7859-140262762759472"
Sep 05 08:00:45.942959 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:45.942938    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-x4x7f\""
Sep 05 08:00:45.946624 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:45.946604    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7859-140262762759472"]
Sep 05 08:00:46.041651 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:46.041617    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wm7gt\" (UniqueName: \"kubernetes.io/projected/07e9fa64-30a4-40a5-9a1a-5b2491af5391-kube-api-access-wm7gt\") pod \"test-curl-7859-140262762759472\" (UID: \"07e9fa64-30a4-40a5-9a1a-5b2491af5391\") " pod="opendatahub/test-curl-7859-140262762759472"
Sep 05 08:00:46.049424 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:46.049391    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wm7gt\" (UniqueName: \"kubernetes.io/projected/07e9fa64-30a4-40a5-9a1a-5b2491af5391-kube-api-access-wm7gt\") pod \"test-curl-7859-140262762759472\" (UID: \"07e9fa64-30a4-40a5-9a1a-5b2491af5391\") " pod="opendatahub/test-curl-7859-140262762759472"
Sep 05 08:00:46.251190 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:46.251103    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7859-140262762759472"
Sep 05 08:00:46.521132 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:46.521002    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7859-140262762759472"]
Sep 05 08:00:46.523866 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:00:46.523819    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07e9fa64_30a4_40a5_9a1a_5b2491af5391.slice/crio-d77f434954e5582978d5aba95a8722e473aa623096f5120d8105bc67e3e8a66a WatchSource:0}: Error finding container d77f434954e5582978d5aba95a8722e473aa623096f5120d8105bc67e3e8a66a: Status 404 returned error can't find the container with id d77f434954e5582978d5aba95a8722e473aa623096f5120d8105bc67e3e8a66a
Sep 05 08:00:47.028578 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:47.028544    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7859-140262762759472" event={"ID":"07e9fa64-30a4-40a5-9a1a-5b2491af5391","Type":"ContainerStarted","Data":"d77f434954e5582978d5aba95a8722e473aa623096f5120d8105bc67e3e8a66a"}
Sep 05 08:00:47.352946 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:47.352912    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:47.430229 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:47.430199    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:47.641285 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:47.641196    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:47.960875 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:47.960773    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:48.201456 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:48.201412    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:49.039092 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:49.039052    2563 generic.go:356] "Generic (PLEG): container finished" podID="07e9fa64-30a4-40a5-9a1a-5b2491af5391" containerID="bd995adc34e4714cd1f58add31c327027c456c85cb2f70605adc671fbd23e5fd" exitCode=0
Sep 05 08:00:49.039258 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:49.039137    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7859-140262762759472" event={"ID":"07e9fa64-30a4-40a5-9a1a-5b2491af5391","Type":"ContainerDied","Data":"bd995adc34e4714cd1f58add31c327027c456c85cb2f70605adc671fbd23e5fd"}
Sep 05 08:00:50.185887 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:50.185836    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7859-140262762759472"
Sep 05 08:00:50.208996 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:50.208945    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7859-140262762759472_07e9fa64-30a4-40a5-9a1a-5b2491af5391/test-curl-7859-140262762759472/0.log"
Sep 05 08:00:50.236166 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:50.236106    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7859-140262762759472"]
Sep 05 08:00:50.237965 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:50.237939    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7859-140262762759472"]
Sep 05 08:00:50.277691 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:50.277660    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wm7gt\" (UniqueName: \"kubernetes.io/projected/07e9fa64-30a4-40a5-9a1a-5b2491af5391-kube-api-access-wm7gt\") pod \"07e9fa64-30a4-40a5-9a1a-5b2491af5391\" (UID: \"07e9fa64-30a4-40a5-9a1a-5b2491af5391\") "
Sep 05 08:00:50.280411 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:50.280368    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07e9fa64-30a4-40a5-9a1a-5b2491af5391-kube-api-access-wm7gt" (OuterVolumeSpecName: "kube-api-access-wm7gt") pod "07e9fa64-30a4-40a5-9a1a-5b2491af5391" (UID: "07e9fa64-30a4-40a5-9a1a-5b2491af5391"). InnerVolumeSpecName "kube-api-access-wm7gt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:00:50.378992 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:50.378962    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wm7gt\" (UniqueName: \"kubernetes.io/projected/07e9fa64-30a4-40a5-9a1a-5b2491af5391-kube-api-access-wm7gt\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:00:50.711194 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:50.711076    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:50.818041 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:50.817998    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:50.918696 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:50.918651    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:51.050451 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.050373    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d77f434954e5582978d5aba95a8722e473aa623096f5120d8105bc67e3e8a66a"
Sep 05 08:00:51.050451 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.050430    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7859-140262762759472"
Sep 05 08:00:51.060639 ip-10-0-133-14 kubenswrapper[2563]: E0905 08:00:51.060612    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7859-140262762759472\" is forbidden: User \"system:node:ip-10-0-133-14.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-14.ec2.internal' and this object" podUID="07e9fa64-30a4-40a5-9a1a-5b2491af5391" pod="opendatahub/test-curl-7859-140262762759472"
Sep 05 08:00:51.134935 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.134897    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"]
Sep 05 08:00:51.135261 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.135249    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07e9fa64-30a4-40a5-9a1a-5b2491af5391" containerName="test-curl-7859-140262762759472"
Sep 05 08:00:51.135307 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.135264    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="07e9fa64-30a4-40a5-9a1a-5b2491af5391" containerName="test-curl-7859-140262762759472"
Sep 05 08:00:51.135353 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.135344    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="07e9fa64-30a4-40a5-9a1a-5b2491af5391" containerName="test-curl-7859-140262762759472"
Sep 05 08:00:51.138355 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.138339    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"
Sep 05 08:00:51.140108 ip-10-0-133-14 kubenswrapper[2563]: E0905 08:00:51.140075    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7859-140262762759472\" is forbidden: User \"system:node:ip-10-0-133-14.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-14.ec2.internal' and this object" podUID="07e9fa64-30a4-40a5-9a1a-5b2491af5391" pod="opendatahub/test-curl-7859-140262762759472"
Sep 05 08:00:51.140584 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.140559    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-4faf97-openshift-default-dockercfg-zmtr7\""
Sep 05 08:00:51.154012 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.153989    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"]
Sep 05 08:00:51.182013 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.181983    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:51.185147 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.185120    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b064f4cc-d9d3-4cf1-9053-0253e730c30c-workload-socket\") pod \"e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc\" (UID: \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\") " pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"
Sep 05 08:00:51.185269 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.185157    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b064f4cc-d9d3-4cf1-9053-0253e730c30c-credential-socket\") pod \"e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc\" (UID: \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\") " pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"
Sep 05 08:00:51.185269 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.185200    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b064f4cc-d9d3-4cf1-9053-0253e730c30c-istio-podinfo\") pod \"e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc\" (UID: \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\") " pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"
Sep 05 08:00:51.185269 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.185244    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b064f4cc-d9d3-4cf1-9053-0253e730c30c-workload-certs\") pod \"e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc\" (UID: \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\") " pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"
Sep 05 08:00:51.185417 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.185301    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b064f4cc-d9d3-4cf1-9053-0253e730c30c-istio-envoy\") pod \"e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc\" (UID: \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\") " pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"
Sep 05 08:00:51.185417 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.185322    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b064f4cc-d9d3-4cf1-9053-0253e730c30c-istio-data\") pod \"e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc\" (UID: \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\") " pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"
Sep 05 08:00:51.185417 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.185351    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b064f4cc-d9d3-4cf1-9053-0253e730c30c-istio-token\") pod \"e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc\" (UID: \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\") " pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"
Sep 05 08:00:51.185417 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.185372    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b064f4cc-d9d3-4cf1-9053-0253e730c30c-istiod-ca-cert\") pod \"e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc\" (UID: \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\") " pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"
Sep 05 08:00:51.185417 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.185387    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gp776\" (UniqueName: \"kubernetes.io/projected/b064f4cc-d9d3-4cf1-9053-0253e730c30c-kube-api-access-gp776\") pod \"e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc\" (UID: \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\") " pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"
Sep 05 08:00:51.286364 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.286322    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b064f4cc-d9d3-4cf1-9053-0253e730c30c-istio-token\") pod \"e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc\" (UID: \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\") " pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"
Sep 05 08:00:51.286364 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.286367    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b064f4cc-d9d3-4cf1-9053-0253e730c30c-istiod-ca-cert\") pod \"e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc\" (UID: \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\") " pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"
Sep 05 08:00:51.286944 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.286393    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gp776\" (UniqueName: \"kubernetes.io/projected/b064f4cc-d9d3-4cf1-9053-0253e730c30c-kube-api-access-gp776\") pod \"e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc\" (UID: \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\") " pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"
Sep 05 08:00:51.286944 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.286425    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b064f4cc-d9d3-4cf1-9053-0253e730c30c-workload-socket\") pod \"e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc\" (UID: \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\") " pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"
Sep 05 08:00:51.286944 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.286464    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b064f4cc-d9d3-4cf1-9053-0253e730c30c-credential-socket\") pod \"e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc\" (UID: \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\") " pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"
Sep 05 08:00:51.286944 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.286536    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b064f4cc-d9d3-4cf1-9053-0253e730c30c-istio-podinfo\") pod \"e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc\" (UID: \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\") " pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"
Sep 05 08:00:51.286944 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.286610    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b064f4cc-d9d3-4cf1-9053-0253e730c30c-workload-certs\") pod \"e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc\" (UID: \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\") " pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"
Sep 05 08:00:51.286944 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.286649    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b064f4cc-d9d3-4cf1-9053-0253e730c30c-istio-envoy\") pod \"e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc\" (UID: \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\") " pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"
Sep 05 08:00:51.286944 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.286676    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b064f4cc-d9d3-4cf1-9053-0253e730c30c-istio-data\") pod \"e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc\" (UID: \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\") " pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"
Sep 05 08:00:51.286944 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.286866    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b064f4cc-d9d3-4cf1-9053-0253e730c30c-workload-socket\") pod \"e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc\" (UID: \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\") " pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"
Sep 05 08:00:51.287244 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.286948    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b064f4cc-d9d3-4cf1-9053-0253e730c30c-credential-socket\") pod \"e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc\" (UID: \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\") " pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"
Sep 05 08:00:51.287244 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.287050    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b064f4cc-d9d3-4cf1-9053-0253e730c30c-istio-data\") pod \"e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc\" (UID: \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\") " pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"
Sep 05 08:00:51.287244 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.287064    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b064f4cc-d9d3-4cf1-9053-0253e730c30c-workload-certs\") pod \"e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc\" (UID: \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\") " pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"
Sep 05 08:00:51.287244 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.287135    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b064f4cc-d9d3-4cf1-9053-0253e730c30c-istiod-ca-cert\") pod \"e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc\" (UID: \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\") " pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"
Sep 05 08:00:51.289052 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.289033    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b064f4cc-d9d3-4cf1-9053-0253e730c30c-istio-envoy\") pod \"e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc\" (UID: \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\") " pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"
Sep 05 08:00:51.289356 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.289336    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b064f4cc-d9d3-4cf1-9053-0253e730c30c-istio-podinfo\") pod \"e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc\" (UID: \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\") " pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"
Sep 05 08:00:51.297710 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.297689    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b064f4cc-d9d3-4cf1-9053-0253e730c30c-istio-token\") pod \"e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc\" (UID: \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\") " pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"
Sep 05 08:00:51.297962 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.297941    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gp776\" (UniqueName: \"kubernetes.io/projected/b064f4cc-d9d3-4cf1-9053-0253e730c30c-kube-api-access-gp776\") pod \"e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc\" (UID: \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\") " pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"
Sep 05 08:00:51.448604 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.448570    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"
Sep 05 08:00:51.568348 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.568313    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:51.730620 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.730547    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07e9fa64-30a4-40a5-9a1a-5b2491af5391" path="/var/lib/kubelet/pods/07e9fa64-30a4-40a5-9a1a-5b2491af5391/volumes"
Sep 05 08:00:51.735142 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.735121    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"]
Sep 05 08:00:51.736716 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:00:51.736693    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb064f4cc_d9d3_4cf1_9053_0253e730c30c.slice/crio-53bca35379406655a6b56effc4ee5765132fc79c68cc49f0b4b9516ddd705944 WatchSource:0}: Error finding container 53bca35379406655a6b56effc4ee5765132fc79c68cc49f0b4b9516ddd705944: Status 404 returned error can't find the container with id 53bca35379406655a6b56effc4ee5765132fc79c68cc49f0b4b9516ddd705944
Sep 05 08:00:51.738807 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.738777    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 05 08:00:51.738890 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.738866    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 05 08:00:51.738936 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.738898    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 05 08:00:51.850421 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.850386    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:51.964138 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:51.964103    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:52.052892 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:52.052782    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:52.057836 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:52.057810    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc" event={"ID":"b064f4cc-d9d3-4cf1-9053-0253e730c30c","Type":"ContainerStarted","Data":"d181cf5896f7a04647402d1f2ae987c8dbef824529c6cbfeada4ae94df4ab67d"}
Sep 05 08:00:52.057997 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:52.057882    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc" event={"ID":"b064f4cc-d9d3-4cf1-9053-0253e730c30c","Type":"ContainerStarted","Data":"53bca35379406655a6b56effc4ee5765132fc79c68cc49f0b4b9516ddd705944"}
Sep 05 08:00:52.083037 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:52.082987    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc" podStartSLOduration=1.082970266 podStartE2EDuration="1.082970266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:00:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:00:52.082527881 +0000 UTC m=+754.852670241" watchObservedRunningTime="2026-09-05 08:00:52.082970266 +0000 UTC m=+754.853112603"
Sep 05 08:00:52.416311 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:52.416277    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:52.449701 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:52.449664    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"
Sep 05 08:00:52.454496 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:52.454472    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"
Sep 05 08:00:52.511154 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:52.511122    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:53.063807 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:53.063778    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"
Sep 05 08:00:53.064590 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:53.064572    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"
Sep 05 08:00:54.103535 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:54.103503    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:54.203350 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:54.203315    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:55.449125 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:55.449092    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:55.549825 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:55.549786    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:56.037750 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:56.037714    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:56.099545 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:56.099518    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:58.669692 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:58.669661    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:58.882443 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:58.882408    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:58.970871 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:58.970772    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:59.071287 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:59.071255    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:00:59.616201 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:00:59.616169    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:00.705204 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:00.705172    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:00.802158 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:00.802122    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:00.925017 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:00.924980    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:01.244414 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:01.244371    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:04.577666 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:04.577621    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:07.228393 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:07.228354    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:07.676496 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:07.676466    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:07.889871 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:07.886949    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"]
Sep 05 08:01:07.903114 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:07.903089    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"
Sep 05 08:01:07.913492 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:07.913288    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-61f3ac-openshift-default-dockercfg-8r9nl\""
Sep 05 08:01:07.915899 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:07.915380    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"]
Sep 05 08:01:07.930093 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:07.930025    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:08.047047 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:08.047006    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3777fc1-aff3-44c1-8da4-0e0e59e45460-istio-data\") pod \"e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t\" (UID: \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\") " pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"
Sep 05 08:01:08.047047 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:08.047049    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3777fc1-aff3-44c1-8da4-0e0e59e45460-workload-certs\") pod \"e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t\" (UID: \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\") " pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"
Sep 05 08:01:08.047275 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:08.047079    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3777fc1-aff3-44c1-8da4-0e0e59e45460-workload-socket\") pod \"e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t\" (UID: \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\") " pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"
Sep 05 08:01:08.047275 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:08.047110    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3777fc1-aff3-44c1-8da4-0e0e59e45460-istio-podinfo\") pod \"e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t\" (UID: \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\") " pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"
Sep 05 08:01:08.047275 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:08.047127    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3777fc1-aff3-44c1-8da4-0e0e59e45460-istio-envoy\") pod \"e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t\" (UID: \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\") " pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"
Sep 05 08:01:08.047275 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:08.047149    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3777fc1-aff3-44c1-8da4-0e0e59e45460-istiod-ca-cert\") pod \"e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t\" (UID: \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\") " pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"
Sep 05 08:01:08.047275 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:08.047167    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5nqj\" (UniqueName: \"kubernetes.io/projected/b3777fc1-aff3-44c1-8da4-0e0e59e45460-kube-api-access-d5nqj\") pod \"e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t\" (UID: \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\") " pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"
Sep 05 08:01:08.047275 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:08.047216    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3777fc1-aff3-44c1-8da4-0e0e59e45460-credential-socket\") pod \"e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t\" (UID: \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\") " pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"
Sep 05 08:01:08.047275 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:08.047251    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3777fc1-aff3-44c1-8da4-0e0e59e45460-istio-token\") pod \"e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t\" (UID: \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\") " pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"
Sep 05 08:01:08.148013 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:08.147976    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3777fc1-aff3-44c1-8da4-0e0e59e45460-istio-data\") pod \"e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t\" (UID: \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\") " pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"
Sep 05 08:01:08.148013 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:08.148010    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3777fc1-aff3-44c1-8da4-0e0e59e45460-workload-certs\") pod \"e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t\" (UID: \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\") " pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"
Sep 05 08:01:08.148256 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:08.148030    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3777fc1-aff3-44c1-8da4-0e0e59e45460-workload-socket\") pod \"e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t\" (UID: \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\") " pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"
Sep 05 08:01:08.148256 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:08.148047    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3777fc1-aff3-44c1-8da4-0e0e59e45460-istio-podinfo\") pod \"e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t\" (UID: \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\") " pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"
Sep 05 08:01:08.148256 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:08.148062    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3777fc1-aff3-44c1-8da4-0e0e59e45460-istio-envoy\") pod \"e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t\" (UID: \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\") " pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"
Sep 05 08:01:08.148256 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:08.148077    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3777fc1-aff3-44c1-8da4-0e0e59e45460-istiod-ca-cert\") pod \"e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t\" (UID: \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\") " pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"
Sep 05 08:01:08.148256 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:08.148101    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5nqj\" (UniqueName: \"kubernetes.io/projected/b3777fc1-aff3-44c1-8da4-0e0e59e45460-kube-api-access-d5nqj\") pod \"e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t\" (UID: \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\") " pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"
Sep 05 08:01:08.148256 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:08.148138    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3777fc1-aff3-44c1-8da4-0e0e59e45460-credential-socket\") pod \"e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t\" (UID: \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\") " pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"
Sep 05 08:01:08.148256 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:08.148162    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3777fc1-aff3-44c1-8da4-0e0e59e45460-istio-token\") pod \"e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t\" (UID: \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\") " pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"
Sep 05 08:01:08.148624 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:08.148355    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3777fc1-aff3-44c1-8da4-0e0e59e45460-workload-certs\") pod \"e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t\" (UID: \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\") " pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"
Sep 05 08:01:08.148624 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:08.148453    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3777fc1-aff3-44c1-8da4-0e0e59e45460-workload-socket\") pod \"e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t\" (UID: \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\") " pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"
Sep 05 08:01:08.148624 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:08.148474    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3777fc1-aff3-44c1-8da4-0e0e59e45460-istio-data\") pod \"e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t\" (UID: \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\") " pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"
Sep 05 08:01:08.148783 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:08.148679    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3777fc1-aff3-44c1-8da4-0e0e59e45460-credential-socket\") pod \"e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t\" (UID: \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\") " pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"
Sep 05 08:01:08.149052 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:08.149030    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3777fc1-aff3-44c1-8da4-0e0e59e45460-istiod-ca-cert\") pod \"e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t\" (UID: \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\") " pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"
Sep 05 08:01:08.150613 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:08.150592    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3777fc1-aff3-44c1-8da4-0e0e59e45460-istio-envoy\") pod \"e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t\" (UID: \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\") " pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"
Sep 05 08:01:08.150711 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:08.150686    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3777fc1-aff3-44c1-8da4-0e0e59e45460-istio-podinfo\") pod \"e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t\" (UID: \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\") " pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"
Sep 05 08:01:08.160627 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:08.160599    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3777fc1-aff3-44c1-8da4-0e0e59e45460-istio-token\") pod \"e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t\" (UID: \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\") " pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"
Sep 05 08:01:08.160742 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:08.160601    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5nqj\" (UniqueName: \"kubernetes.io/projected/b3777fc1-aff3-44c1-8da4-0e0e59e45460-kube-api-access-d5nqj\") pod \"e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t\" (UID: \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\") " pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"
Sep 05 08:01:08.189885 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:08.189746    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:08.222896 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:08.222870    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"
Sep 05 08:01:08.291817 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:08.291637    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:08.412353 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:08.412303    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:08.499535 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:08.499442    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:08.506024 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:08.505991    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"]
Sep 05 08:01:08.514078 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:08.513383    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 05 08:01:08.514078 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:08.513479    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 05 08:01:08.514078 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:08.513522    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 05 08:01:08.881001 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:08.880968    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:09.145871 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:09.145777    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t" event={"ID":"b3777fc1-aff3-44c1-8da4-0e0e59e45460","Type":"ContainerStarted","Data":"cb9ebbec8e35b2a61c531ad77a2471db127dd41327051e775562f5f2d49d8d66"}
Sep 05 08:01:09.145871 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:09.145835    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t" event={"ID":"b3777fc1-aff3-44c1-8da4-0e0e59e45460","Type":"ContainerStarted","Data":"e8380b3588ba36578cdd0ec72b9cea7ebe6ccc736c2be7ac5bb68a0b455aa3dc"}
Sep 05 08:01:09.172339 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:09.172202    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t" podStartSLOduration=2.172187416 podStartE2EDuration="2.172187416s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:01:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:01:09.170702225 +0000 UTC m=+771.940844591" watchObservedRunningTime="2026-09-05 08:01:09.172187416 +0000 UTC m=+771.942329839"
Sep 05 08:01:09.223743 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:09.223707    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"
Sep 05 08:01:09.228883 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:09.228856    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"
Sep 05 08:01:10.150165 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:10.150132    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"
Sep 05 08:01:10.150968 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:10.150947    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"
Sep 05 08:01:11.096075 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:11.096038    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:11.229199 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:11.229168    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:12.428332 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:12.428297    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:12.555078 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:12.555052    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:13.177269 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:13.177236    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:13.260423 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:13.260367    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:13.866298 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:13.866254    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:14.175480 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:14.175409    2563 generic.go:356] "Generic (PLEG): container finished" podID="b064f4cc-d9d3-4cf1-9053-0253e730c30c" containerID="d181cf5896f7a04647402d1f2ae987c8dbef824529c6cbfeada4ae94df4ab67d" exitCode=0
Sep 05 08:01:14.175671 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:14.175472    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc" event={"ID":"b064f4cc-d9d3-4cf1-9053-0253e730c30c","Type":"ContainerDied","Data":"d181cf5896f7a04647402d1f2ae987c8dbef824529c6cbfeada4ae94df4ab67d"}
Sep 05 08:01:14.175926 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:14.175909    2563 scope.go:117] "RemoveContainer" containerID="d181cf5896f7a04647402d1f2ae987c8dbef824529c6cbfeada4ae94df4ab67d"
Sep 05 08:01:14.177039 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:14.177011    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 05 08:01:14.177131 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:14.177086    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 05 08:01:14.177131 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:14.177117    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 05 08:01:14.802881 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:14.802789    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:15.182673 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:15.182644    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc" event={"ID":"b064f4cc-d9d3-4cf1-9053-0253e730c30c","Type":"ContainerStarted","Data":"6f157d21cae858b48dd261c8edd7ecce18be12b55bdc35c28482021dd6c1b928"}
Sep 05 08:01:15.449167 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:15.449088    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"
Sep 05 08:01:15.449301 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:15.449278    2563 patch_prober.go:28] interesting pod/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Sep 05 08:01:15.449341 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:15.449321    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc" podUID="b064f4cc-d9d3-4cf1-9053-0253e730c30c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Sep 05 08:01:15.646129 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:15.646077    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:16.221998 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:16.221966    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73e112cd-dxsng"]
Sep 05 08:01:16.224500 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:16.224483    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73e112cd-dxsng"
Sep 05 08:01:16.226712 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:16.226690    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mpmmp\""
Sep 05 08:01:16.227586 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:16.227570    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 05 08:01:16.234410 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:16.234385    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73e112cd-dxsng"]
Sep 05 08:01:16.322809 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:16.322776    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba2152f0-e62c-45b7-a9fa-6500028ab4cb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-73e112cd-dxsng\" (UID: \"ba2152f0-e62c-45b7-a9fa-6500028ab4cb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73e112cd-dxsng"
Sep 05 08:01:16.357355 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:16.357323    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:16.424240 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:16.424202    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba2152f0-e62c-45b7-a9fa-6500028ab4cb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-73e112cd-dxsng\" (UID: \"ba2152f0-e62c-45b7-a9fa-6500028ab4cb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73e112cd-dxsng"
Sep 05 08:01:16.424774 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:16.424754    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba2152f0-e62c-45b7-a9fa-6500028ab4cb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-73e112cd-dxsng\" (UID: \"ba2152f0-e62c-45b7-a9fa-6500028ab4cb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73e112cd-dxsng"
Sep 05 08:01:16.448056 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:16.448027    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:16.449117 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:16.449093    2563 patch_prober.go:28] interesting pod/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Sep 05 08:01:16.449184 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:16.449163    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc" podUID="b064f4cc-d9d3-4cf1-9053-0253e730c30c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Sep 05 08:01:16.535376 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:16.535284    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73e112cd-dxsng"
Sep 05 08:01:16.818541 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:16.818439    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73e112cd-dxsng"]
Sep 05 08:01:16.821173 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:01:16.821138    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba2152f0_e62c_45b7_a9fa_6500028ab4cb.slice/crio-8ade0064aeb49bcb87be82502a3970bc8166e0e198ceb79e180f6202cde241d1 WatchSource:0}: Error finding container 8ade0064aeb49bcb87be82502a3970bc8166e0e198ceb79e180f6202cde241d1: Status 404 returned error can't find the container with id 8ade0064aeb49bcb87be82502a3970bc8166e0e198ceb79e180f6202cde241d1
Sep 05 08:01:17.004778 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:17.004616    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:17.197385 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:17.197274    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73e112cd-dxsng" event={"ID":"ba2152f0-e62c-45b7-a9fa-6500028ab4cb","Type":"ContainerStarted","Data":"8ade0064aeb49bcb87be82502a3970bc8166e0e198ceb79e180f6202cde241d1"}
Sep 05 08:01:17.449647 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:17.449562    2563 patch_prober.go:28] interesting pod/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Sep 05 08:01:17.449647 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:17.449625    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc" podUID="b064f4cc-d9d3-4cf1-9053-0253e730c30c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Sep 05 08:01:18.449757 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:18.449685    2563 patch_prober.go:28] interesting pod/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Sep 05 08:01:18.449757 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:18.449749    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc" podUID="b064f4cc-d9d3-4cf1-9053-0253e730c30c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Sep 05 08:01:19.083646 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:19.083602    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:19.452117 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:19.452016    2563 patch_prober.go:28] interesting pod/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Sep 05 08:01:19.452117 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:19.452084    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc" podUID="b064f4cc-d9d3-4cf1-9053-0253e730c30c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Sep 05 08:01:20.229059 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:20.228968    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73e112cd-dxsng" event={"ID":"ba2152f0-e62c-45b7-a9fa-6500028ab4cb","Type":"ContainerStarted","Data":"3cc4f6737994b42efa3eec658dfc64ccfdff35a54c112bab7007d834aa86458f"}
Sep 05 08:01:20.248549 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:20.248473    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73e112cd-dxsng" podStartSLOduration=1.421883391 podStartE2EDuration="4.248452104s" totalImagesPullingTime="2.826568713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:01:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:01:20.2446906 +0000 UTC m=+783.014832937" watchObservedRunningTime="2026-09-05 08:01:20.248452104 +0000 UTC m=+783.018594443"
Sep 05 08:01:20.544126 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:20.544086    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"
Sep 05 08:01:20.544629 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:20.544572    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"
Sep 05 08:01:20.545860 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:20.545820    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"
Sep 05 08:01:21.235550 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:21.235517    2563 generic.go:356] "Generic (PLEG): container finished" podID="ba2152f0-e62c-45b7-a9fa-6500028ab4cb" containerID="3cc4f6737994b42efa3eec658dfc64ccfdff35a54c112bab7007d834aa86458f" exitCode=0
Sep 05 08:01:21.235750 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:21.235601    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73e112cd-dxsng" event={"ID":"ba2152f0-e62c-45b7-a9fa-6500028ab4cb","Type":"ContainerDied","Data":"3cc4f6737994b42efa3eec658dfc64ccfdff35a54c112bab7007d834aa86458f"}
Sep 05 08:01:22.416922 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:22.416898    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73e112cd-dxsng"
Sep 05 08:01:22.481479 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:22.481449    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba2152f0-e62c-45b7-a9fa-6500028ab4cb-maas-api-service-ca\") pod \"ba2152f0-e62c-45b7-a9fa-6500028ab4cb\" (UID: \"ba2152f0-e62c-45b7-a9fa-6500028ab4cb\") "
Sep 05 08:01:22.481819 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:22.481793    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba2152f0-e62c-45b7-a9fa-6500028ab4cb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ba2152f0-e62c-45b7-a9fa-6500028ab4cb" (UID: "ba2152f0-e62c-45b7-a9fa-6500028ab4cb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:01:22.582289 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:22.582246    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba2152f0-e62c-45b7-a9fa-6500028ab4cb-maas-api-service-ca\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:01:23.246160 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:23.246130    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73e112cd-dxsng"
Sep 05 08:01:23.246335 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:23.246161    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73e112cd-dxsng" event={"ID":"ba2152f0-e62c-45b7-a9fa-6500028ab4cb","Type":"ContainerDied","Data":"8ade0064aeb49bcb87be82502a3970bc8166e0e198ceb79e180f6202cde241d1"}
Sep 05 08:01:23.246335 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:23.246206    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8ade0064aeb49bcb87be82502a3970bc8166e0e198ceb79e180f6202cde241d1"
Sep 05 08:01:30.285354 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:30.285314    2563 generic.go:356] "Generic (PLEG): container finished" podID="b3777fc1-aff3-44c1-8da4-0e0e59e45460" containerID="cb9ebbec8e35b2a61c531ad77a2471db127dd41327051e775562f5f2d49d8d66" exitCode=0
Sep 05 08:01:30.285873 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:30.285377    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t" event={"ID":"b3777fc1-aff3-44c1-8da4-0e0e59e45460","Type":"ContainerDied","Data":"cb9ebbec8e35b2a61c531ad77a2471db127dd41327051e775562f5f2d49d8d66"}
Sep 05 08:01:30.285873 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:30.285785    2563 scope.go:117] "RemoveContainer" containerID="cb9ebbec8e35b2a61c531ad77a2471db127dd41327051e775562f5f2d49d8d66"
Sep 05 08:01:30.286836 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:30.286809    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 05 08:01:30.286925 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:30.286901    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 05 08:01:30.286962 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:30.286935    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 05 08:01:31.295247 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:31.295210    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t" event={"ID":"b3777fc1-aff3-44c1-8da4-0e0e59e45460","Type":"ContainerStarted","Data":"52bfd21651d0d5b97e039aa579311598693050c77c153ac285e329db4b1e055f"}
Sep 05 08:01:32.140312 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:32.140284    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:32.223314 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:32.223272    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"
Sep 05 08:01:32.223533 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:32.223510    2563 patch_prober.go:28] interesting pod/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Sep 05 08:01:32.223630 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:32.223606    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t" podUID="b3777fc1-aff3-44c1-8da4-0e0e59e45460" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Sep 05 08:01:32.827944 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:32.827914    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:32.914579 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:32.914537    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:33.043055 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:33.043002    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:33.113003 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:33.112912    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73e112cd-dxsng"]
Sep 05 08:01:33.113281 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:33.113265    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73e112cd-dxsng"]
Sep 05 08:01:33.155251 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:33.155219    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:33.223471 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:33.223441    2563 patch_prober.go:28] interesting pod/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Sep 05 08:01:33.223645 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:33.223499    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t" podUID="b3777fc1-aff3-44c1-8da4-0e0e59e45460" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Sep 05 08:01:33.311302 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:33.311249    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:33.668788 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:33.668753    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:33.730450 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:33.730409    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba2152f0-e62c-45b7-a9fa-6500028ab4cb" path="/var/lib/kubelet/pods/ba2152f0-e62c-45b7-a9fa-6500028ab4cb/volumes"
Sep 05 08:01:33.772273 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:33.772243    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:33.871271 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:33.871237    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:33.999729 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:33.999628    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:34.223397 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:34.223366    2563 patch_prober.go:28] interesting pod/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Sep 05 08:01:34.223563 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:34.223419    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t" podUID="b3777fc1-aff3-44c1-8da4-0e0e59e45460" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Sep 05 08:01:34.313363 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:34.313313    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:34.518218 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:34.518186    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:35.224062 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:35.224024    2563 patch_prober.go:28] interesting pod/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Sep 05 08:01:35.224465 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:35.224082    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t" podUID="b3777fc1-aff3-44c1-8da4-0e0e59e45460" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Sep 05 08:01:36.068670 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:36.068640    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:36.223455 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:36.223421    2563 patch_prober.go:28] interesting pod/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Sep 05 08:01:36.223625 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:36.223479    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t" podUID="b3777fc1-aff3-44c1-8da4-0e0e59e45460" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Sep 05 08:01:37.142277 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:37.142244    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:37.227246 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:37.227213    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"
Sep 05 08:01:37.227612 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:37.227591    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"
Sep 05 08:01:37.228290 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:37.228270    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"
Sep 05 08:01:37.241488 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:37.241467    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:37.441538 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:37.441463    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:37.533795 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:37.533764    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:38.044135 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:38.044102    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:38.150756 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:38.150727    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:38.298574 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:38.298489    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:42.898909 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:42.898817    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8"]
Sep 05 08:01:42.899398 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:42.899221    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba2152f0-e62c-45b7-a9fa-6500028ab4cb" containerName="revoke-keys"
Sep 05 08:01:42.899398 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:42.899236    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba2152f0-e62c-45b7-a9fa-6500028ab4cb" containerName="revoke-keys"
Sep 05 08:01:42.899398 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:42.899324    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba2152f0-e62c-45b7-a9fa-6500028ab4cb" containerName="revoke-keys"
Sep 05 08:01:42.901200 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:42.901184    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8"
Sep 05 08:01:42.903524 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:42.903495    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-a-9cecc80d-kserve-self-signed-certs\""
Sep 05 08:01:42.911734 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:42.911710    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8"]
Sep 05 08:01:42.987327 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:42.987295    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f85512e1-9fc1-4325-ad8d-1294116f2bde-tmp-dir\") pod \"e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8\" (UID: \"f85512e1-9fc1-4325-ad8d-1294116f2bde\") " pod="llm/e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8"
Sep 05 08:01:42.987327 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:42.987333    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f85512e1-9fc1-4325-ad8d-1294116f2bde-home\") pod \"e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8\" (UID: \"f85512e1-9fc1-4325-ad8d-1294116f2bde\") " pod="llm/e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8"
Sep 05 08:01:42.987532 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:42.987353    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f85512e1-9fc1-4325-ad8d-1294116f2bde-dshm\") pod \"e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8\" (UID: \"f85512e1-9fc1-4325-ad8d-1294116f2bde\") " pod="llm/e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8"
Sep 05 08:01:42.987532 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:42.987402    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkb2s\" (UniqueName: \"kubernetes.io/projected/f85512e1-9fc1-4325-ad8d-1294116f2bde-kube-api-access-pkb2s\") pod \"e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8\" (UID: \"f85512e1-9fc1-4325-ad8d-1294116f2bde\") " pod="llm/e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8"
Sep 05 08:01:42.987532 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:42.987449    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f85512e1-9fc1-4325-ad8d-1294116f2bde-model-cache\") pod \"e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8\" (UID: \"f85512e1-9fc1-4325-ad8d-1294116f2bde\") " pod="llm/e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8"
Sep 05 08:01:42.987532 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:42.987474    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f85512e1-9fc1-4325-ad8d-1294116f2bde-tls-certs\") pod \"e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8\" (UID: \"f85512e1-9fc1-4325-ad8d-1294116f2bde\") " pod="llm/e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8"
Sep 05 08:01:42.997929 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:42.997892    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:01:43.088084 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:43.088057    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f85512e1-9fc1-4325-ad8d-1294116f2bde-tmp-dir\") pod \"e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8\" (UID: \"f85512e1-9fc1-4325-ad8d-1294116f2bde\") " pod="llm/e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8"
Sep 05 08:01:43.088255 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:43.088089    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f85512e1-9fc1-4325-ad8d-1294116f2bde-home\") pod \"e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8\" (UID: \"f85512e1-9fc1-4325-ad8d-1294116f2bde\") " pod="llm/e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8"
Sep 05 08:01:43.088255 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:43.088115    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f85512e1-9fc1-4325-ad8d-1294116f2bde-dshm\") pod \"e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8\" (UID: \"f85512e1-9fc1-4325-ad8d-1294116f2bde\") " pod="llm/e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8"
Sep 05 08:01:43.088255 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:43.088138    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkb2s\" (UniqueName: \"kubernetes.io/projected/f85512e1-9fc1-4325-ad8d-1294116f2bde-kube-api-access-pkb2s\") pod \"e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8\" (UID: \"f85512e1-9fc1-4325-ad8d-1294116f2bde\") " pod="llm/e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8"
Sep 05 08:01:43.088255 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:43.088173    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f85512e1-9fc1-4325-ad8d-1294116f2bde-model-cache\") pod \"e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8\" (UID: \"f85512e1-9fc1-4325-ad8d-1294116f2bde\") " pod="llm/e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8"
Sep 05 08:01:43.088255 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:43.088204    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f85512e1-9fc1-4325-ad8d-1294116f2bde-tls-certs\") pod \"e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8\" (UID: \"f85512e1-9fc1-4325-ad8d-1294116f2bde\") " pod="llm/e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8"
Sep 05 08:01:43.088533 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:43.088507    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f85512e1-9fc1-4325-ad8d-1294116f2bde-tmp-dir\") pod \"e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8\" (UID: \"f85512e1-9fc1-4325-ad8d-1294116f2bde\") " pod="llm/e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8"
Sep 05 08:01:43.088596 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:43.088551    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f85512e1-9fc1-4325-ad8d-1294116f2bde-home\") pod \"e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8\" (UID: \"f85512e1-9fc1-4325-ad8d-1294116f2bde\") " pod="llm/e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8"
Sep 05 08:01:43.088689 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:43.088670    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f85512e1-9fc1-4325-ad8d-1294116f2bde-model-cache\") pod \"e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8\" (UID: \"f85512e1-9fc1-4325-ad8d-1294116f2bde\") " pod="llm/e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8"
Sep 05 08:01:43.090631 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:43.090607    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f85512e1-9fc1-4325-ad8d-1294116f2bde-dshm\") pod \"e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8\" (UID: \"f85512e1-9fc1-4325-ad8d-1294116f2bde\") " pod="llm/e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8"
Sep 05 08:01:43.090787 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:43.090771    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f85512e1-9fc1-4325-ad8d-1294116f2bde-tls-certs\") pod \"e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8\" (UID: \"f85512e1-9fc1-4325-ad8d-1294116f2bde\") " pod="llm/e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8"
Sep 05 08:01:43.097205 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:43.097183    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkb2s\" (UniqueName: \"kubernetes.io/projected/f85512e1-9fc1-4325-ad8d-1294116f2bde-kube-api-access-pkb2s\") pod \"e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8\" (UID: \"f85512e1-9fc1-4325-ad8d-1294116f2bde\") " pod="llm/e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8"
Sep 05 08:01:43.210713 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:43.210626    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8"
Sep 05 08:01:43.484390 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:43.484363    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8"]
Sep 05 08:01:43.486108 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:01:43.486084    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf85512e1_9fc1_4325_ad8d_1294116f2bde.slice/crio-4fc5e88c1b4461ce59e58e86db000af8da0258b4dd4e22d5e58be29de83c9d51 WatchSource:0}: Error finding container 4fc5e88c1b4461ce59e58e86db000af8da0258b4dd4e22d5e58be29de83c9d51: Status 404 returned error can't find the container with id 4fc5e88c1b4461ce59e58e86db000af8da0258b4dd4e22d5e58be29de83c9d51
Sep 05 08:01:44.370318 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:44.370273    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8" event={"ID":"f85512e1-9fc1-4325-ad8d-1294116f2bde","Type":"ContainerStarted","Data":"333fcc4991907e15f36ac9ea802f88e06b22ca7d51502fa40293578a142caf05"}
Sep 05 08:01:44.370318 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:44.370320    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8" event={"ID":"f85512e1-9fc1-4325-ad8d-1294116f2bde","Type":"ContainerStarted","Data":"4fc5e88c1b4461ce59e58e86db000af8da0258b4dd4e22d5e58be29de83c9d51"}
Sep 05 08:01:44.389930 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:44.389883    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8" podStartSLOduration=2.389867577 podStartE2EDuration="2.389867577s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:01:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:01:44.388416812 +0000 UTC m=+807.158559141" watchObservedRunningTime="2026-09-05 08:01:44.389867577 +0000 UTC m=+807.160009915"
Sep 05 08:01:53.211017 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:53.210974    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8"
Sep 05 08:01:53.211017 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:53.211022    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8"
Sep 05 08:01:53.223705 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:53.223668    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8"
Sep 05 08:01:53.424888 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:01:53.424858    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8"
Sep 05 08:02:02.158039 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:02.157998    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7dd976de-zswcd"]
Sep 05 08:02:02.169742 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:02.169712    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7dd976de-zswcd"
Sep 05 08:02:02.171536 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:02.171500    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7dd976de-zswcd"]
Sep 05 08:02:02.174904 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:02.174883    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 05 08:02:02.175059 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:02.174883    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mpmmp\""
Sep 05 08:02:02.267016 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:02.266982    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c0235c6-bae4-43f2-a3f4-9ce5ab713d65-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7dd976de-zswcd\" (UID: \"9c0235c6-bae4-43f2-a3f4-9ce5ab713d65\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7dd976de-zswcd"
Sep 05 08:02:02.368319 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:02.368286    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c0235c6-bae4-43f2-a3f4-9ce5ab713d65-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7dd976de-zswcd\" (UID: \"9c0235c6-bae4-43f2-a3f4-9ce5ab713d65\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7dd976de-zswcd"
Sep 05 08:02:02.368930 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:02.368910    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c0235c6-bae4-43f2-a3f4-9ce5ab713d65-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7dd976de-zswcd\" (UID: \"9c0235c6-bae4-43f2-a3f4-9ce5ab713d65\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7dd976de-zswcd"
Sep 05 08:02:02.482707 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:02.482626    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7dd976de-zswcd"
Sep 05 08:02:02.768880 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:02.768785    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7dd976de-zswcd"]
Sep 05 08:02:02.772741 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:02:02.772711    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c0235c6_bae4_43f2_a3f4_9ce5ab713d65.slice/crio-9c2c176874a328c2b1dc0b2682fdd2032d5cf3e7fadb8d8f941548a3921c526a WatchSource:0}: Error finding container 9c2c176874a328c2b1dc0b2682fdd2032d5cf3e7fadb8d8f941548a3921c526a: Status 404 returned error can't find the container with id 9c2c176874a328c2b1dc0b2682fdd2032d5cf3e7fadb8d8f941548a3921c526a
Sep 05 08:02:03.438093 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:03.438055    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:02:03.462204 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:03.462172    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7dd976de-zswcd" event={"ID":"9c0235c6-bae4-43f2-a3f4-9ce5ab713d65","Type":"ContainerStarted","Data":"4d697d461f5ba22743c7c41e7f421385bd2cc784f6a0fa0bdedcd4759985114c"}
Sep 05 08:02:03.462348 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:03.462222    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7dd976de-zswcd" event={"ID":"9c0235c6-bae4-43f2-a3f4-9ce5ab713d65","Type":"ContainerStarted","Data":"9c2c176874a328c2b1dc0b2682fdd2032d5cf3e7fadb8d8f941548a3921c526a"}
Sep 05 08:02:03.483617 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:03.483563    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7dd976de-zswcd" podStartSLOduration=1.4835475900000001 podStartE2EDuration="1.48354759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:02:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:02:03.48183434 +0000 UTC m=+826.251976697" watchObservedRunningTime="2026-09-05 08:02:03.48354759 +0000 UTC m=+826.253689926"
Sep 05 08:02:03.540890 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:03.540859    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:02:04.468639 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:04.468608    2563 generic.go:356] "Generic (PLEG): container finished" podID="9c0235c6-bae4-43f2-a3f4-9ce5ab713d65" containerID="4d697d461f5ba22743c7c41e7f421385bd2cc784f6a0fa0bdedcd4759985114c" exitCode=0
Sep 05 08:02:04.469046 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:04.468676    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7dd976de-zswcd" event={"ID":"9c0235c6-bae4-43f2-a3f4-9ce5ab713d65","Type":"ContainerDied","Data":"4d697d461f5ba22743c7c41e7f421385bd2cc784f6a0fa0bdedcd4759985114c"}
Sep 05 08:02:05.609928 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:05.609905    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7dd976de-zswcd"
Sep 05 08:02:05.700203 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:05.700161    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c0235c6-bae4-43f2-a3f4-9ce5ab713d65-maas-api-service-ca\") pod \"9c0235c6-bae4-43f2-a3f4-9ce5ab713d65\" (UID: \"9c0235c6-bae4-43f2-a3f4-9ce5ab713d65\") "
Sep 05 08:02:05.700586 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:05.700560    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c0235c6-bae4-43f2-a3f4-9ce5ab713d65-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9c0235c6-bae4-43f2-a3f4-9ce5ab713d65" (UID: "9c0235c6-bae4-43f2-a3f4-9ce5ab713d65"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:02:05.801199 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:05.801169    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c0235c6-bae4-43f2-a3f4-9ce5ab713d65-maas-api-service-ca\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:02:06.480330 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:06.480302    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7dd976de-zswcd"
Sep 05 08:02:06.480330 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:06.480305    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7dd976de-zswcd" event={"ID":"9c0235c6-bae4-43f2-a3f4-9ce5ab713d65","Type":"ContainerDied","Data":"9c2c176874a328c2b1dc0b2682fdd2032d5cf3e7fadb8d8f941548a3921c526a"}
Sep 05 08:02:06.480536 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:06.480378    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9c2c176874a328c2b1dc0b2682fdd2032d5cf3e7fadb8d8f941548a3921c526a"
Sep 05 08:02:08.848143 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:08.848113    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:02:08.968029 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:08.967991    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:02:09.059967 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:09.059938    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:02:09.112787 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:09.112703    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7dd976de-zswcd"]
Sep 05 08:02:09.115334 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:09.115310    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7dd976de-zswcd"]
Sep 05 08:02:09.156085 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:09.156055    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:02:09.286922 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:09.286879    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:02:09.730142 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:09.730111    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c0235c6-bae4-43f2-a3f4-9ce5ab713d65" path="/var/lib/kubelet/pods/9c0235c6-bae4-43f2-a3f4-9ce5ab713d65/volumes"
Sep 05 08:02:09.749333 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:09.749299    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:02:10.130149 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:10.130119    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:02:10.701383 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:10.701351    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:02:12.291711 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:12.291678    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:02:12.391034 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:12.391002    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:02:13.510732 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:13.510698    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:02:13.607245 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:13.607211    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:02:14.022450 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:14.022419    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:02:14.121341 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:14.121313    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:02:18.989500 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:18.989468    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-4faf97/autores-model-4faf97-kserve-7c948884cf-tjl7r"]
Sep 05 08:02:18.989967 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:18.989836    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c0235c6-bae4-43f2-a3f4-9ce5ab713d65" containerName="revoke-keys"
Sep 05 08:02:18.989967 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:18.989864    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c0235c6-bae4-43f2-a3f4-9ce5ab713d65" containerName="revoke-keys"
Sep 05 08:02:18.989967 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:18.989937    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c0235c6-bae4-43f2-a3f4-9ce5ab713d65" containerName="revoke-keys"
Sep 05 08:02:18.992656 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:18.992638    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-4faf97/autores-model-4faf97-kserve-7c948884cf-tjl7r"
Sep 05 08:02:18.995756 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:18.995734    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-4faf97\"/\"autores-model-4faf97-kserve-self-signed-certs\""
Sep 05 08:02:18.995912 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:18.995778    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-4faf97\"/\"kube-root-ca.crt\""
Sep 05 08:02:18.995912 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:18.995778    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-4faf97\"/\"default-dockercfg-6qjrw\""
Sep 05 08:02:18.995912 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:18.995864    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-4faf97\"/\"openshift-service-ca.crt\""
Sep 05 08:02:19.005041 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:19.005021    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-4faf97/autores-model-4faf97-kserve-7c948884cf-tjl7r"]
Sep 05 08:02:19.123883 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:19.123824    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a46732df-90a2-4003-8a9a-a169571c6f7d-model-cache\") pod \"autores-model-4faf97-kserve-7c948884cf-tjl7r\" (UID: \"a46732df-90a2-4003-8a9a-a169571c6f7d\") " pod="ai-tenant-e2e-autores-4faf97/autores-model-4faf97-kserve-7c948884cf-tjl7r"
Sep 05 08:02:19.124050 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:19.123903    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vh6s2\" (UniqueName: \"kubernetes.io/projected/a46732df-90a2-4003-8a9a-a169571c6f7d-kube-api-access-vh6s2\") pod \"autores-model-4faf97-kserve-7c948884cf-tjl7r\" (UID: \"a46732df-90a2-4003-8a9a-a169571c6f7d\") " pod="ai-tenant-e2e-autores-4faf97/autores-model-4faf97-kserve-7c948884cf-tjl7r"
Sep 05 08:02:19.124050 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:19.123934    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a46732df-90a2-4003-8a9a-a169571c6f7d-dshm\") pod \"autores-model-4faf97-kserve-7c948884cf-tjl7r\" (UID: \"a46732df-90a2-4003-8a9a-a169571c6f7d\") " pod="ai-tenant-e2e-autores-4faf97/autores-model-4faf97-kserve-7c948884cf-tjl7r"
Sep 05 08:02:19.124050 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:19.123968    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a46732df-90a2-4003-8a9a-a169571c6f7d-home\") pod \"autores-model-4faf97-kserve-7c948884cf-tjl7r\" (UID: \"a46732df-90a2-4003-8a9a-a169571c6f7d\") " pod="ai-tenant-e2e-autores-4faf97/autores-model-4faf97-kserve-7c948884cf-tjl7r"
Sep 05 08:02:19.124050 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:19.124010    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a46732df-90a2-4003-8a9a-a169571c6f7d-tls-certs\") pod \"autores-model-4faf97-kserve-7c948884cf-tjl7r\" (UID: \"a46732df-90a2-4003-8a9a-a169571c6f7d\") " pod="ai-tenant-e2e-autores-4faf97/autores-model-4faf97-kserve-7c948884cf-tjl7r"
Sep 05 08:02:19.124189 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:19.124079    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a46732df-90a2-4003-8a9a-a169571c6f7d-tmp-dir\") pod \"autores-model-4faf97-kserve-7c948884cf-tjl7r\" (UID: \"a46732df-90a2-4003-8a9a-a169571c6f7d\") " pod="ai-tenant-e2e-autores-4faf97/autores-model-4faf97-kserve-7c948884cf-tjl7r"
Sep 05 08:02:19.225189 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:19.225152    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vh6s2\" (UniqueName: \"kubernetes.io/projected/a46732df-90a2-4003-8a9a-a169571c6f7d-kube-api-access-vh6s2\") pod \"autores-model-4faf97-kserve-7c948884cf-tjl7r\" (UID: \"a46732df-90a2-4003-8a9a-a169571c6f7d\") " pod="ai-tenant-e2e-autores-4faf97/autores-model-4faf97-kserve-7c948884cf-tjl7r"
Sep 05 08:02:19.225189 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:19.225194    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a46732df-90a2-4003-8a9a-a169571c6f7d-dshm\") pod \"autores-model-4faf97-kserve-7c948884cf-tjl7r\" (UID: \"a46732df-90a2-4003-8a9a-a169571c6f7d\") " pod="ai-tenant-e2e-autores-4faf97/autores-model-4faf97-kserve-7c948884cf-tjl7r"
Sep 05 08:02:19.225420 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:19.225224    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a46732df-90a2-4003-8a9a-a169571c6f7d-home\") pod \"autores-model-4faf97-kserve-7c948884cf-tjl7r\" (UID: \"a46732df-90a2-4003-8a9a-a169571c6f7d\") " pod="ai-tenant-e2e-autores-4faf97/autores-model-4faf97-kserve-7c948884cf-tjl7r"
Sep 05 08:02:19.225420 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:19.225390    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a46732df-90a2-4003-8a9a-a169571c6f7d-tls-certs\") pod \"autores-model-4faf97-kserve-7c948884cf-tjl7r\" (UID: \"a46732df-90a2-4003-8a9a-a169571c6f7d\") " pod="ai-tenant-e2e-autores-4faf97/autores-model-4faf97-kserve-7c948884cf-tjl7r"
Sep 05 08:02:19.225529 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:19.225516    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a46732df-90a2-4003-8a9a-a169571c6f7d-home\") pod \"autores-model-4faf97-kserve-7c948884cf-tjl7r\" (UID: \"a46732df-90a2-4003-8a9a-a169571c6f7d\") " pod="ai-tenant-e2e-autores-4faf97/autores-model-4faf97-kserve-7c948884cf-tjl7r"
Sep 05 08:02:19.225577 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:19.225519    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a46732df-90a2-4003-8a9a-a169571c6f7d-tmp-dir\") pod \"autores-model-4faf97-kserve-7c948884cf-tjl7r\" (UID: \"a46732df-90a2-4003-8a9a-a169571c6f7d\") " pod="ai-tenant-e2e-autores-4faf97/autores-model-4faf97-kserve-7c948884cf-tjl7r"
Sep 05 08:02:19.225692 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:19.225583    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a46732df-90a2-4003-8a9a-a169571c6f7d-model-cache\") pod \"autores-model-4faf97-kserve-7c948884cf-tjl7r\" (UID: \"a46732df-90a2-4003-8a9a-a169571c6f7d\") " pod="ai-tenant-e2e-autores-4faf97/autores-model-4faf97-kserve-7c948884cf-tjl7r"
Sep 05 08:02:19.225767 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:19.225751    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a46732df-90a2-4003-8a9a-a169571c6f7d-tmp-dir\") pod \"autores-model-4faf97-kserve-7c948884cf-tjl7r\" (UID: \"a46732df-90a2-4003-8a9a-a169571c6f7d\") " pod="ai-tenant-e2e-autores-4faf97/autores-model-4faf97-kserve-7c948884cf-tjl7r"
Sep 05 08:02:19.225819 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:19.225783    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a46732df-90a2-4003-8a9a-a169571c6f7d-model-cache\") pod \"autores-model-4faf97-kserve-7c948884cf-tjl7r\" (UID: \"a46732df-90a2-4003-8a9a-a169571c6f7d\") " pod="ai-tenant-e2e-autores-4faf97/autores-model-4faf97-kserve-7c948884cf-tjl7r"
Sep 05 08:02:19.227835 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:19.227809    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a46732df-90a2-4003-8a9a-a169571c6f7d-dshm\") pod \"autores-model-4faf97-kserve-7c948884cf-tjl7r\" (UID: \"a46732df-90a2-4003-8a9a-a169571c6f7d\") " pod="ai-tenant-e2e-autores-4faf97/autores-model-4faf97-kserve-7c948884cf-tjl7r"
Sep 05 08:02:19.228008 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:19.227993    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a46732df-90a2-4003-8a9a-a169571c6f7d-tls-certs\") pod \"autores-model-4faf97-kserve-7c948884cf-tjl7r\" (UID: \"a46732df-90a2-4003-8a9a-a169571c6f7d\") " pod="ai-tenant-e2e-autores-4faf97/autores-model-4faf97-kserve-7c948884cf-tjl7r"
Sep 05 08:02:19.234004 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:19.233980    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vh6s2\" (UniqueName: \"kubernetes.io/projected/a46732df-90a2-4003-8a9a-a169571c6f7d-kube-api-access-vh6s2\") pod \"autores-model-4faf97-kserve-7c948884cf-tjl7r\" (UID: \"a46732df-90a2-4003-8a9a-a169571c6f7d\") " pod="ai-tenant-e2e-autores-4faf97/autores-model-4faf97-kserve-7c948884cf-tjl7r"
Sep 05 08:02:19.297565 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:19.297474    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:02:19.301452 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:19.301434    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-4faf97/autores-model-4faf97-kserve-7c948884cf-tjl7r"
Sep 05 08:02:19.578622 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:19.578589    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-4faf97/autores-model-4faf97-kserve-7c948884cf-tjl7r"]
Sep 05 08:02:19.582302 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:02:19.582275    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda46732df_90a2_4003_8a9a_a169571c6f7d.slice/crio-426b442c3136575ba69474df30221b2a1f24a2d59b0ff7d789a35f414ecef00b WatchSource:0}: Error finding container 426b442c3136575ba69474df30221b2a1f24a2d59b0ff7d789a35f414ecef00b: Status 404 returned error can't find the container with id 426b442c3136575ba69474df30221b2a1f24a2d59b0ff7d789a35f414ecef00b
Sep 05 08:02:20.551067 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:20.551036    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-4faf97/autores-model-4faf97-kserve-7c948884cf-tjl7r" event={"ID":"a46732df-90a2-4003-8a9a-a169571c6f7d","Type":"ContainerStarted","Data":"f86824f02bde19d7e4f9a1d0036289121ac4029f870826da03a21de7e5fab0e4"}
Sep 05 08:02:20.551446 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:20.551075    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-4faf97/autores-model-4faf97-kserve-7c948884cf-tjl7r" event={"ID":"a46732df-90a2-4003-8a9a-a169571c6f7d","Type":"ContainerStarted","Data":"426b442c3136575ba69474df30221b2a1f24a2d59b0ff7d789a35f414ecef00b"}
Sep 05 08:02:20.570292 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:20.570244    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-4faf97/autores-model-4faf97-kserve-7c948884cf-tjl7r" podStartSLOduration=2.570228472 podStartE2EDuration="2.570228472s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:02:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:02:20.569329917 +0000 UTC m=+843.339472266" watchObservedRunningTime="2026-09-05 08:02:20.570228472 +0000 UTC m=+843.340370809"
Sep 05 08:02:29.302087 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:29.302036    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-4faf97/autores-model-4faf97-kserve-7c948884cf-tjl7r"
Sep 05 08:02:29.302514 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:29.302307    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-4faf97/autores-model-4faf97-kserve-7c948884cf-tjl7r"
Sep 05 08:02:29.314835 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:29.314805    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-4faf97/autores-model-4faf97-kserve-7c948884cf-tjl7r"
Sep 05 08:02:29.607161 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:29.607128    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-4faf97/autores-model-4faf97-kserve-7c948884cf-tjl7r"
Sep 05 08:02:38.069943 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:38.069912    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-29f90b97-7qwtj"]
Sep 05 08:02:38.073315 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:38.073298    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-29f90b97-7qwtj"
Sep 05 08:02:38.075700 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:38.075684    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 05 08:02:38.075784 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:38.075684    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mpmmp\""
Sep 05 08:02:38.082042 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:38.082022    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-29f90b97-7qwtj"]
Sep 05 08:02:38.199624 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:38.199590    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ead2c2be-58f6-4b8c-ab11-c83dadd0570f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-29f90b97-7qwtj\" (UID: \"ead2c2be-58f6-4b8c-ab11-c83dadd0570f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-29f90b97-7qwtj"
Sep 05 08:02:38.300998 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:38.300967    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ead2c2be-58f6-4b8c-ab11-c83dadd0570f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-29f90b97-7qwtj\" (UID: \"ead2c2be-58f6-4b8c-ab11-c83dadd0570f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-29f90b97-7qwtj"
Sep 05 08:02:38.301587 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:38.301568    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ead2c2be-58f6-4b8c-ab11-c83dadd0570f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-29f90b97-7qwtj\" (UID: \"ead2c2be-58f6-4b8c-ab11-c83dadd0570f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-29f90b97-7qwtj"
Sep 05 08:02:38.385143 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:38.385069    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-29f90b97-7qwtj"
Sep 05 08:02:38.657660 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:38.657632    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-29f90b97-7qwtj"]
Sep 05 08:02:38.658837 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:02:38.658817    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podead2c2be_58f6_4b8c_ab11_c83dadd0570f.slice/crio-b5af588df646483b45cf0e056f1ce2f04903b2568b816a05a072efdb1c50326b WatchSource:0}: Error finding container b5af588df646483b45cf0e056f1ce2f04903b2568b816a05a072efdb1c50326b: Status 404 returned error can't find the container with id b5af588df646483b45cf0e056f1ce2f04903b2568b816a05a072efdb1c50326b
Sep 05 08:02:38.718521 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:38.718487    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-e7c0ff/test-model-e7c0ff-kserve-5869b4447-4v9hb"]
Sep 05 08:02:38.722784 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:38.722769    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e7c0ff/test-model-e7c0ff-kserve-5869b4447-4v9hb"
Sep 05 08:02:38.726649 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:38.726626    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-e7c0ff\"/\"openshift-service-ca.crt\""
Sep 05 08:02:38.726778 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:38.726671    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-e7c0ff\"/\"kube-root-ca.crt\""
Sep 05 08:02:38.726778 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:38.726698    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-e7c0ff\"/\"default-dockercfg-zhv4t\""
Sep 05 08:02:38.726778 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:38.726680    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-e7c0ff\"/\"test-model-e7c0ff-kserve-self-signed-certs\""
Sep 05 08:02:38.731803 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:38.731784    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-e7c0ff/test-model-e7c0ff-kserve-5869b4447-4v9hb"]
Sep 05 08:02:38.805907 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:38.805867    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c248d\" (UniqueName: \"kubernetes.io/projected/cfcad4e4-896c-4ee9-85db-3d97fb35e934-kube-api-access-c248d\") pod \"test-model-e7c0ff-kserve-5869b4447-4v9hb\" (UID: \"cfcad4e4-896c-4ee9-85db-3d97fb35e934\") " pod="ai-tenant-e2e-inf-a-e7c0ff/test-model-e7c0ff-kserve-5869b4447-4v9hb"
Sep 05 08:02:38.806029 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:38.805958    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfcad4e4-896c-4ee9-85db-3d97fb35e934-home\") pod \"test-model-e7c0ff-kserve-5869b4447-4v9hb\" (UID: \"cfcad4e4-896c-4ee9-85db-3d97fb35e934\") " pod="ai-tenant-e2e-inf-a-e7c0ff/test-model-e7c0ff-kserve-5869b4447-4v9hb"
Sep 05 08:02:38.806076 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:38.806032    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfcad4e4-896c-4ee9-85db-3d97fb35e934-model-cache\") pod \"test-model-e7c0ff-kserve-5869b4447-4v9hb\" (UID: \"cfcad4e4-896c-4ee9-85db-3d97fb35e934\") " pod="ai-tenant-e2e-inf-a-e7c0ff/test-model-e7c0ff-kserve-5869b4447-4v9hb"
Sep 05 08:02:38.806131 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:38.806114    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfcad4e4-896c-4ee9-85db-3d97fb35e934-dshm\") pod \"test-model-e7c0ff-kserve-5869b4447-4v9hb\" (UID: \"cfcad4e4-896c-4ee9-85db-3d97fb35e934\") " pod="ai-tenant-e2e-inf-a-e7c0ff/test-model-e7c0ff-kserve-5869b4447-4v9hb"
Sep 05 08:02:38.806192 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:38.806176    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfcad4e4-896c-4ee9-85db-3d97fb35e934-tmp-dir\") pod \"test-model-e7c0ff-kserve-5869b4447-4v9hb\" (UID: \"cfcad4e4-896c-4ee9-85db-3d97fb35e934\") " pod="ai-tenant-e2e-inf-a-e7c0ff/test-model-e7c0ff-kserve-5869b4447-4v9hb"
Sep 05 08:02:38.806263 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:38.806248    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfcad4e4-896c-4ee9-85db-3d97fb35e934-tls-certs\") pod \"test-model-e7c0ff-kserve-5869b4447-4v9hb\" (UID: \"cfcad4e4-896c-4ee9-85db-3d97fb35e934\") " pod="ai-tenant-e2e-inf-a-e7c0ff/test-model-e7c0ff-kserve-5869b4447-4v9hb"
Sep 05 08:02:38.907336 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:38.907313    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfcad4e4-896c-4ee9-85db-3d97fb35e934-home\") pod \"test-model-e7c0ff-kserve-5869b4447-4v9hb\" (UID: \"cfcad4e4-896c-4ee9-85db-3d97fb35e934\") " pod="ai-tenant-e2e-inf-a-e7c0ff/test-model-e7c0ff-kserve-5869b4447-4v9hb"
Sep 05 08:02:38.907415 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:38.907359    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfcad4e4-896c-4ee9-85db-3d97fb35e934-model-cache\") pod \"test-model-e7c0ff-kserve-5869b4447-4v9hb\" (UID: \"cfcad4e4-896c-4ee9-85db-3d97fb35e934\") " pod="ai-tenant-e2e-inf-a-e7c0ff/test-model-e7c0ff-kserve-5869b4447-4v9hb"
Sep 05 08:02:38.907475 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:38.907459    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfcad4e4-896c-4ee9-85db-3d97fb35e934-dshm\") pod \"test-model-e7c0ff-kserve-5869b4447-4v9hb\" (UID: \"cfcad4e4-896c-4ee9-85db-3d97fb35e934\") " pod="ai-tenant-e2e-inf-a-e7c0ff/test-model-e7c0ff-kserve-5869b4447-4v9hb"
Sep 05 08:02:38.907513 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:38.907503    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfcad4e4-896c-4ee9-85db-3d97fb35e934-tmp-dir\") pod \"test-model-e7c0ff-kserve-5869b4447-4v9hb\" (UID: \"cfcad4e4-896c-4ee9-85db-3d97fb35e934\") " pod="ai-tenant-e2e-inf-a-e7c0ff/test-model-e7c0ff-kserve-5869b4447-4v9hb"
Sep 05 08:02:38.907580 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:38.907565    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfcad4e4-896c-4ee9-85db-3d97fb35e934-tls-certs\") pod \"test-model-e7c0ff-kserve-5869b4447-4v9hb\" (UID: \"cfcad4e4-896c-4ee9-85db-3d97fb35e934\") " pod="ai-tenant-e2e-inf-a-e7c0ff/test-model-e7c0ff-kserve-5869b4447-4v9hb"
Sep 05 08:02:38.907631 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:38.907600    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c248d\" (UniqueName: \"kubernetes.io/projected/cfcad4e4-896c-4ee9-85db-3d97fb35e934-kube-api-access-c248d\") pod \"test-model-e7c0ff-kserve-5869b4447-4v9hb\" (UID: \"cfcad4e4-896c-4ee9-85db-3d97fb35e934\") " pod="ai-tenant-e2e-inf-a-e7c0ff/test-model-e7c0ff-kserve-5869b4447-4v9hb"
Sep 05 08:02:38.907703 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:38.907681    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfcad4e4-896c-4ee9-85db-3d97fb35e934-home\") pod \"test-model-e7c0ff-kserve-5869b4447-4v9hb\" (UID: \"cfcad4e4-896c-4ee9-85db-3d97fb35e934\") " pod="ai-tenant-e2e-inf-a-e7c0ff/test-model-e7c0ff-kserve-5869b4447-4v9hb"
Sep 05 08:02:38.907761 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:38.907716    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfcad4e4-896c-4ee9-85db-3d97fb35e934-model-cache\") pod \"test-model-e7c0ff-kserve-5869b4447-4v9hb\" (UID: \"cfcad4e4-896c-4ee9-85db-3d97fb35e934\") " pod="ai-tenant-e2e-inf-a-e7c0ff/test-model-e7c0ff-kserve-5869b4447-4v9hb"
Sep 05 08:02:38.907928 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:38.907910    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfcad4e4-896c-4ee9-85db-3d97fb35e934-tmp-dir\") pod \"test-model-e7c0ff-kserve-5869b4447-4v9hb\" (UID: \"cfcad4e4-896c-4ee9-85db-3d97fb35e934\") " pod="ai-tenant-e2e-inf-a-e7c0ff/test-model-e7c0ff-kserve-5869b4447-4v9hb"
Sep 05 08:02:38.909922 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:38.909892    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfcad4e4-896c-4ee9-85db-3d97fb35e934-dshm\") pod \"test-model-e7c0ff-kserve-5869b4447-4v9hb\" (UID: \"cfcad4e4-896c-4ee9-85db-3d97fb35e934\") " pod="ai-tenant-e2e-inf-a-e7c0ff/test-model-e7c0ff-kserve-5869b4447-4v9hb"
Sep 05 08:02:38.910093 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:38.910079    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfcad4e4-896c-4ee9-85db-3d97fb35e934-tls-certs\") pod \"test-model-e7c0ff-kserve-5869b4447-4v9hb\" (UID: \"cfcad4e4-896c-4ee9-85db-3d97fb35e934\") " pod="ai-tenant-e2e-inf-a-e7c0ff/test-model-e7c0ff-kserve-5869b4447-4v9hb"
Sep 05 08:02:38.915882 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:38.915862    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c248d\" (UniqueName: \"kubernetes.io/projected/cfcad4e4-896c-4ee9-85db-3d97fb35e934-kube-api-access-c248d\") pod \"test-model-e7c0ff-kserve-5869b4447-4v9hb\" (UID: \"cfcad4e4-896c-4ee9-85db-3d97fb35e934\") " pod="ai-tenant-e2e-inf-a-e7c0ff/test-model-e7c0ff-kserve-5869b4447-4v9hb"
Sep 05 08:02:38.922892 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:38.922859    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:02:39.032996 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:39.032968    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e7c0ff/test-model-e7c0ff-kserve-5869b4447-4v9hb"
Sep 05 08:02:39.307638 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:39.307613    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-e7c0ff/test-model-e7c0ff-kserve-5869b4447-4v9hb"]
Sep 05 08:02:39.309374 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:02:39.309349    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcfcad4e4_896c_4ee9_85db_3d97fb35e934.slice/crio-d243359540bf8258e218a83c6dc271b0d2ea1eea8e528fb12cab4e36975b0f2f WatchSource:0}: Error finding container d243359540bf8258e218a83c6dc271b0d2ea1eea8e528fb12cab4e36975b0f2f: Status 404 returned error can't find the container with id d243359540bf8258e218a83c6dc271b0d2ea1eea8e528fb12cab4e36975b0f2f
Sep 05 08:02:39.647539 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:39.647450    2563 generic.go:356] "Generic (PLEG): container finished" podID="ead2c2be-58f6-4b8c-ab11-c83dadd0570f" containerID="cc9875edd326bcb00e4f49bdb36c4a29200c0029e01402c9d35f7e711a2de3b3" exitCode=0
Sep 05 08:02:39.647539 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:39.647526    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-29f90b97-7qwtj" event={"ID":"ead2c2be-58f6-4b8c-ab11-c83dadd0570f","Type":"ContainerDied","Data":"cc9875edd326bcb00e4f49bdb36c4a29200c0029e01402c9d35f7e711a2de3b3"}
Sep 05 08:02:39.647737 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:39.647568    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-29f90b97-7qwtj" event={"ID":"ead2c2be-58f6-4b8c-ab11-c83dadd0570f","Type":"ContainerStarted","Data":"b5af588df646483b45cf0e056f1ce2f04903b2568b816a05a072efdb1c50326b"}
Sep 05 08:02:39.648982 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:39.648958    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e7c0ff/test-model-e7c0ff-kserve-5869b4447-4v9hb" event={"ID":"cfcad4e4-896c-4ee9-85db-3d97fb35e934","Type":"ContainerStarted","Data":"1ea0f5b9ebceaec467c26445df9af0fa2ca416669c128001e7cfbf754ab4bc9f"}
Sep 05 08:02:39.649064 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:39.648990    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e7c0ff/test-model-e7c0ff-kserve-5869b4447-4v9hb" event={"ID":"cfcad4e4-896c-4ee9-85db-3d97fb35e934","Type":"ContainerStarted","Data":"d243359540bf8258e218a83c6dc271b0d2ea1eea8e528fb12cab4e36975b0f2f"}
Sep 05 08:02:39.683900 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:39.683829    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-e7c0ff/test-model-e7c0ff-kserve-5869b4447-4v9hb" podStartSLOduration=1.6838156149999999 podStartE2EDuration="1.683815615s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:02:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:02:39.682658791 +0000 UTC m=+862.452801148" watchObservedRunningTime="2026-09-05 08:02:39.683815615 +0000 UTC m=+862.453957952"
Sep 05 08:02:40.791360 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:40.791332    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-29f90b97-7qwtj"
Sep 05 08:02:40.926178 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:40.926078    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ead2c2be-58f6-4b8c-ab11-c83dadd0570f-maas-api-service-ca\") pod \"ead2c2be-58f6-4b8c-ab11-c83dadd0570f\" (UID: \"ead2c2be-58f6-4b8c-ab11-c83dadd0570f\") "
Sep 05 08:02:40.926487 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:40.926461    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ead2c2be-58f6-4b8c-ab11-c83dadd0570f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ead2c2be-58f6-4b8c-ab11-c83dadd0570f" (UID: "ead2c2be-58f6-4b8c-ab11-c83dadd0570f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:02:41.027452 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:41.027403    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ead2c2be-58f6-4b8c-ab11-c83dadd0570f-maas-api-service-ca\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:02:41.661074 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:41.661044    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-29f90b97-7qwtj"
Sep 05 08:02:41.661248 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:41.661075    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-29f90b97-7qwtj" event={"ID":"ead2c2be-58f6-4b8c-ab11-c83dadd0570f","Type":"ContainerDied","Data":"b5af588df646483b45cf0e056f1ce2f04903b2568b816a05a072efdb1c50326b"}
Sep 05 08:02:41.661248 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:41.661146    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b5af588df646483b45cf0e056f1ce2f04903b2568b816a05a072efdb1c50326b"
Sep 05 08:02:44.039888 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:44.039837    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:02:44.120451 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:44.120413    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:02:44.199317 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:44.199285    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:02:44.260835 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:44.260797    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-29f90b97-7qwtj"]
Sep 05 08:02:44.263976 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:44.263953    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-29f90b97-7qwtj"]
Sep 05 08:02:44.270553 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:44.270532    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:02:45.469011 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:45.468969    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:02:45.730726 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:45.730642    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ead2c2be-58f6-4b8c-ab11-c83dadd0570f" path="/var/lib/kubelet/pods/ead2c2be-58f6-4b8c-ab11-c83dadd0570f/volumes"
Sep 05 08:02:45.937392 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:45.937340    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:02:46.279117 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:46.279067    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:02:46.396048 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:46.395989    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:02:48.521554 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:48.521507    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:02:48.620057 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:48.620028    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:02:49.034156 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:49.034128    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-e7c0ff/test-model-e7c0ff-kserve-5869b4447-4v9hb"
Sep 05 08:02:49.034448 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:49.034423    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-e7c0ff/test-model-e7c0ff-kserve-5869b4447-4v9hb"
Sep 05 08:02:49.046723 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:49.046702    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-e7c0ff/test-model-e7c0ff-kserve-5869b4447-4v9hb"
Sep 05 08:02:49.711726 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:49.711693    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-e7c0ff/test-model-e7c0ff-kserve-5869b4447-4v9hb"
Sep 05 08:02:49.854405 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:49.854369    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:02:49.944816 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:49.944783    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:02:50.150687 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:50.150647    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:02:50.540572 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:50.540494    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:02:50.649975 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:50.649936    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:02:52.733941 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:52.733907    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-4faf97-tk79l"]
Sep 05 08:02:52.734308 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:52.734283    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ead2c2be-58f6-4b8c-ab11-c83dadd0570f" containerName="revoke-keys"
Sep 05 08:02:52.734308 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:52.734294    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ead2c2be-58f6-4b8c-ab11-c83dadd0570f" containerName="revoke-keys"
Sep 05 08:02:52.734379 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:52.734354    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ead2c2be-58f6-4b8c-ab11-c83dadd0570f" containerName="revoke-keys"
Sep 05 08:02:52.737583 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:52.737565    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-4faf97-tk79l"
Sep 05 08:02:52.739999 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:52.739953    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 05 08:02:52.740125 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:52.740046    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mpmmp\""
Sep 05 08:02:52.746144 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:52.746121    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-4faf97-tk79l"]
Sep 05 08:02:52.837566 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:52.837534    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:02:52.844823 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:52.844785    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b06c94c5-8d89-45c1-a48e-363fff533afa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-4faf97-tk79l\" (UID: \"b06c94c5-8d89-45c1-a48e-363fff533afa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-4faf97-tk79l"
Sep 05 08:02:52.945709 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:52.945658    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b06c94c5-8d89-45c1-a48e-363fff533afa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-4faf97-tk79l\" (UID: \"b06c94c5-8d89-45c1-a48e-363fff533afa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-4faf97-tk79l"
Sep 05 08:02:52.946449 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:52.946427    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b06c94c5-8d89-45c1-a48e-363fff533afa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-4faf97-tk79l\" (UID: \"b06c94c5-8d89-45c1-a48e-363fff533afa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-4faf97-tk79l"
Sep 05 08:02:53.048671 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:53.048587    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-4faf97-tk79l"
Sep 05 08:02:53.324909 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:53.324881    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-4faf97-tk79l"]
Sep 05 08:02:53.326464 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:02:53.326433    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb06c94c5_8d89_45c1_a48e_363fff533afa.slice/crio-2c7dd0e65ab842bca85421424b1a535ebd3da35de75c4338fe84999c303c44c7 WatchSource:0}: Error finding container 2c7dd0e65ab842bca85421424b1a535ebd3da35de75c4338fe84999c303c44c7: Status 404 returned error can't find the container with id 2c7dd0e65ab842bca85421424b1a535ebd3da35de75c4338fe84999c303c44c7
Sep 05 08:02:53.721537 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:53.721442    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-4faf97-tk79l" event={"ID":"b06c94c5-8d89-45c1-a48e-363fff533afa","Type":"ContainerStarted","Data":"86556cd8b6737014c0b107dd15164bc684a04c6e0e8366fa43d95145f4413415"}
Sep 05 08:02:53.721537 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:53.721493    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-4faf97-tk79l" event={"ID":"b06c94c5-8d89-45c1-a48e-363fff533afa","Type":"ContainerStarted","Data":"2c7dd0e65ab842bca85421424b1a535ebd3da35de75c4338fe84999c303c44c7"}
Sep 05 08:02:53.742540 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:53.742489    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-4faf97-tk79l" podStartSLOduration=1.7424728310000002 podStartE2EDuration="1.742472831s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:02:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:02:53.74161762 +0000 UTC m=+876.511759957" watchObservedRunningTime="2026-09-05 08:02:53.742472831 +0000 UTC m=+876.512615167"
Sep 05 08:02:54.727665 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:54.727628    2563 generic.go:356] "Generic (PLEG): container finished" podID="b06c94c5-8d89-45c1-a48e-363fff533afa" containerID="86556cd8b6737014c0b107dd15164bc684a04c6e0e8366fa43d95145f4413415" exitCode=0
Sep 05 08:02:54.727898 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:54.727675    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-4faf97-tk79l" event={"ID":"b06c94c5-8d89-45c1-a48e-363fff533afa","Type":"ContainerDied","Data":"86556cd8b6737014c0b107dd15164bc684a04c6e0e8366fa43d95145f4413415"}
Sep 05 08:02:55.866568 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:55.866541    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-4faf97-tk79l"
Sep 05 08:02:55.974310 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:55.974283    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b06c94c5-8d89-45c1-a48e-363fff533afa-maas-api-service-ca\") pod \"b06c94c5-8d89-45c1-a48e-363fff533afa\" (UID: \"b06c94c5-8d89-45c1-a48e-363fff533afa\") "
Sep 05 08:02:55.974646 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:55.974622    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b06c94c5-8d89-45c1-a48e-363fff533afa-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b06c94c5-8d89-45c1-a48e-363fff533afa" (UID: "b06c94c5-8d89-45c1-a48e-363fff533afa"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:02:56.075781 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:56.075732    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b06c94c5-8d89-45c1-a48e-363fff533afa-maas-api-service-ca\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:02:56.742676 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:56.742646    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-4faf97-tk79l"
Sep 05 08:02:56.742676 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:56.742672    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-4faf97-tk79l" event={"ID":"b06c94c5-8d89-45c1-a48e-363fff533afa","Type":"ContainerDied","Data":"2c7dd0e65ab842bca85421424b1a535ebd3da35de75c4338fe84999c303c44c7"}
Sep 05 08:02:56.742965 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:56.742713    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2c7dd0e65ab842bca85421424b1a535ebd3da35de75c4338fe84999c303c44c7"
Sep 05 08:02:58.640185 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:02:58.640152    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:03:01.370716 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:01.370674    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:03:02.960185 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:02.960156    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:03:03.650566 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:03.650516    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:03:03.763141 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:03.763113    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:03:03.830576 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:03.830543    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:03:03.945219 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:03.945134    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:03:04.031333 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:04.031297    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-4faf97-tk79l"]
Sep 05 08:03:04.035667 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:04.035642    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-4faf97-tk79l"]
Sep 05 08:03:04.109775 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:04.109748    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:03:04.371240 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:04.371209    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:03:05.729816 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:05.729781    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b06c94c5-8d89-45c1-a48e-363fff533afa" path="/var/lib/kubelet/pods/b06c94c5-8d89-45c1-a48e-363fff533afa/volumes"
Sep 05 08:03:09.351296 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:09.351211    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-4faf97/autores-model-4faf97-kserve-7c948884cf-tjl7r"]
Sep 05 08:03:09.448885 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:09.448838    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:03:11.028329 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:11.028292    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:03:13.386316 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:13.386280    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-46865726-jlttq"]
Sep 05 08:03:13.386683 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:13.386666    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b06c94c5-8d89-45c1-a48e-363fff533afa" containerName="revoke-keys"
Sep 05 08:03:13.386683 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:13.386677    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b06c94c5-8d89-45c1-a48e-363fff533afa" containerName="revoke-keys"
Sep 05 08:03:13.386762 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:13.386729    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b06c94c5-8d89-45c1-a48e-363fff533afa" containerName="revoke-keys"
Sep 05 08:03:13.389396 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:13.389381    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-46865726-jlttq"
Sep 05 08:03:13.391869 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:13.391834    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mpmmp\""
Sep 05 08:03:13.392075 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:13.392054    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 05 08:03:13.402233 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:13.402213    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-46865726-jlttq"]
Sep 05 08:03:13.447283 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:13.447255    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3700ad6f-a86f-4d93-9889-df51f3ae03e4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-46865726-jlttq\" (UID: \"3700ad6f-a86f-4d93-9889-df51f3ae03e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-46865726-jlttq"
Sep 05 08:03:13.548429 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:13.548387    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3700ad6f-a86f-4d93-9889-df51f3ae03e4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-46865726-jlttq\" (UID: \"3700ad6f-a86f-4d93-9889-df51f3ae03e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-46865726-jlttq"
Sep 05 08:03:13.549012 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:13.548992    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3700ad6f-a86f-4d93-9889-df51f3ae03e4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-46865726-jlttq\" (UID: \"3700ad6f-a86f-4d93-9889-df51f3ae03e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-46865726-jlttq"
Sep 05 08:03:13.699331 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:13.699250    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-46865726-jlttq"
Sep 05 08:03:13.974870 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:13.974830    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-46865726-jlttq"]
Sep 05 08:03:13.977470 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:03:13.977443    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3700ad6f_a86f_4d93_9889_df51f3ae03e4.slice/crio-ad41d6044b18dd58d458b4cd7282a52ff6b125cd8f08196c2ec6e4d5b10f7e61 WatchSource:0}: Error finding container ad41d6044b18dd58d458b4cd7282a52ff6b125cd8f08196c2ec6e4d5b10f7e61: Status 404 returned error can't find the container with id ad41d6044b18dd58d458b4cd7282a52ff6b125cd8f08196c2ec6e4d5b10f7e61
Sep 05 08:03:14.833480 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:14.833442    2563 generic.go:356] "Generic (PLEG): container finished" podID="3700ad6f-a86f-4d93-9889-df51f3ae03e4" containerID="5d9db85608f713c0681bb29fdeb381ef244e821789552374198890e1b684d5d0" exitCode=0
Sep 05 08:03:14.833934 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:14.833481    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-46865726-jlttq" event={"ID":"3700ad6f-a86f-4d93-9889-df51f3ae03e4","Type":"ContainerDied","Data":"5d9db85608f713c0681bb29fdeb381ef244e821789552374198890e1b684d5d0"}
Sep 05 08:03:14.833934 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:14.833529    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-46865726-jlttq" event={"ID":"3700ad6f-a86f-4d93-9889-df51f3ae03e4","Type":"ContainerStarted","Data":"ad41d6044b18dd58d458b4cd7282a52ff6b125cd8f08196c2ec6e4d5b10f7e61"}
Sep 05 08:03:15.970379 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:15.970350    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-46865726-jlttq"
Sep 05 08:03:16.073560 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:16.073527    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3700ad6f-a86f-4d93-9889-df51f3ae03e4-maas-api-service-ca\") pod \"3700ad6f-a86f-4d93-9889-df51f3ae03e4\" (UID: \"3700ad6f-a86f-4d93-9889-df51f3ae03e4\") "
Sep 05 08:03:16.073945 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:16.073918    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3700ad6f-a86f-4d93-9889-df51f3ae03e4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3700ad6f-a86f-4d93-9889-df51f3ae03e4" (UID: "3700ad6f-a86f-4d93-9889-df51f3ae03e4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:03:16.175270 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:16.175190    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3700ad6f-a86f-4d93-9889-df51f3ae03e4-maas-api-service-ca\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:03:16.847227 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:16.847185    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-46865726-jlttq" event={"ID":"3700ad6f-a86f-4d93-9889-df51f3ae03e4","Type":"ContainerDied","Data":"ad41d6044b18dd58d458b4cd7282a52ff6b125cd8f08196c2ec6e4d5b10f7e61"}
Sep 05 08:03:16.847428 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:16.847292    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ad41d6044b18dd58d458b4cd7282a52ff6b125cd8f08196c2ec6e4d5b10f7e61"
Sep 05 08:03:16.847428 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:16.847372    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-46865726-jlttq"
Sep 05 08:03:17.705994 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:17.705965    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2bzp8_569cffae-1c5c-4b02-af57-9ca53c455e4c/ovn-acl-logging/0.log"
Sep 05 08:03:17.706401 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:17.706238    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2bzp8_569cffae-1c5c-4b02-af57-9ca53c455e4c/ovn-acl-logging/0.log"
Sep 05 08:03:20.409796 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:20.409759    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:03:20.488444 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:20.488406    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:03:20.589439 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:20.589402    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:03:20.638246 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:20.638202    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-46865726-jlttq"]
Sep 05 08:03:20.645882 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:20.643483    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-46865726-jlttq"]
Sep 05 08:03:20.691516 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:20.691413    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:03:20.820706 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:20.820670    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:03:21.520829 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:21.520791    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:03:21.607313 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:21.607258    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:03:21.703357 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:21.703321    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:03:21.730585 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:21.730551    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3700ad6f-a86f-4d93-9889-df51f3ae03e4" path="/var/lib/kubelet/pods/3700ad6f-a86f-4d93-9889-df51f3ae03e4/volumes"
Sep 05 08:03:23.146065 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:23.146032    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:03:24.373836 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:24.373791    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-4faf97/autores-model-4faf97-kserve-7c948884cf-tjl7r" podUID="a46732df-90a2-4003-8a9a-a169571c6f7d" containerName="main" containerID="cri-o://f86824f02bde19d7e4f9a1d0036289121ac4029f870826da03a21de7e5fab0e4" gracePeriod=45
Sep 05 08:03:24.623463 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:24.623440    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-4faf97/autores-model-4faf97-kserve-7c948884cf-tjl7r"
Sep 05 08:03:24.756618 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:24.756536    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vh6s2\" (UniqueName: \"kubernetes.io/projected/a46732df-90a2-4003-8a9a-a169571c6f7d-kube-api-access-vh6s2\") pod \"a46732df-90a2-4003-8a9a-a169571c6f7d\" (UID: \"a46732df-90a2-4003-8a9a-a169571c6f7d\") "
Sep 05 08:03:24.756618 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:24.756596    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a46732df-90a2-4003-8a9a-a169571c6f7d-model-cache\") pod \"a46732df-90a2-4003-8a9a-a169571c6f7d\" (UID: \"a46732df-90a2-4003-8a9a-a169571c6f7d\") "
Sep 05 08:03:24.756876 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:24.756621    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a46732df-90a2-4003-8a9a-a169571c6f7d-dshm\") pod \"a46732df-90a2-4003-8a9a-a169571c6f7d\" (UID: \"a46732df-90a2-4003-8a9a-a169571c6f7d\") "
Sep 05 08:03:24.756876 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:24.756715    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a46732df-90a2-4003-8a9a-a169571c6f7d-home\") pod \"a46732df-90a2-4003-8a9a-a169571c6f7d\" (UID: \"a46732df-90a2-4003-8a9a-a169571c6f7d\") "
Sep 05 08:03:24.756876 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:24.756747    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a46732df-90a2-4003-8a9a-a169571c6f7d-tls-certs\") pod \"a46732df-90a2-4003-8a9a-a169571c6f7d\" (UID: \"a46732df-90a2-4003-8a9a-a169571c6f7d\") "
Sep 05 08:03:24.756876 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:24.756786    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a46732df-90a2-4003-8a9a-a169571c6f7d-tmp-dir\") pod \"a46732df-90a2-4003-8a9a-a169571c6f7d\" (UID: \"a46732df-90a2-4003-8a9a-a169571c6f7d\") "
Sep 05 08:03:24.757107 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:24.756930    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a46732df-90a2-4003-8a9a-a169571c6f7d-model-cache" (OuterVolumeSpecName: "model-cache") pod "a46732df-90a2-4003-8a9a-a169571c6f7d" (UID: "a46732df-90a2-4003-8a9a-a169571c6f7d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:03:24.757107 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:24.756992    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a46732df-90a2-4003-8a9a-a169571c6f7d-home" (OuterVolumeSpecName: "home") pod "a46732df-90a2-4003-8a9a-a169571c6f7d" (UID: "a46732df-90a2-4003-8a9a-a169571c6f7d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:03:24.757107 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:24.757072    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a46732df-90a2-4003-8a9a-a169571c6f7d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a46732df-90a2-4003-8a9a-a169571c6f7d" (UID: "a46732df-90a2-4003-8a9a-a169571c6f7d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:03:24.757424 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:24.757184    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a46732df-90a2-4003-8a9a-a169571c6f7d-home\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:03:24.757424 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:24.757212    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a46732df-90a2-4003-8a9a-a169571c6f7d-tmp-dir\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:03:24.757424 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:24.757221    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a46732df-90a2-4003-8a9a-a169571c6f7d-model-cache\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:03:24.759004 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:24.758979    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a46732df-90a2-4003-8a9a-a169571c6f7d-dshm" (OuterVolumeSpecName: "dshm") pod "a46732df-90a2-4003-8a9a-a169571c6f7d" (UID: "a46732df-90a2-4003-8a9a-a169571c6f7d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:03:24.759106 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:24.759019    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a46732df-90a2-4003-8a9a-a169571c6f7d-kube-api-access-vh6s2" (OuterVolumeSpecName: "kube-api-access-vh6s2") pod "a46732df-90a2-4003-8a9a-a169571c6f7d" (UID: "a46732df-90a2-4003-8a9a-a169571c6f7d"). InnerVolumeSpecName "kube-api-access-vh6s2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:03:24.759106 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:24.759082    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a46732df-90a2-4003-8a9a-a169571c6f7d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a46732df-90a2-4003-8a9a-a169571c6f7d" (UID: "a46732df-90a2-4003-8a9a-a169571c6f7d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:03:24.858348 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:24.858319    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a46732df-90a2-4003-8a9a-a169571c6f7d-tls-certs\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:03:24.858348 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:24.858346    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vh6s2\" (UniqueName: \"kubernetes.io/projected/a46732df-90a2-4003-8a9a-a169571c6f7d-kube-api-access-vh6s2\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:03:24.858348 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:24.858355    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a46732df-90a2-4003-8a9a-a169571c6f7d-dshm\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:03:24.888490 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:24.888458    2563 generic.go:356] "Generic (PLEG): container finished" podID="a46732df-90a2-4003-8a9a-a169571c6f7d" containerID="f86824f02bde19d7e4f9a1d0036289121ac4029f870826da03a21de7e5fab0e4" exitCode=0
Sep 05 08:03:24.888631 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:24.888499    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-4faf97/autores-model-4faf97-kserve-7c948884cf-tjl7r"
Sep 05 08:03:24.888631 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:24.888514    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-4faf97/autores-model-4faf97-kserve-7c948884cf-tjl7r" event={"ID":"a46732df-90a2-4003-8a9a-a169571c6f7d","Type":"ContainerDied","Data":"f86824f02bde19d7e4f9a1d0036289121ac4029f870826da03a21de7e5fab0e4"}
Sep 05 08:03:24.888631 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:24.888547    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-4faf97/autores-model-4faf97-kserve-7c948884cf-tjl7r" event={"ID":"a46732df-90a2-4003-8a9a-a169571c6f7d","Type":"ContainerDied","Data":"426b442c3136575ba69474df30221b2a1f24a2d59b0ff7d789a35f414ecef00b"}
Sep 05 08:03:24.888631 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:24.888564    2563 scope.go:117] "RemoveContainer" containerID="f86824f02bde19d7e4f9a1d0036289121ac4029f870826da03a21de7e5fab0e4"
Sep 05 08:03:24.897605 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:24.897589    2563 scope.go:117] "RemoveContainer" containerID="f86824f02bde19d7e4f9a1d0036289121ac4029f870826da03a21de7e5fab0e4"
Sep 05 08:03:24.897885 ip-10-0-133-14 kubenswrapper[2563]: E0905 08:03:24.897862    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f86824f02bde19d7e4f9a1d0036289121ac4029f870826da03a21de7e5fab0e4\": container with ID starting with f86824f02bde19d7e4f9a1d0036289121ac4029f870826da03a21de7e5fab0e4 not found: ID does not exist" containerID="f86824f02bde19d7e4f9a1d0036289121ac4029f870826da03a21de7e5fab0e4"
Sep 05 08:03:24.897987 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:24.897895    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f86824f02bde19d7e4f9a1d0036289121ac4029f870826da03a21de7e5fab0e4"} err="failed to get container status \"f86824f02bde19d7e4f9a1d0036289121ac4029f870826da03a21de7e5fab0e4\": rpc error: code = NotFound desc = could not find container \"f86824f02bde19d7e4f9a1d0036289121ac4029f870826da03a21de7e5fab0e4\": container with ID starting with f86824f02bde19d7e4f9a1d0036289121ac4029f870826da03a21de7e5fab0e4 not found: ID does not exist"
Sep 05 08:03:24.910206 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:24.910176    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-4faf97/autores-model-4faf97-kserve-7c948884cf-tjl7r"]
Sep 05 08:03:24.916184 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:24.916161    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-4faf97/autores-model-4faf97-kserve-7c948884cf-tjl7r"]
Sep 05 08:03:25.737960 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:25.737919    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a46732df-90a2-4003-8a9a-a169571c6f7d" path="/var/lib/kubelet/pods/a46732df-90a2-4003-8a9a-a169571c6f7d/volumes"
Sep 05 08:03:29.200213 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:29.200184    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:03:29.343906 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:29.343876    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:03:29.551093 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:29.551003    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:03:29.687860 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:29.687807    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:03:30.024068 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:30.024036    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:03:43.567774 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:43.567711    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:03:46.405307 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:46.405277    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:03:46.538468 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:46.538429    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:03:47.049033 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:47.049003    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:03:55.935627 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:03:55.935586    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:00.456722 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:00.456685    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:02.108205 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:02.108173    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:02.183624 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:02.183590    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:02.289078 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:02.289041    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:09.951550 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:09.951520    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:10.042923 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:10.042894    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:10.186730 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:10.186699    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"]
Sep 05 08:04:10.186962 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:10.186941    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc" podUID="b064f4cc-d9d3-4cf1-9053-0253e730c30c" containerName="istio-proxy" containerID="cri-o://6f157d21cae858b48dd261c8edd7ecce18be12b55bdc35c28482021dd6c1b928" gracePeriod=30
Sep 05 08:04:10.209902 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:10.209794    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:10.677639 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:10.677605    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:10.979346 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:10.979264    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:11.347649 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:11.347616    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:14.564017 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:14.563983    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:14.653266 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:14.653233    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:15.451118 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:15.451063    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"
Sep 05 08:04:15.632381 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:15.632350    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b064f4cc-d9d3-4cf1-9053-0253e730c30c-istio-envoy\") pod \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\" (UID: \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\") "
Sep 05 08:04:15.632836 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:15.632423    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b064f4cc-d9d3-4cf1-9053-0253e730c30c-istiod-ca-cert\") pod \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\" (UID: \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\") "
Sep 05 08:04:15.632836 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:15.632459    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b064f4cc-d9d3-4cf1-9053-0253e730c30c-istio-token\") pod \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\" (UID: \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\") "
Sep 05 08:04:15.632836 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:15.632486    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b064f4cc-d9d3-4cf1-9053-0253e730c30c-workload-socket\") pod \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\" (UID: \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\") "
Sep 05 08:04:15.632836 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:15.632522    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gp776\" (UniqueName: \"kubernetes.io/projected/b064f4cc-d9d3-4cf1-9053-0253e730c30c-kube-api-access-gp776\") pod \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\" (UID: \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\") "
Sep 05 08:04:15.632836 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:15.632552    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b064f4cc-d9d3-4cf1-9053-0253e730c30c-workload-certs\") pod \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\" (UID: \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\") "
Sep 05 08:04:15.632836 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:15.632619    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b064f4cc-d9d3-4cf1-9053-0253e730c30c-istio-data\") pod \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\" (UID: \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\") "
Sep 05 08:04:15.632836 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:15.632651    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b064f4cc-d9d3-4cf1-9053-0253e730c30c-istio-podinfo\") pod \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\" (UID: \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\") "
Sep 05 08:04:15.632836 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:15.632678    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b064f4cc-d9d3-4cf1-9053-0253e730c30c-credential-socket\") pod \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\" (UID: \"b064f4cc-d9d3-4cf1-9053-0253e730c30c\") "
Sep 05 08:04:15.633487 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:15.632826    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b064f4cc-d9d3-4cf1-9053-0253e730c30c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b064f4cc-d9d3-4cf1-9053-0253e730c30c" (UID: "b064f4cc-d9d3-4cf1-9053-0253e730c30c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:04:15.633487 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:15.632872    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b064f4cc-d9d3-4cf1-9053-0253e730c30c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b064f4cc-d9d3-4cf1-9053-0253e730c30c" (UID: "b064f4cc-d9d3-4cf1-9053-0253e730c30c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:04:15.633487 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:15.632904    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b064f4cc-d9d3-4cf1-9053-0253e730c30c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b064f4cc-d9d3-4cf1-9053-0253e730c30c" (UID: "b064f4cc-d9d3-4cf1-9053-0253e730c30c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:04:15.633487 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:15.633076    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b064f4cc-d9d3-4cf1-9053-0253e730c30c-istiod-ca-cert\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:04:15.633487 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:15.633094    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b064f4cc-d9d3-4cf1-9053-0253e730c30c-workload-socket\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:04:15.633487 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:15.633096    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b064f4cc-d9d3-4cf1-9053-0253e730c30c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b064f4cc-d9d3-4cf1-9053-0253e730c30c" (UID: "b064f4cc-d9d3-4cf1-9053-0253e730c30c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:04:15.633487 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:15.633107    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b064f4cc-d9d3-4cf1-9053-0253e730c30c-workload-certs\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:04:15.633487 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:15.633307    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b064f4cc-d9d3-4cf1-9053-0253e730c30c-istio-data" (OuterVolumeSpecName: "istio-data") pod "b064f4cc-d9d3-4cf1-9053-0253e730c30c" (UID: "b064f4cc-d9d3-4cf1-9053-0253e730c30c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:04:15.634932 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:15.634907    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b064f4cc-d9d3-4cf1-9053-0253e730c30c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b064f4cc-d9d3-4cf1-9053-0253e730c30c" (UID: "b064f4cc-d9d3-4cf1-9053-0253e730c30c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:04:15.634932 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:15.634920    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b064f4cc-d9d3-4cf1-9053-0253e730c30c-kube-api-access-gp776" (OuterVolumeSpecName: "kube-api-access-gp776") pod "b064f4cc-d9d3-4cf1-9053-0253e730c30c" (UID: "b064f4cc-d9d3-4cf1-9053-0253e730c30c"). InnerVolumeSpecName "kube-api-access-gp776". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:04:15.635074 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:15.634950    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b064f4cc-d9d3-4cf1-9053-0253e730c30c-istio-token" (OuterVolumeSpecName: "istio-token") pod "b064f4cc-d9d3-4cf1-9053-0253e730c30c" (UID: "b064f4cc-d9d3-4cf1-9053-0253e730c30c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:04:15.635283 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:15.635270    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b064f4cc-d9d3-4cf1-9053-0253e730c30c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b064f4cc-d9d3-4cf1-9053-0253e730c30c" (UID: "b064f4cc-d9d3-4cf1-9053-0253e730c30c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 05 08:04:15.733624 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:15.733543    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b064f4cc-d9d3-4cf1-9053-0253e730c30c-istio-data\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:04:15.733624 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:15.733570    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b064f4cc-d9d3-4cf1-9053-0253e730c30c-istio-podinfo\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:04:15.733624 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:15.733585    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b064f4cc-d9d3-4cf1-9053-0253e730c30c-credential-socket\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:04:15.733624 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:15.733596    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b064f4cc-d9d3-4cf1-9053-0253e730c30c-istio-envoy\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:04:15.733624 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:15.733607    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b064f4cc-d9d3-4cf1-9053-0253e730c30c-istio-token\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:04:15.733624 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:15.733618    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gp776\" (UniqueName: \"kubernetes.io/projected/b064f4cc-d9d3-4cf1-9053-0253e730c30c-kube-api-access-gp776\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:04:15.890020 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:15.889982    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:15.960798 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:15.960755    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:16.139691 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:16.139663    2563 generic.go:356] "Generic (PLEG): container finished" podID="b064f4cc-d9d3-4cf1-9053-0253e730c30c" containerID="6f157d21cae858b48dd261c8edd7ecce18be12b55bdc35c28482021dd6c1b928" exitCode=0
Sep 05 08:04:16.139914 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:16.139715    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"
Sep 05 08:04:16.139914 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:16.139719    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc" event={"ID":"b064f4cc-d9d3-4cf1-9053-0253e730c30c","Type":"ContainerDied","Data":"6f157d21cae858b48dd261c8edd7ecce18be12b55bdc35c28482021dd6c1b928"}
Sep 05 08:04:16.139914 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:16.139879    2563 scope.go:117] "RemoveContainer" containerID="6f157d21cae858b48dd261c8edd7ecce18be12b55bdc35c28482021dd6c1b928"
Sep 05 08:04:16.139914 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:16.139875    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc" event={"ID":"b064f4cc-d9d3-4cf1-9053-0253e730c30c","Type":"ContainerDied","Data":"53bca35379406655a6b56effc4ee5765132fc79c68cc49f0b4b9516ddd705944"}
Sep 05 08:04:16.149405 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:16.149389    2563 scope.go:117] "RemoveContainer" containerID="d181cf5896f7a04647402d1f2ae987c8dbef824529c6cbfeada4ae94df4ab67d"
Sep 05 08:04:16.157318 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:16.157300    2563 scope.go:117] "RemoveContainer" containerID="6f157d21cae858b48dd261c8edd7ecce18be12b55bdc35c28482021dd6c1b928"
Sep 05 08:04:16.157593 ip-10-0-133-14 kubenswrapper[2563]: E0905 08:04:16.157568    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f157d21cae858b48dd261c8edd7ecce18be12b55bdc35c28482021dd6c1b928\": container with ID starting with 6f157d21cae858b48dd261c8edd7ecce18be12b55bdc35c28482021dd6c1b928 not found: ID does not exist" containerID="6f157d21cae858b48dd261c8edd7ecce18be12b55bdc35c28482021dd6c1b928"
Sep 05 08:04:16.157684 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:16.157604    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f157d21cae858b48dd261c8edd7ecce18be12b55bdc35c28482021dd6c1b928"} err="failed to get container status \"6f157d21cae858b48dd261c8edd7ecce18be12b55bdc35c28482021dd6c1b928\": rpc error: code = NotFound desc = could not find container \"6f157d21cae858b48dd261c8edd7ecce18be12b55bdc35c28482021dd6c1b928\": container with ID starting with 6f157d21cae858b48dd261c8edd7ecce18be12b55bdc35c28482021dd6c1b928 not found: ID does not exist"
Sep 05 08:04:16.157684 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:16.157627    2563 scope.go:117] "RemoveContainer" containerID="d181cf5896f7a04647402d1f2ae987c8dbef824529c6cbfeada4ae94df4ab67d"
Sep 05 08:04:16.157928 ip-10-0-133-14 kubenswrapper[2563]: E0905 08:04:16.157900    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d181cf5896f7a04647402d1f2ae987c8dbef824529c6cbfeada4ae94df4ab67d\": container with ID starting with d181cf5896f7a04647402d1f2ae987c8dbef824529c6cbfeada4ae94df4ab67d not found: ID does not exist" containerID="d181cf5896f7a04647402d1f2ae987c8dbef824529c6cbfeada4ae94df4ab67d"
Sep 05 08:04:16.158023 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:16.157935    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d181cf5896f7a04647402d1f2ae987c8dbef824529c6cbfeada4ae94df4ab67d"} err="failed to get container status \"d181cf5896f7a04647402d1f2ae987c8dbef824529c6cbfeada4ae94df4ab67d\": rpc error: code = NotFound desc = could not find container \"d181cf5896f7a04647402d1f2ae987c8dbef824529c6cbfeada4ae94df4ab67d\": container with ID starting with d181cf5896f7a04647402d1f2ae987c8dbef824529c6cbfeada4ae94df4ab67d not found: ID does not exist"
Sep 05 08:04:16.158885 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:16.158868    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"]
Sep 05 08:04:16.165942 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:16.165922    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-4faf97-openshift-default-6b946d97c6-mggcc"]
Sep 05 08:04:16.193588 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:16.193564    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:16.466954 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:16.466877    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:16.562472 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:16.562443    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:17.730284 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:17.730251    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b064f4cc-d9d3-4cf1-9053-0253e730c30c" path="/var/lib/kubelet/pods/b064f4cc-d9d3-4cf1-9053-0253e730c30c/volumes"
Sep 05 08:04:18.229652 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:18.229617    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:22.181277 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:22.181233    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8"]
Sep 05 08:04:22.203034 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:22.203005    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:22.797672 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:22.797624    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:24.237557 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:24.237522    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:24.375052 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:24.375020    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:24.453545 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:24.453514    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:33.000559 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:33.000529    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-61f3ac/test-model-61f3ac-kserve-5bd76c8949-ffrn2"]
Sep 05 08:04:33.001300 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:33.000893    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a46732df-90a2-4003-8a9a-a169571c6f7d" containerName="main"
Sep 05 08:04:33.001300 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:33.000905    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a46732df-90a2-4003-8a9a-a169571c6f7d" containerName="main"
Sep 05 08:04:33.001300 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:33.000914    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b064f4cc-d9d3-4cf1-9053-0253e730c30c" containerName="istio-proxy"
Sep 05 08:04:33.001300 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:33.000920    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b064f4cc-d9d3-4cf1-9053-0253e730c30c" containerName="istio-proxy"
Sep 05 08:04:33.001300 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:33.000927    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3700ad6f-a86f-4d93-9889-df51f3ae03e4" containerName="revoke-keys"
Sep 05 08:04:33.001300 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:33.000934    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3700ad6f-a86f-4d93-9889-df51f3ae03e4" containerName="revoke-keys"
Sep 05 08:04:33.001300 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:33.000943    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b064f4cc-d9d3-4cf1-9053-0253e730c30c" containerName="istio-proxy"
Sep 05 08:04:33.001300 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:33.000948    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b064f4cc-d9d3-4cf1-9053-0253e730c30c" containerName="istio-proxy"
Sep 05 08:04:33.001300 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:33.001013    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a46732df-90a2-4003-8a9a-a169571c6f7d" containerName="main"
Sep 05 08:04:33.001300 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:33.001021    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b064f4cc-d9d3-4cf1-9053-0253e730c30c" containerName="istio-proxy"
Sep 05 08:04:33.001300 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:33.001027    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3700ad6f-a86f-4d93-9889-df51f3ae03e4" containerName="revoke-keys"
Sep 05 08:04:33.001300 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:33.001033    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b064f4cc-d9d3-4cf1-9053-0253e730c30c" containerName="istio-proxy"
Sep 05 08:04:33.005542 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:33.005523    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-61f3ac/test-model-61f3ac-kserve-5bd76c8949-ffrn2"
Sep 05 08:04:33.007931 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:33.007911    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-61f3ac\"/\"openshift-service-ca.crt\""
Sep 05 08:04:33.008019 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:33.007914    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-61f3ac\"/\"default-dockercfg-z9dbq\""
Sep 05 08:04:33.008933 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:33.008918    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-61f3ac\"/\"test-model-61f3ac-kserve-self-signed-certs\""
Sep 05 08:04:33.008981 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:33.008946    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-61f3ac\"/\"kube-root-ca.crt\""
Sep 05 08:04:33.015229 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:33.015208    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-61f3ac/test-model-61f3ac-kserve-5bd76c8949-ffrn2"]
Sep 05 08:04:33.032607 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:33.032579    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:33.096257 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:33.096222    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf7941c7-eb85-4807-9573-3a8a2b2f773a-model-cache\") pod \"test-model-61f3ac-kserve-5bd76c8949-ffrn2\" (UID: \"cf7941c7-eb85-4807-9573-3a8a2b2f773a\") " pod="ai-tenant-e2e-inf-b-61f3ac/test-model-61f3ac-kserve-5bd76c8949-ffrn2"
Sep 05 08:04:33.096379 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:33.096280    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf7941c7-eb85-4807-9573-3a8a2b2f773a-dshm\") pod \"test-model-61f3ac-kserve-5bd76c8949-ffrn2\" (UID: \"cf7941c7-eb85-4807-9573-3a8a2b2f773a\") " pod="ai-tenant-e2e-inf-b-61f3ac/test-model-61f3ac-kserve-5bd76c8949-ffrn2"
Sep 05 08:04:33.096379 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:33.096336    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf7941c7-eb85-4807-9573-3a8a2b2f773a-home\") pod \"test-model-61f3ac-kserve-5bd76c8949-ffrn2\" (UID: \"cf7941c7-eb85-4807-9573-3a8a2b2f773a\") " pod="ai-tenant-e2e-inf-b-61f3ac/test-model-61f3ac-kserve-5bd76c8949-ffrn2"
Sep 05 08:04:33.096379 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:33.096363    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf7941c7-eb85-4807-9573-3a8a2b2f773a-tls-certs\") pod \"test-model-61f3ac-kserve-5bd76c8949-ffrn2\" (UID: \"cf7941c7-eb85-4807-9573-3a8a2b2f773a\") " pod="ai-tenant-e2e-inf-b-61f3ac/test-model-61f3ac-kserve-5bd76c8949-ffrn2"
Sep 05 08:04:33.096515 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:33.096425    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8zm2\" (UniqueName: \"kubernetes.io/projected/cf7941c7-eb85-4807-9573-3a8a2b2f773a-kube-api-access-f8zm2\") pod \"test-model-61f3ac-kserve-5bd76c8949-ffrn2\" (UID: \"cf7941c7-eb85-4807-9573-3a8a2b2f773a\") " pod="ai-tenant-e2e-inf-b-61f3ac/test-model-61f3ac-kserve-5bd76c8949-ffrn2"
Sep 05 08:04:33.096515 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:33.096467    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf7941c7-eb85-4807-9573-3a8a2b2f773a-tmp-dir\") pod \"test-model-61f3ac-kserve-5bd76c8949-ffrn2\" (UID: \"cf7941c7-eb85-4807-9573-3a8a2b2f773a\") " pod="ai-tenant-e2e-inf-b-61f3ac/test-model-61f3ac-kserve-5bd76c8949-ffrn2"
Sep 05 08:04:33.197697 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:33.197664    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf7941c7-eb85-4807-9573-3a8a2b2f773a-tmp-dir\") pod \"test-model-61f3ac-kserve-5bd76c8949-ffrn2\" (UID: \"cf7941c7-eb85-4807-9573-3a8a2b2f773a\") " pod="ai-tenant-e2e-inf-b-61f3ac/test-model-61f3ac-kserve-5bd76c8949-ffrn2"
Sep 05 08:04:33.197913 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:33.197717    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf7941c7-eb85-4807-9573-3a8a2b2f773a-model-cache\") pod \"test-model-61f3ac-kserve-5bd76c8949-ffrn2\" (UID: \"cf7941c7-eb85-4807-9573-3a8a2b2f773a\") " pod="ai-tenant-e2e-inf-b-61f3ac/test-model-61f3ac-kserve-5bd76c8949-ffrn2"
Sep 05 08:04:33.197913 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:33.197771    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf7941c7-eb85-4807-9573-3a8a2b2f773a-dshm\") pod \"test-model-61f3ac-kserve-5bd76c8949-ffrn2\" (UID: \"cf7941c7-eb85-4807-9573-3a8a2b2f773a\") " pod="ai-tenant-e2e-inf-b-61f3ac/test-model-61f3ac-kserve-5bd76c8949-ffrn2"
Sep 05 08:04:33.197913 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:33.197801    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf7941c7-eb85-4807-9573-3a8a2b2f773a-home\") pod \"test-model-61f3ac-kserve-5bd76c8949-ffrn2\" (UID: \"cf7941c7-eb85-4807-9573-3a8a2b2f773a\") " pod="ai-tenant-e2e-inf-b-61f3ac/test-model-61f3ac-kserve-5bd76c8949-ffrn2"
Sep 05 08:04:33.197913 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:33.197830    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf7941c7-eb85-4807-9573-3a8a2b2f773a-tls-certs\") pod \"test-model-61f3ac-kserve-5bd76c8949-ffrn2\" (UID: \"cf7941c7-eb85-4807-9573-3a8a2b2f773a\") " pod="ai-tenant-e2e-inf-b-61f3ac/test-model-61f3ac-kserve-5bd76c8949-ffrn2"
Sep 05 08:04:33.197913 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:33.197893    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8zm2\" (UniqueName: \"kubernetes.io/projected/cf7941c7-eb85-4807-9573-3a8a2b2f773a-kube-api-access-f8zm2\") pod \"test-model-61f3ac-kserve-5bd76c8949-ffrn2\" (UID: \"cf7941c7-eb85-4807-9573-3a8a2b2f773a\") " pod="ai-tenant-e2e-inf-b-61f3ac/test-model-61f3ac-kserve-5bd76c8949-ffrn2"
Sep 05 08:04:33.198172 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:33.198152    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf7941c7-eb85-4807-9573-3a8a2b2f773a-tmp-dir\") pod \"test-model-61f3ac-kserve-5bd76c8949-ffrn2\" (UID: \"cf7941c7-eb85-4807-9573-3a8a2b2f773a\") " pod="ai-tenant-e2e-inf-b-61f3ac/test-model-61f3ac-kserve-5bd76c8949-ffrn2"
Sep 05 08:04:33.198240 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:33.198218    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf7941c7-eb85-4807-9573-3a8a2b2f773a-home\") pod \"test-model-61f3ac-kserve-5bd76c8949-ffrn2\" (UID: \"cf7941c7-eb85-4807-9573-3a8a2b2f773a\") " pod="ai-tenant-e2e-inf-b-61f3ac/test-model-61f3ac-kserve-5bd76c8949-ffrn2"
Sep 05 08:04:33.198282 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:33.198230    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf7941c7-eb85-4807-9573-3a8a2b2f773a-model-cache\") pod \"test-model-61f3ac-kserve-5bd76c8949-ffrn2\" (UID: \"cf7941c7-eb85-4807-9573-3a8a2b2f773a\") " pod="ai-tenant-e2e-inf-b-61f3ac/test-model-61f3ac-kserve-5bd76c8949-ffrn2"
Sep 05 08:04:33.200259 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:33.200240    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf7941c7-eb85-4807-9573-3a8a2b2f773a-dshm\") pod \"test-model-61f3ac-kserve-5bd76c8949-ffrn2\" (UID: \"cf7941c7-eb85-4807-9573-3a8a2b2f773a\") " pod="ai-tenant-e2e-inf-b-61f3ac/test-model-61f3ac-kserve-5bd76c8949-ffrn2"
Sep 05 08:04:33.200550 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:33.200530    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf7941c7-eb85-4807-9573-3a8a2b2f773a-tls-certs\") pod \"test-model-61f3ac-kserve-5bd76c8949-ffrn2\" (UID: \"cf7941c7-eb85-4807-9573-3a8a2b2f773a\") " pod="ai-tenant-e2e-inf-b-61f3ac/test-model-61f3ac-kserve-5bd76c8949-ffrn2"
Sep 05 08:04:33.207971 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:33.207953    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8zm2\" (UniqueName: \"kubernetes.io/projected/cf7941c7-eb85-4807-9573-3a8a2b2f773a-kube-api-access-f8zm2\") pod \"test-model-61f3ac-kserve-5bd76c8949-ffrn2\" (UID: \"cf7941c7-eb85-4807-9573-3a8a2b2f773a\") " pod="ai-tenant-e2e-inf-b-61f3ac/test-model-61f3ac-kserve-5bd76c8949-ffrn2"
Sep 05 08:04:33.316056 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:33.316011    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-61f3ac/test-model-61f3ac-kserve-5bd76c8949-ffrn2"
Sep 05 08:04:33.592731 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:33.592704    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-61f3ac/test-model-61f3ac-kserve-5bd76c8949-ffrn2"]
Sep 05 08:04:33.595440 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:04:33.595416    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcf7941c7_eb85_4807_9573_3a8a2b2f773a.slice/crio-704cc67e810aaca661cbfe318ffc050ada540a0e2b8509cd7aeb0d4e822219f2 WatchSource:0}: Error finding container 704cc67e810aaca661cbfe318ffc050ada540a0e2b8509cd7aeb0d4e822219f2: Status 404 returned error can't find the container with id 704cc67e810aaca661cbfe318ffc050ada540a0e2b8509cd7aeb0d4e822219f2
Sep 05 08:04:34.239753 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:34.239719    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-61f3ac/test-model-61f3ac-kserve-5bd76c8949-ffrn2" event={"ID":"cf7941c7-eb85-4807-9573-3a8a2b2f773a","Type":"ContainerStarted","Data":"97304b3b916a830114b5944a2a6cc4a8eea3072a2998bb16826bc0d6970404e3"}
Sep 05 08:04:34.239753 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:34.239762    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-61f3ac/test-model-61f3ac-kserve-5bd76c8949-ffrn2" event={"ID":"cf7941c7-eb85-4807-9573-3a8a2b2f773a","Type":"ContainerStarted","Data":"704cc67e810aaca661cbfe318ffc050ada540a0e2b8509cd7aeb0d4e822219f2"}
Sep 05 08:04:34.258869 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:34.258807    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-61f3ac/test-model-61f3ac-kserve-5bd76c8949-ffrn2" podStartSLOduration=2.2587917060000002 podStartE2EDuration="2.258791706s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:04:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:04:34.257943521 +0000 UTC m=+977.028085873" watchObservedRunningTime="2026-09-05 08:04:34.258791706 +0000 UTC m=+977.028934044"
Sep 05 08:04:34.955037 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:34.955002    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:35.736554 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:35.736524    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:37.201705 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:37.201644    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8" podUID="f85512e1-9fc1-4325-ad8d-1294116f2bde" containerName="main" containerID="cri-o://333fcc4991907e15f36ac9ea802f88e06b22ca7d51502fa40293578a142caf05" gracePeriod=45
Sep 05 08:04:37.456888 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:37.456811    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8"
Sep 05 08:04:37.540857 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:37.540818    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f85512e1-9fc1-4325-ad8d-1294116f2bde-tls-certs\") pod \"f85512e1-9fc1-4325-ad8d-1294116f2bde\" (UID: \"f85512e1-9fc1-4325-ad8d-1294116f2bde\") "
Sep 05 08:04:37.541036 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:37.540940    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f85512e1-9fc1-4325-ad8d-1294116f2bde-model-cache\") pod \"f85512e1-9fc1-4325-ad8d-1294116f2bde\" (UID: \"f85512e1-9fc1-4325-ad8d-1294116f2bde\") "
Sep 05 08:04:37.541036 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:37.540979    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f85512e1-9fc1-4325-ad8d-1294116f2bde-home\") pod \"f85512e1-9fc1-4325-ad8d-1294116f2bde\" (UID: \"f85512e1-9fc1-4325-ad8d-1294116f2bde\") "
Sep 05 08:04:37.541036 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:37.540999    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f85512e1-9fc1-4325-ad8d-1294116f2bde-tmp-dir\") pod \"f85512e1-9fc1-4325-ad8d-1294116f2bde\" (UID: \"f85512e1-9fc1-4325-ad8d-1294116f2bde\") "
Sep 05 08:04:37.541164 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:37.541038    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pkb2s\" (UniqueName: \"kubernetes.io/projected/f85512e1-9fc1-4325-ad8d-1294116f2bde-kube-api-access-pkb2s\") pod \"f85512e1-9fc1-4325-ad8d-1294116f2bde\" (UID: \"f85512e1-9fc1-4325-ad8d-1294116f2bde\") "
Sep 05 08:04:37.541164 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:37.541064    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f85512e1-9fc1-4325-ad8d-1294116f2bde-dshm\") pod \"f85512e1-9fc1-4325-ad8d-1294116f2bde\" (UID: \"f85512e1-9fc1-4325-ad8d-1294116f2bde\") "
Sep 05 08:04:37.541281 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:37.541252    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f85512e1-9fc1-4325-ad8d-1294116f2bde-home" (OuterVolumeSpecName: "home") pod "f85512e1-9fc1-4325-ad8d-1294116f2bde" (UID: "f85512e1-9fc1-4325-ad8d-1294116f2bde"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:04:37.541341 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:37.541272    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f85512e1-9fc1-4325-ad8d-1294116f2bde-model-cache" (OuterVolumeSpecName: "model-cache") pod "f85512e1-9fc1-4325-ad8d-1294116f2bde" (UID: "f85512e1-9fc1-4325-ad8d-1294116f2bde"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:04:37.541341 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:37.541266    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f85512e1-9fc1-4325-ad8d-1294116f2bde-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f85512e1-9fc1-4325-ad8d-1294116f2bde" (UID: "f85512e1-9fc1-4325-ad8d-1294116f2bde"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:04:37.543312 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:37.543283    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f85512e1-9fc1-4325-ad8d-1294116f2bde-dshm" (OuterVolumeSpecName: "dshm") pod "f85512e1-9fc1-4325-ad8d-1294116f2bde" (UID: "f85512e1-9fc1-4325-ad8d-1294116f2bde"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:04:37.543416 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:37.543325    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f85512e1-9fc1-4325-ad8d-1294116f2bde-kube-api-access-pkb2s" (OuterVolumeSpecName: "kube-api-access-pkb2s") pod "f85512e1-9fc1-4325-ad8d-1294116f2bde" (UID: "f85512e1-9fc1-4325-ad8d-1294116f2bde"). InnerVolumeSpecName "kube-api-access-pkb2s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:04:37.543416 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:37.543398    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f85512e1-9fc1-4325-ad8d-1294116f2bde-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f85512e1-9fc1-4325-ad8d-1294116f2bde" (UID: "f85512e1-9fc1-4325-ad8d-1294116f2bde"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:04:37.642156 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:37.642123    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f85512e1-9fc1-4325-ad8d-1294116f2bde-tmp-dir\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:04:37.642156 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:37.642151    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pkb2s\" (UniqueName: \"kubernetes.io/projected/f85512e1-9fc1-4325-ad8d-1294116f2bde-kube-api-access-pkb2s\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:04:37.642156 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:37.642163    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f85512e1-9fc1-4325-ad8d-1294116f2bde-dshm\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:04:37.642393 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:37.642172    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f85512e1-9fc1-4325-ad8d-1294116f2bde-tls-certs\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:04:37.642393 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:37.642179    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f85512e1-9fc1-4325-ad8d-1294116f2bde-model-cache\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:04:37.642393 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:37.642186    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f85512e1-9fc1-4325-ad8d-1294116f2bde-home\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:04:38.261130 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:38.261087    2563 generic.go:356] "Generic (PLEG): container finished" podID="f85512e1-9fc1-4325-ad8d-1294116f2bde" containerID="333fcc4991907e15f36ac9ea802f88e06b22ca7d51502fa40293578a142caf05" exitCode=0
Sep 05 08:04:38.261130 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:38.261135    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8"
Sep 05 08:04:38.261603 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:38.261180    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8" event={"ID":"f85512e1-9fc1-4325-ad8d-1294116f2bde","Type":"ContainerDied","Data":"333fcc4991907e15f36ac9ea802f88e06b22ca7d51502fa40293578a142caf05"}
Sep 05 08:04:38.261603 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:38.261229    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8" event={"ID":"f85512e1-9fc1-4325-ad8d-1294116f2bde","Type":"ContainerDied","Data":"4fc5e88c1b4461ce59e58e86db000af8da0258b4dd4e22d5e58be29de83c9d51"}
Sep 05 08:04:38.261603 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:38.261248    2563 scope.go:117] "RemoveContainer" containerID="333fcc4991907e15f36ac9ea802f88e06b22ca7d51502fa40293578a142caf05"
Sep 05 08:04:38.272000 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:38.271971    2563 scope.go:117] "RemoveContainer" containerID="333fcc4991907e15f36ac9ea802f88e06b22ca7d51502fa40293578a142caf05"
Sep 05 08:04:38.272309 ip-10-0-133-14 kubenswrapper[2563]: E0905 08:04:38.272289    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"333fcc4991907e15f36ac9ea802f88e06b22ca7d51502fa40293578a142caf05\": container with ID starting with 333fcc4991907e15f36ac9ea802f88e06b22ca7d51502fa40293578a142caf05 not found: ID does not exist" containerID="333fcc4991907e15f36ac9ea802f88e06b22ca7d51502fa40293578a142caf05"
Sep 05 08:04:38.272364 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:38.272319    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"333fcc4991907e15f36ac9ea802f88e06b22ca7d51502fa40293578a142caf05"} err="failed to get container status \"333fcc4991907e15f36ac9ea802f88e06b22ca7d51502fa40293578a142caf05\": rpc error: code = NotFound desc = could not find container \"333fcc4991907e15f36ac9ea802f88e06b22ca7d51502fa40293578a142caf05\": container with ID starting with 333fcc4991907e15f36ac9ea802f88e06b22ca7d51502fa40293578a142caf05 not found: ID does not exist"
Sep 05 08:04:38.282330 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:38.282297    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8"]
Sep 05 08:04:38.287407 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:38.287384    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-a-9cecc80d-kserve-576c478884-6xql8"]
Sep 05 08:04:39.529616 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:39.529534    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:39.730472 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:39.730442    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f85512e1-9fc1-4325-ad8d-1294116f2bde" path="/var/lib/kubelet/pods/f85512e1-9fc1-4325-ad8d-1294116f2bde/volumes"
Sep 05 08:04:41.948610 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:41.948569    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:42.049999 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:42.049965    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:42.149056 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:42.149019    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:42.247078 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:42.246996    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:42.890105 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:42.890066    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:42.989324 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:42.989292    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:43.091828 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:43.091791    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:43.316781 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:43.316754    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-61f3ac/test-model-61f3ac-kserve-5bd76c8949-ffrn2"
Sep 05 08:04:43.316966 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:43.316800    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-61f3ac/test-model-61f3ac-kserve-5bd76c8949-ffrn2"
Sep 05 08:04:43.329423 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:43.329401    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-61f3ac/test-model-61f3ac-kserve-5bd76c8949-ffrn2"
Sep 05 08:04:44.311655 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:44.311626    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-61f3ac/test-model-61f3ac-kserve-5bd76c8949-ffrn2"
Sep 05 08:04:44.641628 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:44.641531    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:51.023132 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:51.023094    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:51.094225 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:51.094194    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:51.704983 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:51.704921    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:55.507120 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:55.507083    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-48a4b5/explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc"]
Sep 05 08:04:55.507495 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:55.507429    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f85512e1-9fc1-4325-ad8d-1294116f2bde" containerName="main"
Sep 05 08:04:55.507495 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:55.507440    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f85512e1-9fc1-4325-ad8d-1294116f2bde" containerName="main"
Sep 05 08:04:55.507495 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:55.507496    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f85512e1-9fc1-4325-ad8d-1294116f2bde" containerName="main"
Sep 05 08:04:55.510427 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:55.510410    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-48a4b5/explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc"
Sep 05 08:04:55.513362 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:55.513338    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-48a4b5\"/\"kube-root-ca.crt\""
Sep 05 08:04:55.513486 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:55.513338    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-48a4b5\"/\"default-dockercfg-hjlkt\""
Sep 05 08:04:55.513628 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:55.513609    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-48a4b5\"/\"openshift-service-ca.crt\""
Sep 05 08:04:55.513756 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:55.513729    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-48a4b5\"/\"explicit-model-48a4b5-kserve-self-signed-certs\""
Sep 05 08:04:55.516942 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:55.516912    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5db8b22c-385b-4cf5-9b3c-70b29b7845a4-tls-certs\") pod \"explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc\" (UID: \"5db8b22c-385b-4cf5-9b3c-70b29b7845a4\") " pod="ai-tenant-e2e-explicit-48a4b5/explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc"
Sep 05 08:04:55.517358 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:55.516976    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5db8b22c-385b-4cf5-9b3c-70b29b7845a4-tmp-dir\") pod \"explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc\" (UID: \"5db8b22c-385b-4cf5-9b3c-70b29b7845a4\") " pod="ai-tenant-e2e-explicit-48a4b5/explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc"
Sep 05 08:04:55.517358 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:55.517025    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tc7l9\" (UniqueName: \"kubernetes.io/projected/5db8b22c-385b-4cf5-9b3c-70b29b7845a4-kube-api-access-tc7l9\") pod \"explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc\" (UID: \"5db8b22c-385b-4cf5-9b3c-70b29b7845a4\") " pod="ai-tenant-e2e-explicit-48a4b5/explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc"
Sep 05 08:04:55.517358 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:55.517138    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5db8b22c-385b-4cf5-9b3c-70b29b7845a4-dshm\") pod \"explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc\" (UID: \"5db8b22c-385b-4cf5-9b3c-70b29b7845a4\") " pod="ai-tenant-e2e-explicit-48a4b5/explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc"
Sep 05 08:04:55.517358 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:55.517171    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5db8b22c-385b-4cf5-9b3c-70b29b7845a4-home\") pod \"explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc\" (UID: \"5db8b22c-385b-4cf5-9b3c-70b29b7845a4\") " pod="ai-tenant-e2e-explicit-48a4b5/explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc"
Sep 05 08:04:55.517358 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:55.517206    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5db8b22c-385b-4cf5-9b3c-70b29b7845a4-model-cache\") pod \"explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc\" (UID: \"5db8b22c-385b-4cf5-9b3c-70b29b7845a4\") " pod="ai-tenant-e2e-explicit-48a4b5/explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc"
Sep 05 08:04:55.519987 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:55.519966    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-48a4b5/explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc"]
Sep 05 08:04:55.618194 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:55.618159    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5db8b22c-385b-4cf5-9b3c-70b29b7845a4-tmp-dir\") pod \"explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc\" (UID: \"5db8b22c-385b-4cf5-9b3c-70b29b7845a4\") " pod="ai-tenant-e2e-explicit-48a4b5/explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc"
Sep 05 08:04:55.618360 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:55.618206    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tc7l9\" (UniqueName: \"kubernetes.io/projected/5db8b22c-385b-4cf5-9b3c-70b29b7845a4-kube-api-access-tc7l9\") pod \"explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc\" (UID: \"5db8b22c-385b-4cf5-9b3c-70b29b7845a4\") " pod="ai-tenant-e2e-explicit-48a4b5/explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc"
Sep 05 08:04:55.618360 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:55.618281    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5db8b22c-385b-4cf5-9b3c-70b29b7845a4-dshm\") pod \"explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc\" (UID: \"5db8b22c-385b-4cf5-9b3c-70b29b7845a4\") " pod="ai-tenant-e2e-explicit-48a4b5/explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc"
Sep 05 08:04:55.618432 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:55.618408    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5db8b22c-385b-4cf5-9b3c-70b29b7845a4-home\") pod \"explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc\" (UID: \"5db8b22c-385b-4cf5-9b3c-70b29b7845a4\") " pod="ai-tenant-e2e-explicit-48a4b5/explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc"
Sep 05 08:04:55.618476 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:55.618444    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5db8b22c-385b-4cf5-9b3c-70b29b7845a4-model-cache\") pod \"explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc\" (UID: \"5db8b22c-385b-4cf5-9b3c-70b29b7845a4\") " pod="ai-tenant-e2e-explicit-48a4b5/explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc"
Sep 05 08:04:55.618525 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:55.618474    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5db8b22c-385b-4cf5-9b3c-70b29b7845a4-tls-certs\") pod \"explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc\" (UID: \"5db8b22c-385b-4cf5-9b3c-70b29b7845a4\") " pod="ai-tenant-e2e-explicit-48a4b5/explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc"
Sep 05 08:04:55.618633 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:55.618614    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5db8b22c-385b-4cf5-9b3c-70b29b7845a4-tmp-dir\") pod \"explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc\" (UID: \"5db8b22c-385b-4cf5-9b3c-70b29b7845a4\") " pod="ai-tenant-e2e-explicit-48a4b5/explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc"
Sep 05 08:04:55.618752 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:55.618735    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5db8b22c-385b-4cf5-9b3c-70b29b7845a4-model-cache\") pod \"explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc\" (UID: \"5db8b22c-385b-4cf5-9b3c-70b29b7845a4\") " pod="ai-tenant-e2e-explicit-48a4b5/explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc"
Sep 05 08:04:55.618829 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:55.618807    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5db8b22c-385b-4cf5-9b3c-70b29b7845a4-home\") pod \"explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc\" (UID: \"5db8b22c-385b-4cf5-9b3c-70b29b7845a4\") " pod="ai-tenant-e2e-explicit-48a4b5/explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc"
Sep 05 08:04:55.620636 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:55.620611    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5db8b22c-385b-4cf5-9b3c-70b29b7845a4-dshm\") pod \"explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc\" (UID: \"5db8b22c-385b-4cf5-9b3c-70b29b7845a4\") " pod="ai-tenant-e2e-explicit-48a4b5/explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc"
Sep 05 08:04:55.620926 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:55.620910    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5db8b22c-385b-4cf5-9b3c-70b29b7845a4-tls-certs\") pod \"explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc\" (UID: \"5db8b22c-385b-4cf5-9b3c-70b29b7845a4\") " pod="ai-tenant-e2e-explicit-48a4b5/explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc"
Sep 05 08:04:55.627491 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:55.627469    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tc7l9\" (UniqueName: \"kubernetes.io/projected/5db8b22c-385b-4cf5-9b3c-70b29b7845a4-kube-api-access-tc7l9\") pod \"explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc\" (UID: \"5db8b22c-385b-4cf5-9b3c-70b29b7845a4\") " pod="ai-tenant-e2e-explicit-48a4b5/explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc"
Sep 05 08:04:55.697284 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:55.697255    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:55.820488 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:55.820451    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-48a4b5/explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc"
Sep 05 08:04:56.094465 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:56.094436    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-48a4b5/explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc"]
Sep 05 08:04:56.095875 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:04:56.095832    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5db8b22c_385b_4cf5_9b3c_70b29b7845a4.slice/crio-7062043c7fb3c6048315cdd78f3473b454df7077c2a9e2e9caf90a13048c41b4 WatchSource:0}: Error finding container 7062043c7fb3c6048315cdd78f3473b454df7077c2a9e2e9caf90a13048c41b4: Status 404 returned error can't find the container with id 7062043c7fb3c6048315cdd78f3473b454df7077c2a9e2e9caf90a13048c41b4
Sep 05 08:04:56.362381 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:56.362284    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-48a4b5/explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc" event={"ID":"5db8b22c-385b-4cf5-9b3c-70b29b7845a4","Type":"ContainerStarted","Data":"e737c63795a365548d3943afbc964b7385e89216fbe0268c88d82fdc40c251a6"}
Sep 05 08:04:56.362381 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:56.362332    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-48a4b5/explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc" event={"ID":"5db8b22c-385b-4cf5-9b3c-70b29b7845a4","Type":"ContainerStarted","Data":"7062043c7fb3c6048315cdd78f3473b454df7077c2a9e2e9caf90a13048c41b4"}
Sep 05 08:04:56.383245 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:56.383195    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-48a4b5/explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc" podStartSLOduration=1.383180117 podStartE2EDuration="1.383180117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:04:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:04:56.382071021 +0000 UTC m=+999.152213361" watchObservedRunningTime="2026-09-05 08:04:56.383180117 +0000 UTC m=+999.153322453"
Sep 05 08:04:57.898655 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:57.898622    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:58.015489 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:58.015451    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:58.125994 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:58.125965    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:58.485653 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:58.485617    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:04:58.558648 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:04:58.558615    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:01.897920 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:01.897879    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:02.032036 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:02.031993    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:03.147172 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:03.147137    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:03.250471 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:03.250424    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:03.348364 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:03.348332    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:03.626214 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:03.626183    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:03.721100 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:03.721068    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:04.532342 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:04.532296    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:04.633008 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:04.632968    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:05.821522 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:05.821488    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-48a4b5/explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc"
Sep 05 08:05:05.822005 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:05.821541    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-48a4b5/explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc"
Sep 05 08:05:05.834169 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:05.834149    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-48a4b5/explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc"
Sep 05 08:05:06.425832 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:06.425804    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-48a4b5/explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc"
Sep 05 08:05:18.062712 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:18.062680    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-48a4b5/explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc"]
Sep 05 08:05:18.082801 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:18.082746    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:18.172409 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:18.172370    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-48a4b5-xk7bw"]
Sep 05 08:05:18.176795 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:18.176777    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-48a4b5-xk7bw"
Sep 05 08:05:18.179225 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:18.179200    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 05 08:05:18.179350 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:18.179229    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mpmmp\""
Sep 05 08:05:18.185101 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:18.184912    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-48a4b5-xk7bw"]
Sep 05 08:05:18.335415 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:18.335333    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a3b676d-0611-4af1-9168-07e899154d71-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-48a4b5-xk7bw\" (UID: \"6a3b676d-0611-4af1-9168-07e899154d71\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-48a4b5-xk7bw"
Sep 05 08:05:18.436064 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:18.436031    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a3b676d-0611-4af1-9168-07e899154d71-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-48a4b5-xk7bw\" (UID: \"6a3b676d-0611-4af1-9168-07e899154d71\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-48a4b5-xk7bw"
Sep 05 08:05:18.436641 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:18.436621    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a3b676d-0611-4af1-9168-07e899154d71-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-48a4b5-xk7bw\" (UID: \"6a3b676d-0611-4af1-9168-07e899154d71\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-48a4b5-xk7bw"
Sep 05 08:05:18.472629 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:18.472593    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:18.488589 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:18.488564    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-48a4b5-xk7bw"
Sep 05 08:05:18.761759 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:18.761738    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-48a4b5-xk7bw"]
Sep 05 08:05:18.763779 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:05:18.763758    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a3b676d_0611_4af1_9168_07e899154d71.slice/crio-2f6ad931df7221f1588a5255a3fad548542fe2c60f3540f553580f767563d5b0 WatchSource:0}: Error finding container 2f6ad931df7221f1588a5255a3fad548542fe2c60f3540f553580f767563d5b0: Status 404 returned error can't find the container with id 2f6ad931df7221f1588a5255a3fad548542fe2c60f3540f553580f767563d5b0
Sep 05 08:05:19.483453 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:19.483420    2563 generic.go:356] "Generic (PLEG): container finished" podID="6a3b676d-0611-4af1-9168-07e899154d71" containerID="35ba458bdd625187cce661301efea1a58605304d4e738511da6ed739cb05b4c5" exitCode=0
Sep 05 08:05:19.483963 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:19.483505    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-48a4b5-xk7bw" event={"ID":"6a3b676d-0611-4af1-9168-07e899154d71","Type":"ContainerDied","Data":"35ba458bdd625187cce661301efea1a58605304d4e738511da6ed739cb05b4c5"}
Sep 05 08:05:19.483963 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:19.483571    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-48a4b5-xk7bw" event={"ID":"6a3b676d-0611-4af1-9168-07e899154d71","Type":"ContainerStarted","Data":"2f6ad931df7221f1588a5255a3fad548542fe2c60f3540f553580f767563d5b0"}
Sep 05 08:05:19.492767 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:19.492744    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:20.628226 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:20.628201    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-48a4b5-xk7bw"
Sep 05 08:05:20.757217 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:20.757129    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a3b676d-0611-4af1-9168-07e899154d71-maas-api-service-ca\") pod \"6a3b676d-0611-4af1-9168-07e899154d71\" (UID: \"6a3b676d-0611-4af1-9168-07e899154d71\") "
Sep 05 08:05:20.757467 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:20.757443    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a3b676d-0611-4af1-9168-07e899154d71-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6a3b676d-0611-4af1-9168-07e899154d71" (UID: "6a3b676d-0611-4af1-9168-07e899154d71"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:05:20.858810 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:20.858779    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a3b676d-0611-4af1-9168-07e899154d71-maas-api-service-ca\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:05:21.494403 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:21.494370    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-48a4b5-xk7bw"
Sep 05 08:05:21.494607 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:21.494408    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-48a4b5-xk7bw" event={"ID":"6a3b676d-0611-4af1-9168-07e899154d71","Type":"ContainerDied","Data":"2f6ad931df7221f1588a5255a3fad548542fe2c60f3540f553580f767563d5b0"}
Sep 05 08:05:21.494607 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:21.494451    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2f6ad931df7221f1588a5255a3fad548542fe2c60f3540f553580f767563d5b0"
Sep 05 08:05:24.010907 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:24.010833    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:26.841388 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:26.841346    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:27.543036 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:27.543003    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:27.640992 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:27.640961    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:27.978065 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:27.978010    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:28.550204 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:28.550172    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:29.367003 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:29.366971    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:29.448835 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:29.448792    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:29.523322 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:29.523287    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:29.622919 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:29.622823    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-48a4b5-xk7bw"]
Sep 05 08:05:29.625363 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:29.625334    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:29.626100 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:29.626083    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-48a4b5-xk7bw"]
Sep 05 08:05:29.727052 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:29.727021    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:29.732170 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:29.732139    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a3b676d-0611-4af1-9168-07e899154d71" path="/var/lib/kubelet/pods/6a3b676d-0611-4af1-9168-07e899154d71/volumes"
Sep 05 08:05:29.977067 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:29.976991    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:33.082715 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:33.082649    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-48a4b5/explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc" podUID="5db8b22c-385b-4cf5-9b3c-70b29b7845a4" containerName="main" containerID="cri-o://e737c63795a365548d3943afbc964b7385e89216fbe0268c88d82fdc40c251a6" gracePeriod=45
Sep 05 08:05:33.331451 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:33.331426    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-48a4b5/explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc"
Sep 05 08:05:33.379357 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:33.379283    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tc7l9\" (UniqueName: \"kubernetes.io/projected/5db8b22c-385b-4cf5-9b3c-70b29b7845a4-kube-api-access-tc7l9\") pod \"5db8b22c-385b-4cf5-9b3c-70b29b7845a4\" (UID: \"5db8b22c-385b-4cf5-9b3c-70b29b7845a4\") "
Sep 05 08:05:33.379357 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:33.379322    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5db8b22c-385b-4cf5-9b3c-70b29b7845a4-tls-certs\") pod \"5db8b22c-385b-4cf5-9b3c-70b29b7845a4\" (UID: \"5db8b22c-385b-4cf5-9b3c-70b29b7845a4\") "
Sep 05 08:05:33.379357 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:33.379353    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5db8b22c-385b-4cf5-9b3c-70b29b7845a4-home\") pod \"5db8b22c-385b-4cf5-9b3c-70b29b7845a4\" (UID: \"5db8b22c-385b-4cf5-9b3c-70b29b7845a4\") "
Sep 05 08:05:33.379563 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:33.379436    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5db8b22c-385b-4cf5-9b3c-70b29b7845a4-model-cache\") pod \"5db8b22c-385b-4cf5-9b3c-70b29b7845a4\" (UID: \"5db8b22c-385b-4cf5-9b3c-70b29b7845a4\") "
Sep 05 08:05:33.379563 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:33.379467    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5db8b22c-385b-4cf5-9b3c-70b29b7845a4-dshm\") pod \"5db8b22c-385b-4cf5-9b3c-70b29b7845a4\" (UID: \"5db8b22c-385b-4cf5-9b3c-70b29b7845a4\") "
Sep 05 08:05:33.379563 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:33.379482    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5db8b22c-385b-4cf5-9b3c-70b29b7845a4-tmp-dir\") pod \"5db8b22c-385b-4cf5-9b3c-70b29b7845a4\" (UID: \"5db8b22c-385b-4cf5-9b3c-70b29b7845a4\") "
Sep 05 08:05:33.379713 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:33.379619    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5db8b22c-385b-4cf5-9b3c-70b29b7845a4-home" (OuterVolumeSpecName: "home") pod "5db8b22c-385b-4cf5-9b3c-70b29b7845a4" (UID: "5db8b22c-385b-4cf5-9b3c-70b29b7845a4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:05:33.379772 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:33.379713    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5db8b22c-385b-4cf5-9b3c-70b29b7845a4-model-cache" (OuterVolumeSpecName: "model-cache") pod "5db8b22c-385b-4cf5-9b3c-70b29b7845a4" (UID: "5db8b22c-385b-4cf5-9b3c-70b29b7845a4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:05:33.379830 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:33.379810    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5db8b22c-385b-4cf5-9b3c-70b29b7845a4-home\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:05:33.379830 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:33.379804    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5db8b22c-385b-4cf5-9b3c-70b29b7845a4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5db8b22c-385b-4cf5-9b3c-70b29b7845a4" (UID: "5db8b22c-385b-4cf5-9b3c-70b29b7845a4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:05:33.379830 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:33.379827    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5db8b22c-385b-4cf5-9b3c-70b29b7845a4-model-cache\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:05:33.381774 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:33.381745    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5db8b22c-385b-4cf5-9b3c-70b29b7845a4-dshm" (OuterVolumeSpecName: "dshm") pod "5db8b22c-385b-4cf5-9b3c-70b29b7845a4" (UID: "5db8b22c-385b-4cf5-9b3c-70b29b7845a4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:05:33.382002 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:33.381979    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5db8b22c-385b-4cf5-9b3c-70b29b7845a4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5db8b22c-385b-4cf5-9b3c-70b29b7845a4" (UID: "5db8b22c-385b-4cf5-9b3c-70b29b7845a4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:05:33.382095 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:33.382012    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5db8b22c-385b-4cf5-9b3c-70b29b7845a4-kube-api-access-tc7l9" (OuterVolumeSpecName: "kube-api-access-tc7l9") pod "5db8b22c-385b-4cf5-9b3c-70b29b7845a4" (UID: "5db8b22c-385b-4cf5-9b3c-70b29b7845a4"). InnerVolumeSpecName "kube-api-access-tc7l9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:05:33.480742 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:33.480706    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5db8b22c-385b-4cf5-9b3c-70b29b7845a4-dshm\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:05:33.480952 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:33.480758    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5db8b22c-385b-4cf5-9b3c-70b29b7845a4-tmp-dir\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:05:33.480952 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:33.480768    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tc7l9\" (UniqueName: \"kubernetes.io/projected/5db8b22c-385b-4cf5-9b3c-70b29b7845a4-kube-api-access-tc7l9\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:05:33.480952 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:33.480781    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5db8b22c-385b-4cf5-9b3c-70b29b7845a4-tls-certs\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:05:33.555582 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:33.555550    2563 generic.go:356] "Generic (PLEG): container finished" podID="5db8b22c-385b-4cf5-9b3c-70b29b7845a4" containerID="e737c63795a365548d3943afbc964b7385e89216fbe0268c88d82fdc40c251a6" exitCode=0
Sep 05 08:05:33.555771 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:33.555593    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-48a4b5/explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc"
Sep 05 08:05:33.555771 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:33.555636    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-48a4b5/explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc" event={"ID":"5db8b22c-385b-4cf5-9b3c-70b29b7845a4","Type":"ContainerDied","Data":"e737c63795a365548d3943afbc964b7385e89216fbe0268c88d82fdc40c251a6"}
Sep 05 08:05:33.555771 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:33.555683    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-48a4b5/explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc" event={"ID":"5db8b22c-385b-4cf5-9b3c-70b29b7845a4","Type":"ContainerDied","Data":"7062043c7fb3c6048315cdd78f3473b454df7077c2a9e2e9caf90a13048c41b4"}
Sep 05 08:05:33.555771 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:33.555707    2563 scope.go:117] "RemoveContainer" containerID="e737c63795a365548d3943afbc964b7385e89216fbe0268c88d82fdc40c251a6"
Sep 05 08:05:33.566210 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:33.566188    2563 scope.go:117] "RemoveContainer" containerID="e737c63795a365548d3943afbc964b7385e89216fbe0268c88d82fdc40c251a6"
Sep 05 08:05:33.566495 ip-10-0-133-14 kubenswrapper[2563]: E0905 08:05:33.566477    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e737c63795a365548d3943afbc964b7385e89216fbe0268c88d82fdc40c251a6\": container with ID starting with e737c63795a365548d3943afbc964b7385e89216fbe0268c88d82fdc40c251a6 not found: ID does not exist" containerID="e737c63795a365548d3943afbc964b7385e89216fbe0268c88d82fdc40c251a6"
Sep 05 08:05:33.566538 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:33.566504    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e737c63795a365548d3943afbc964b7385e89216fbe0268c88d82fdc40c251a6"} err="failed to get container status \"e737c63795a365548d3943afbc964b7385e89216fbe0268c88d82fdc40c251a6\": rpc error: code = NotFound desc = could not find container \"e737c63795a365548d3943afbc964b7385e89216fbe0268c88d82fdc40c251a6\": container with ID starting with e737c63795a365548d3943afbc964b7385e89216fbe0268c88d82fdc40c251a6 not found: ID does not exist"
Sep 05 08:05:33.577196 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:33.577175    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-48a4b5/explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc"]
Sep 05 08:05:33.581215 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:33.581194    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-48a4b5/explicit-model-48a4b5-kserve-7fb7456bbf-c8tpc"]
Sep 05 08:05:33.730564 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:33.730479    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5db8b22c-385b-4cf5-9b3c-70b29b7845a4" path="/var/lib/kubelet/pods/5db8b22c-385b-4cf5-9b3c-70b29b7845a4/volumes"
Sep 05 08:05:33.813657 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:33.813623    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:33.914055 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:33.914022    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:34.192270 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:34.192241    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-66998bbc-twwts"]
Sep 05 08:05:34.192658 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:34.192589    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5db8b22c-385b-4cf5-9b3c-70b29b7845a4" containerName="main"
Sep 05 08:05:34.192658 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:34.192601    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5db8b22c-385b-4cf5-9b3c-70b29b7845a4" containerName="main"
Sep 05 08:05:34.192658 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:34.192619    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a3b676d-0611-4af1-9168-07e899154d71" containerName="revoke-keys"
Sep 05 08:05:34.192658 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:34.192624    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a3b676d-0611-4af1-9168-07e899154d71" containerName="revoke-keys"
Sep 05 08:05:34.192810 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:34.192675    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5db8b22c-385b-4cf5-9b3c-70b29b7845a4" containerName="main"
Sep 05 08:05:34.192810 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:34.192684    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a3b676d-0611-4af1-9168-07e899154d71" containerName="revoke-keys"
Sep 05 08:05:34.196822 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:34.196803    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-66998bbc-twwts"
Sep 05 08:05:34.198983 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:34.198963    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 05 08:05:34.199092 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:34.198996    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mpmmp\""
Sep 05 08:05:34.203374 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:34.203351    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-66998bbc-twwts"]
Sep 05 08:05:34.288216 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:34.288188    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/504554fd-fcf8-49a1-baa0-fe85d56ec841-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-66998bbc-twwts\" (UID: \"504554fd-fcf8-49a1-baa0-fe85d56ec841\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-66998bbc-twwts"
Sep 05 08:05:34.348568 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:34.348536    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:34.388957 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:34.388925    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/504554fd-fcf8-49a1-baa0-fe85d56ec841-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-66998bbc-twwts\" (UID: \"504554fd-fcf8-49a1-baa0-fe85d56ec841\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-66998bbc-twwts"
Sep 05 08:05:34.389591 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:34.389574    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/504554fd-fcf8-49a1-baa0-fe85d56ec841-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-66998bbc-twwts\" (UID: \"504554fd-fcf8-49a1-baa0-fe85d56ec841\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-66998bbc-twwts"
Sep 05 08:05:34.509982 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:34.509899    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-66998bbc-twwts"
Sep 05 08:05:34.781834 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:34.781809    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-66998bbc-twwts"]
Sep 05 08:05:34.784166 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:05:34.784139    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod504554fd_fcf8_49a1_baa0_fe85d56ec841.slice/crio-68283a42e9552754c4937a4a70636542b8f17dd9c95bba7c68ba46b2332969f2 WatchSource:0}: Error finding container 68283a42e9552754c4937a4a70636542b8f17dd9c95bba7c68ba46b2332969f2: Status 404 returned error can't find the container with id 68283a42e9552754c4937a4a70636542b8f17dd9c95bba7c68ba46b2332969f2
Sep 05 08:05:35.570911 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:35.570870    2563 generic.go:356] "Generic (PLEG): container finished" podID="504554fd-fcf8-49a1-baa0-fe85d56ec841" containerID="dd6efa31d71c508869b8b906395b4f963f7a9ee29c8b26448363ebd7a87b8dcb" exitCode=0
Sep 05 08:05:35.571289 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:35.570956    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-66998bbc-twwts" event={"ID":"504554fd-fcf8-49a1-baa0-fe85d56ec841","Type":"ContainerDied","Data":"dd6efa31d71c508869b8b906395b4f963f7a9ee29c8b26448363ebd7a87b8dcb"}
Sep 05 08:05:35.571289 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:35.571012    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-66998bbc-twwts" event={"ID":"504554fd-fcf8-49a1-baa0-fe85d56ec841","Type":"ContainerStarted","Data":"68283a42e9552754c4937a4a70636542b8f17dd9c95bba7c68ba46b2332969f2"}
Sep 05 08:05:36.537126 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:36.537092    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:36.637136 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:36.637089    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:36.705689 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:36.705669    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-66998bbc-twwts"
Sep 05 08:05:36.800685 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:36.800603    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:36.810546 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:36.810518    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/504554fd-fcf8-49a1-baa0-fe85d56ec841-maas-api-service-ca\") pod \"504554fd-fcf8-49a1-baa0-fe85d56ec841\" (UID: \"504554fd-fcf8-49a1-baa0-fe85d56ec841\") "
Sep 05 08:05:36.810976 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:36.810943    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/504554fd-fcf8-49a1-baa0-fe85d56ec841-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "504554fd-fcf8-49a1-baa0-fe85d56ec841" (UID: "504554fd-fcf8-49a1-baa0-fe85d56ec841"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:05:36.911763 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:36.911729    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/504554fd-fcf8-49a1-baa0-fe85d56ec841-maas-api-service-ca\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:05:37.260013 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:37.259982    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:37.582698 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:37.582671    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:37.583934 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:37.583910    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-66998bbc-twwts"
Sep 05 08:05:37.584038 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:37.583910    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-66998bbc-twwts" event={"ID":"504554fd-fcf8-49a1-baa0-fe85d56ec841","Type":"ContainerDied","Data":"68283a42e9552754c4937a4a70636542b8f17dd9c95bba7c68ba46b2332969f2"}
Sep 05 08:05:37.584038 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:37.584034    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="68283a42e9552754c4937a4a70636542b8f17dd9c95bba7c68ba46b2332969f2"
Sep 05 08:05:38.273305 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:38.273266    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:38.736077 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:38.736051    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-e7c0ff/test-model-e7c0ff-kserve-5869b4447-4v9hb"]
Sep 05 08:05:38.802600 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:38.802564    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:39.476066 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:39.476035    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:39.568239 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:39.568203    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:39.677053 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:39.677019    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:40.195735 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:40.195698    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-61f3ac/test-model-61f3ac-kserve-5bd76c8949-ffrn2"]
Sep 05 08:05:40.222620 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:40.222561    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:40.317616 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:40.317583    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e7c0ff-sq7pd"]
Sep 05 08:05:40.318159 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:40.318136    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="504554fd-fcf8-49a1-baa0-fe85d56ec841" containerName="revoke-keys"
Sep 05 08:05:40.318159 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:40.318158    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="504554fd-fcf8-49a1-baa0-fe85d56ec841" containerName="revoke-keys"
Sep 05 08:05:40.318331 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:40.318274    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="504554fd-fcf8-49a1-baa0-fe85d56ec841" containerName="revoke-keys"
Sep 05 08:05:40.321619 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:40.321594    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e7c0ff-sq7pd"
Sep 05 08:05:40.324209 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:40.324166    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 05 08:05:40.324323 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:40.324253    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mpmmp\""
Sep 05 08:05:40.335230 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:40.335207    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e7c0ff-sq7pd"]
Sep 05 08:05:40.345506 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:40.345471    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f345a61e-b296-46ef-9fd3-9c879f53a52b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-e7c0ff-sq7pd\" (UID: \"f345a61e-b296-46ef-9fd3-9c879f53a52b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e7c0ff-sq7pd"
Sep 05 08:05:40.446923 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:40.446811    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f345a61e-b296-46ef-9fd3-9c879f53a52b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-e7c0ff-sq7pd\" (UID: \"f345a61e-b296-46ef-9fd3-9c879f53a52b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e7c0ff-sq7pd"
Sep 05 08:05:40.447439 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:40.447418    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f345a61e-b296-46ef-9fd3-9c879f53a52b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-e7c0ff-sq7pd\" (UID: \"f345a61e-b296-46ef-9fd3-9c879f53a52b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e7c0ff-sq7pd"
Sep 05 08:05:40.472555 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:40.472521    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:40.633687 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:40.633658    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e7c0ff-sq7pd"
Sep 05 08:05:40.922099 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:40.922073    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e7c0ff-sq7pd"]
Sep 05 08:05:40.924797 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:05:40.924770    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf345a61e_b296_46ef_9fd3_9c879f53a52b.slice/crio-0940e5ac2ff21127a2a8146f47a1392d5a9dfaf61fb3909ace921079112d1d0c WatchSource:0}: Error finding container 0940e5ac2ff21127a2a8146f47a1392d5a9dfaf61fb3909ace921079112d1d0c: Status 404 returned error can't find the container with id 0940e5ac2ff21127a2a8146f47a1392d5a9dfaf61fb3909ace921079112d1d0c
Sep 05 08:05:41.606726 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:41.606691    2563 generic.go:356] "Generic (PLEG): container finished" podID="f345a61e-b296-46ef-9fd3-9c879f53a52b" containerID="c9354057df982967ed478495bbeb87084bd6a419daf1744deff4b2a24458a628" exitCode=0
Sep 05 08:05:41.606913 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:41.606778    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e7c0ff-sq7pd" event={"ID":"f345a61e-b296-46ef-9fd3-9c879f53a52b","Type":"ContainerDied","Data":"c9354057df982967ed478495bbeb87084bd6a419daf1744deff4b2a24458a628"}
Sep 05 08:05:41.606913 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:41.606828    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e7c0ff-sq7pd" event={"ID":"f345a61e-b296-46ef-9fd3-9c879f53a52b","Type":"ContainerStarted","Data":"0940e5ac2ff21127a2a8146f47a1392d5a9dfaf61fb3909ace921079112d1d0c"}
Sep 05 08:05:42.739900 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:42.739878    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e7c0ff-sq7pd"
Sep 05 08:05:42.768576 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:42.768546    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f345a61e-b296-46ef-9fd3-9c879f53a52b-maas-api-service-ca\") pod \"f345a61e-b296-46ef-9fd3-9c879f53a52b\" (UID: \"f345a61e-b296-46ef-9fd3-9c879f53a52b\") "
Sep 05 08:05:42.768902 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:42.768878    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f345a61e-b296-46ef-9fd3-9c879f53a52b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f345a61e-b296-46ef-9fd3-9c879f53a52b" (UID: "f345a61e-b296-46ef-9fd3-9c879f53a52b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:05:42.869824 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:42.869736    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f345a61e-b296-46ef-9fd3-9c879f53a52b-maas-api-service-ca\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:05:43.618402 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:43.618372    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e7c0ff-sq7pd"
Sep 05 08:05:43.618576 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:43.618418    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e7c0ff-sq7pd" event={"ID":"f345a61e-b296-46ef-9fd3-9c879f53a52b","Type":"ContainerDied","Data":"0940e5ac2ff21127a2a8146f47a1392d5a9dfaf61fb3909ace921079112d1d0c"}
Sep 05 08:05:43.618576 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:43.618472    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0940e5ac2ff21127a2a8146f47a1392d5a9dfaf61fb3909ace921079112d1d0c"
Sep 05 08:05:44.358736 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:44.358704    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:44.927692 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:44.927650    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:45.027030 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:45.026981    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:45.149230 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:45.149200    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:45.177461 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:45.177423    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-66998bbc-twwts"]
Sep 05 08:05:45.179635 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:45.179571    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-66998bbc-twwts"]
Sep 05 08:05:45.231329 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:45.231298    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:45.549795 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:45.549719    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:45.730578 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:45.730542    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="504554fd-fcf8-49a1-baa0-fe85d56ec841" path="/var/lib/kubelet/pods/504554fd-fcf8-49a1-baa0-fe85d56ec841/volumes"
Sep 05 08:05:50.453738 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:50.453708    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:51.078094 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:51.078062    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:51.182597 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:51.182566    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:51.265036 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:51.265003    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:51.318118 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:51.318082    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e7c0ff-sq7pd"]
Sep 05 08:05:51.320663 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:51.320636    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e7c0ff-sq7pd"]
Sep 05 08:05:51.342573 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:51.342505    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:51.699363 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:51.699277    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:51.729688 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:51.729654    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f345a61e-b296-46ef-9fd3-9c879f53a52b" path="/var/lib/kubelet/pods/f345a61e-b296-46ef-9fd3-9c879f53a52b/volumes"
Sep 05 08:05:52.263236 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:52.263185    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:52.364954 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:52.364896    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:52.529626 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:52.529545    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:52.875653 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:52.875619    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:52.973672 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:52.973639    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:53.102670 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:53.102633    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:53.755883 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:53.755825    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-e7c0ff/test-model-e7c0ff-kserve-5869b4447-4v9hb" podUID="cfcad4e4-896c-4ee9-85db-3d97fb35e934" containerName="main" containerID="cri-o://1ea0f5b9ebceaec467c26445df9af0fa2ca416669c128001e7cfbf754ab4bc9f" gracePeriod=45
Sep 05 08:05:54.008034 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:54.007973    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e7c0ff/test-model-e7c0ff-kserve-5869b4447-4v9hb"
Sep 05 08:05:54.077350 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:54.077319    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c248d\" (UniqueName: \"kubernetes.io/projected/cfcad4e4-896c-4ee9-85db-3d97fb35e934-kube-api-access-c248d\") pod \"cfcad4e4-896c-4ee9-85db-3d97fb35e934\" (UID: \"cfcad4e4-896c-4ee9-85db-3d97fb35e934\") "
Sep 05 08:05:54.077514 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:54.077366    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfcad4e4-896c-4ee9-85db-3d97fb35e934-home\") pod \"cfcad4e4-896c-4ee9-85db-3d97fb35e934\" (UID: \"cfcad4e4-896c-4ee9-85db-3d97fb35e934\") "
Sep 05 08:05:54.077514 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:54.077403    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfcad4e4-896c-4ee9-85db-3d97fb35e934-dshm\") pod \"cfcad4e4-896c-4ee9-85db-3d97fb35e934\" (UID: \"cfcad4e4-896c-4ee9-85db-3d97fb35e934\") "
Sep 05 08:05:54.077602 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:54.077564    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfcad4e4-896c-4ee9-85db-3d97fb35e934-model-cache\") pod \"cfcad4e4-896c-4ee9-85db-3d97fb35e934\" (UID: \"cfcad4e4-896c-4ee9-85db-3d97fb35e934\") "
Sep 05 08:05:54.077673 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:54.077647    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfcad4e4-896c-4ee9-85db-3d97fb35e934-home" (OuterVolumeSpecName: "home") pod "cfcad4e4-896c-4ee9-85db-3d97fb35e934" (UID: "cfcad4e4-896c-4ee9-85db-3d97fb35e934"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:05:54.077736 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:54.077654    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfcad4e4-896c-4ee9-85db-3d97fb35e934-tmp-dir\") pod \"cfcad4e4-896c-4ee9-85db-3d97fb35e934\" (UID: \"cfcad4e4-896c-4ee9-85db-3d97fb35e934\") "
Sep 05 08:05:54.077792 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:54.077751    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfcad4e4-896c-4ee9-85db-3d97fb35e934-tls-certs\") pod \"cfcad4e4-896c-4ee9-85db-3d97fb35e934\" (UID: \"cfcad4e4-896c-4ee9-85db-3d97fb35e934\") "
Sep 05 08:05:54.077880 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:54.077836    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfcad4e4-896c-4ee9-85db-3d97fb35e934-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cfcad4e4-896c-4ee9-85db-3d97fb35e934" (UID: "cfcad4e4-896c-4ee9-85db-3d97fb35e934"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:05:54.077936 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:54.077885    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfcad4e4-896c-4ee9-85db-3d97fb35e934-model-cache" (OuterVolumeSpecName: "model-cache") pod "cfcad4e4-896c-4ee9-85db-3d97fb35e934" (UID: "cfcad4e4-896c-4ee9-85db-3d97fb35e934"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:05:54.078383 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:54.078313    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfcad4e4-896c-4ee9-85db-3d97fb35e934-home\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:05:54.078383 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:54.078341    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfcad4e4-896c-4ee9-85db-3d97fb35e934-model-cache\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:05:54.078383 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:54.078357    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfcad4e4-896c-4ee9-85db-3d97fb35e934-tmp-dir\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:05:54.079656 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:54.079635    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfcad4e4-896c-4ee9-85db-3d97fb35e934-dshm" (OuterVolumeSpecName: "dshm") pod "cfcad4e4-896c-4ee9-85db-3d97fb35e934" (UID: "cfcad4e4-896c-4ee9-85db-3d97fb35e934"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:05:54.079749 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:54.079718    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfcad4e4-896c-4ee9-85db-3d97fb35e934-kube-api-access-c248d" (OuterVolumeSpecName: "kube-api-access-c248d") pod "cfcad4e4-896c-4ee9-85db-3d97fb35e934" (UID: "cfcad4e4-896c-4ee9-85db-3d97fb35e934"). InnerVolumeSpecName "kube-api-access-c248d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:05:54.079805 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:54.079786    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfcad4e4-896c-4ee9-85db-3d97fb35e934-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cfcad4e4-896c-4ee9-85db-3d97fb35e934" (UID: "cfcad4e4-896c-4ee9-85db-3d97fb35e934"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:05:54.179390 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:54.179362    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfcad4e4-896c-4ee9-85db-3d97fb35e934-tls-certs\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:05:54.179390 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:54.179388    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c248d\" (UniqueName: \"kubernetes.io/projected/cfcad4e4-896c-4ee9-85db-3d97fb35e934-kube-api-access-c248d\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:05:54.179390 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:54.179397    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfcad4e4-896c-4ee9-85db-3d97fb35e934-dshm\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:05:54.676575 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:54.676540    2563 generic.go:356] "Generic (PLEG): container finished" podID="cfcad4e4-896c-4ee9-85db-3d97fb35e934" containerID="1ea0f5b9ebceaec467c26445df9af0fa2ca416669c128001e7cfbf754ab4bc9f" exitCode=0
Sep 05 08:05:54.676746 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:54.676582    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e7c0ff/test-model-e7c0ff-kserve-5869b4447-4v9hb"
Sep 05 08:05:54.676746 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:54.676631    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e7c0ff/test-model-e7c0ff-kserve-5869b4447-4v9hb" event={"ID":"cfcad4e4-896c-4ee9-85db-3d97fb35e934","Type":"ContainerDied","Data":"1ea0f5b9ebceaec467c26445df9af0fa2ca416669c128001e7cfbf754ab4bc9f"}
Sep 05 08:05:54.676746 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:54.676673    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e7c0ff/test-model-e7c0ff-kserve-5869b4447-4v9hb" event={"ID":"cfcad4e4-896c-4ee9-85db-3d97fb35e934","Type":"ContainerDied","Data":"d243359540bf8258e218a83c6dc271b0d2ea1eea8e528fb12cab4e36975b0f2f"}
Sep 05 08:05:54.676746 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:54.676692    2563 scope.go:117] "RemoveContainer" containerID="1ea0f5b9ebceaec467c26445df9af0fa2ca416669c128001e7cfbf754ab4bc9f"
Sep 05 08:05:54.685709 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:54.685688    2563 scope.go:117] "RemoveContainer" containerID="1ea0f5b9ebceaec467c26445df9af0fa2ca416669c128001e7cfbf754ab4bc9f"
Sep 05 08:05:54.686004 ip-10-0-133-14 kubenswrapper[2563]: E0905 08:05:54.685986    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ea0f5b9ebceaec467c26445df9af0fa2ca416669c128001e7cfbf754ab4bc9f\": container with ID starting with 1ea0f5b9ebceaec467c26445df9af0fa2ca416669c128001e7cfbf754ab4bc9f not found: ID does not exist" containerID="1ea0f5b9ebceaec467c26445df9af0fa2ca416669c128001e7cfbf754ab4bc9f"
Sep 05 08:05:54.686079 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:54.686013    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ea0f5b9ebceaec467c26445df9af0fa2ca416669c128001e7cfbf754ab4bc9f"} err="failed to get container status \"1ea0f5b9ebceaec467c26445df9af0fa2ca416669c128001e7cfbf754ab4bc9f\": rpc error: code = NotFound desc = could not find container \"1ea0f5b9ebceaec467c26445df9af0fa2ca416669c128001e7cfbf754ab4bc9f\": container with ID starting with 1ea0f5b9ebceaec467c26445df9af0fa2ca416669c128001e7cfbf754ab4bc9f not found: ID does not exist"
Sep 05 08:05:54.697238 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:54.697215    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-e7c0ff/test-model-e7c0ff-kserve-5869b4447-4v9hb"]
Sep 05 08:05:54.705765 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:54.705742    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-e7c0ff/test-model-e7c0ff-kserve-5869b4447-4v9hb"]
Sep 05 08:05:55.216010 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:55.215973    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-61f3ac/test-model-61f3ac-kserve-5bd76c8949-ffrn2" podUID="cf7941c7-eb85-4807-9573-3a8a2b2f773a" containerName="main" containerID="cri-o://97304b3b916a830114b5944a2a6cc4a8eea3072a2998bb16826bc0d6970404e3" gracePeriod=45
Sep 05 08:05:55.466434 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:55.466381    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-61f3ac/test-model-61f3ac-kserve-5bd76c8949-ffrn2"
Sep 05 08:05:55.593299 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:55.593265    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf7941c7-eb85-4807-9573-3a8a2b2f773a-tmp-dir\") pod \"cf7941c7-eb85-4807-9573-3a8a2b2f773a\" (UID: \"cf7941c7-eb85-4807-9573-3a8a2b2f773a\") "
Sep 05 08:05:55.593463 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:55.593333    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf7941c7-eb85-4807-9573-3a8a2b2f773a-tls-certs\") pod \"cf7941c7-eb85-4807-9573-3a8a2b2f773a\" (UID: \"cf7941c7-eb85-4807-9573-3a8a2b2f773a\") "
Sep 05 08:05:55.593463 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:55.593362    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf7941c7-eb85-4807-9573-3a8a2b2f773a-dshm\") pod \"cf7941c7-eb85-4807-9573-3a8a2b2f773a\" (UID: \"cf7941c7-eb85-4807-9573-3a8a2b2f773a\") "
Sep 05 08:05:55.593463 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:55.593379    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf7941c7-eb85-4807-9573-3a8a2b2f773a-model-cache\") pod \"cf7941c7-eb85-4807-9573-3a8a2b2f773a\" (UID: \"cf7941c7-eb85-4807-9573-3a8a2b2f773a\") "
Sep 05 08:05:55.593463 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:55.593408    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf7941c7-eb85-4807-9573-3a8a2b2f773a-home\") pod \"cf7941c7-eb85-4807-9573-3a8a2b2f773a\" (UID: \"cf7941c7-eb85-4807-9573-3a8a2b2f773a\") "
Sep 05 08:05:55.593463 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:55.593425    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f8zm2\" (UniqueName: \"kubernetes.io/projected/cf7941c7-eb85-4807-9573-3a8a2b2f773a-kube-api-access-f8zm2\") pod \"cf7941c7-eb85-4807-9573-3a8a2b2f773a\" (UID: \"cf7941c7-eb85-4807-9573-3a8a2b2f773a\") "
Sep 05 08:05:55.593749 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:55.593576    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf7941c7-eb85-4807-9573-3a8a2b2f773a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cf7941c7-eb85-4807-9573-3a8a2b2f773a" (UID: "cf7941c7-eb85-4807-9573-3a8a2b2f773a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:05:55.593749 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:55.593606    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf7941c7-eb85-4807-9573-3a8a2b2f773a-model-cache" (OuterVolumeSpecName: "model-cache") pod "cf7941c7-eb85-4807-9573-3a8a2b2f773a" (UID: "cf7941c7-eb85-4807-9573-3a8a2b2f773a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:05:55.593749 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:55.593725    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf7941c7-eb85-4807-9573-3a8a2b2f773a-home" (OuterVolumeSpecName: "home") pod "cf7941c7-eb85-4807-9573-3a8a2b2f773a" (UID: "cf7941c7-eb85-4807-9573-3a8a2b2f773a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:05:55.593898 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:55.593813    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf7941c7-eb85-4807-9573-3a8a2b2f773a-tmp-dir\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:05:55.593898 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:55.593831    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf7941c7-eb85-4807-9573-3a8a2b2f773a-model-cache\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:05:55.593898 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:55.593860    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf7941c7-eb85-4807-9573-3a8a2b2f773a-home\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:05:55.595927 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:55.595901    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf7941c7-eb85-4807-9573-3a8a2b2f773a-dshm" (OuterVolumeSpecName: "dshm") pod "cf7941c7-eb85-4807-9573-3a8a2b2f773a" (UID: "cf7941c7-eb85-4807-9573-3a8a2b2f773a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:05:55.596034 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:55.595942    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf7941c7-eb85-4807-9573-3a8a2b2f773a-kube-api-access-f8zm2" (OuterVolumeSpecName: "kube-api-access-f8zm2") pod "cf7941c7-eb85-4807-9573-3a8a2b2f773a" (UID: "cf7941c7-eb85-4807-9573-3a8a2b2f773a"). InnerVolumeSpecName "kube-api-access-f8zm2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:05:55.596034 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:55.595990    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf7941c7-eb85-4807-9573-3a8a2b2f773a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cf7941c7-eb85-4807-9573-3a8a2b2f773a" (UID: "cf7941c7-eb85-4807-9573-3a8a2b2f773a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:05:55.683462 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:55.683422    2563 generic.go:356] "Generic (PLEG): container finished" podID="cf7941c7-eb85-4807-9573-3a8a2b2f773a" containerID="97304b3b916a830114b5944a2a6cc4a8eea3072a2998bb16826bc0d6970404e3" exitCode=0
Sep 05 08:05:55.683462 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:55.683466    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-61f3ac/test-model-61f3ac-kserve-5bd76c8949-ffrn2"
Sep 05 08:05:55.683727 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:55.683508    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-61f3ac/test-model-61f3ac-kserve-5bd76c8949-ffrn2" event={"ID":"cf7941c7-eb85-4807-9573-3a8a2b2f773a","Type":"ContainerDied","Data":"97304b3b916a830114b5944a2a6cc4a8eea3072a2998bb16826bc0d6970404e3"}
Sep 05 08:05:55.683727 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:55.683570    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-61f3ac/test-model-61f3ac-kserve-5bd76c8949-ffrn2" event={"ID":"cf7941c7-eb85-4807-9573-3a8a2b2f773a","Type":"ContainerDied","Data":"704cc67e810aaca661cbfe318ffc050ada540a0e2b8509cd7aeb0d4e822219f2"}
Sep 05 08:05:55.683727 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:55.683591    2563 scope.go:117] "RemoveContainer" containerID="97304b3b916a830114b5944a2a6cc4a8eea3072a2998bb16826bc0d6970404e3"
Sep 05 08:05:55.692815 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:55.692798    2563 scope.go:117] "RemoveContainer" containerID="97304b3b916a830114b5944a2a6cc4a8eea3072a2998bb16826bc0d6970404e3"
Sep 05 08:05:55.693101 ip-10-0-133-14 kubenswrapper[2563]: E0905 08:05:55.693086    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97304b3b916a830114b5944a2a6cc4a8eea3072a2998bb16826bc0d6970404e3\": container with ID starting with 97304b3b916a830114b5944a2a6cc4a8eea3072a2998bb16826bc0d6970404e3 not found: ID does not exist" containerID="97304b3b916a830114b5944a2a6cc4a8eea3072a2998bb16826bc0d6970404e3"
Sep 05 08:05:55.693148 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:55.693108    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97304b3b916a830114b5944a2a6cc4a8eea3072a2998bb16826bc0d6970404e3"} err="failed to get container status \"97304b3b916a830114b5944a2a6cc4a8eea3072a2998bb16826bc0d6970404e3\": rpc error: code = NotFound desc = could not find container \"97304b3b916a830114b5944a2a6cc4a8eea3072a2998bb16826bc0d6970404e3\": container with ID starting with 97304b3b916a830114b5944a2a6cc4a8eea3072a2998bb16826bc0d6970404e3 not found: ID does not exist"
Sep 05 08:05:55.694528 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:55.694508    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf7941c7-eb85-4807-9573-3a8a2b2f773a-dshm\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:05:55.694528 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:55.694526    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8zm2\" (UniqueName: \"kubernetes.io/projected/cf7941c7-eb85-4807-9573-3a8a2b2f773a-kube-api-access-f8zm2\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:05:55.694704 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:55.694534    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf7941c7-eb85-4807-9573-3a8a2b2f773a-tls-certs\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:05:55.711196 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:55.711159    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-61f3ac/test-model-61f3ac-kserve-5bd76c8949-ffrn2"]
Sep 05 08:05:55.713063 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:55.713046    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-61f3ac/test-model-61f3ac-kserve-5bd76c8949-ffrn2"]
Sep 05 08:05:55.729473 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:55.729399    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf7941c7-eb85-4807-9573-3a8a2b2f773a" path="/var/lib/kubelet/pods/cf7941c7-eb85-4807-9573-3a8a2b2f773a/volumes"
Sep 05 08:05:55.729799 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:55.729785    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfcad4e4-896c-4ee9-85db-3d97fb35e934" path="/var/lib/kubelet/pods/cfcad4e4-896c-4ee9-85db-3d97fb35e934/volumes"
Sep 05 08:05:57.807496 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:57.807465    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:57.886525 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:57.886491    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:58.072537 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:58.072432    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:58.455779 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:58.455691    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-61f3ac-d7m8x"]
Sep 05 08:05:58.456051 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:58.456039    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f345a61e-b296-46ef-9fd3-9c879f53a52b" containerName="revoke-keys"
Sep 05 08:05:58.456106 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:58.456053    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f345a61e-b296-46ef-9fd3-9c879f53a52b" containerName="revoke-keys"
Sep 05 08:05:58.456106 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:58.456068    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf7941c7-eb85-4807-9573-3a8a2b2f773a" containerName="main"
Sep 05 08:05:58.456106 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:58.456074    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf7941c7-eb85-4807-9573-3a8a2b2f773a" containerName="main"
Sep 05 08:05:58.456106 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:58.456079    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfcad4e4-896c-4ee9-85db-3d97fb35e934" containerName="main"
Sep 05 08:05:58.456106 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:58.456084    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfcad4e4-896c-4ee9-85db-3d97fb35e934" containerName="main"
Sep 05 08:05:58.456275 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:58.456163    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfcad4e4-896c-4ee9-85db-3d97fb35e934" containerName="main"
Sep 05 08:05:58.456275 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:58.456170    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf7941c7-eb85-4807-9573-3a8a2b2f773a" containerName="main"
Sep 05 08:05:58.456275 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:58.456177    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f345a61e-b296-46ef-9fd3-9c879f53a52b" containerName="revoke-keys"
Sep 05 08:05:58.460449 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:58.460432    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-61f3ac-d7m8x"
Sep 05 08:05:58.464277 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:58.464254    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 05 08:05:58.464388 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:58.464284    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mpmmp\""
Sep 05 08:05:58.470382 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:58.470356    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-61f3ac-d7m8x"]
Sep 05 08:05:58.623141 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:58.623102    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6f52f10-2a6e-4d87-9d11-5b08c244eb6f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-61f3ac-d7m8x\" (UID: \"a6f52f10-2a6e-4d87-9d11-5b08c244eb6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-61f3ac-d7m8x"
Sep 05 08:05:58.644168 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:58.644135    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:58.724141 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:58.724051    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6f52f10-2a6e-4d87-9d11-5b08c244eb6f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-61f3ac-d7m8x\" (UID: \"a6f52f10-2a6e-4d87-9d11-5b08c244eb6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-61f3ac-d7m8x"
Sep 05 08:05:58.724681 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:58.724659    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6f52f10-2a6e-4d87-9d11-5b08c244eb6f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-61f3ac-d7m8x\" (UID: \"a6f52f10-2a6e-4d87-9d11-5b08c244eb6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-61f3ac-d7m8x"
Sep 05 08:05:58.771830 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:58.771805    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-61f3ac-d7m8x"
Sep 05 08:05:58.997703 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:58.997628    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-ecd628/noten-model-ecd628-kserve-58d79d486c-dp6w6"]
Sep 05 08:05:59.001139 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:59.001121    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-ecd628/noten-model-ecd628-kserve-58d79d486c-dp6w6"
Sep 05 08:05:59.004981 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:59.004958    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-ecd628\"/\"openshift-service-ca.crt\""
Sep 05 08:05:59.004981 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:59.004975    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-ecd628\"/\"default-dockercfg-rd9gj\""
Sep 05 08:05:59.005180 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:59.004958    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-ecd628\"/\"kube-root-ca.crt\""
Sep 05 08:05:59.005180 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:59.004966    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-ecd628\"/\"noten-model-ecd628-kserve-self-signed-certs\""
Sep 05 08:05:59.011487 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:59.011464    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-ecd628/noten-model-ecd628-kserve-58d79d486c-dp6w6"]
Sep 05 08:05:59.051639 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:59.051614    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-61f3ac-d7m8x"]
Sep 05 08:05:59.054254 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:05:59.054227    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6f52f10_2a6e_4d87_9d11_5b08c244eb6f.slice/crio-ba2af23882278620f1a514d7d6ed2742d9bda979ed4879e9bb4b0a79b8b04e7a WatchSource:0}: Error finding container ba2af23882278620f1a514d7d6ed2742d9bda979ed4879e9bb4b0a79b8b04e7a: Status 404 returned error can't find the container with id ba2af23882278620f1a514d7d6ed2742d9bda979ed4879e9bb4b0a79b8b04e7a
Sep 05 08:05:59.061174 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:59.061151    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:59.128703 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:59.128661    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2537f19c-480f-455b-ad08-654ff7d027a0-model-cache\") pod \"noten-model-ecd628-kserve-58d79d486c-dp6w6\" (UID: \"2537f19c-480f-455b-ad08-654ff7d027a0\") " pod="ai-tenant-e2e-noten-ecd628/noten-model-ecd628-kserve-58d79d486c-dp6w6"
Sep 05 08:05:59.128836 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:59.128728    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2537f19c-480f-455b-ad08-654ff7d027a0-home\") pod \"noten-model-ecd628-kserve-58d79d486c-dp6w6\" (UID: \"2537f19c-480f-455b-ad08-654ff7d027a0\") " pod="ai-tenant-e2e-noten-ecd628/noten-model-ecd628-kserve-58d79d486c-dp6w6"
Sep 05 08:05:59.128836 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:59.128794    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2537f19c-480f-455b-ad08-654ff7d027a0-dshm\") pod \"noten-model-ecd628-kserve-58d79d486c-dp6w6\" (UID: \"2537f19c-480f-455b-ad08-654ff7d027a0\") " pod="ai-tenant-e2e-noten-ecd628/noten-model-ecd628-kserve-58d79d486c-dp6w6"
Sep 05 08:05:59.128940 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:59.128833    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2537f19c-480f-455b-ad08-654ff7d027a0-tls-certs\") pod \"noten-model-ecd628-kserve-58d79d486c-dp6w6\" (UID: \"2537f19c-480f-455b-ad08-654ff7d027a0\") " pod="ai-tenant-e2e-noten-ecd628/noten-model-ecd628-kserve-58d79d486c-dp6w6"
Sep 05 08:05:59.128940 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:59.128908    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvmb7\" (UniqueName: \"kubernetes.io/projected/2537f19c-480f-455b-ad08-654ff7d027a0-kube-api-access-dvmb7\") pod \"noten-model-ecd628-kserve-58d79d486c-dp6w6\" (UID: \"2537f19c-480f-455b-ad08-654ff7d027a0\") " pod="ai-tenant-e2e-noten-ecd628/noten-model-ecd628-kserve-58d79d486c-dp6w6"
Sep 05 08:05:59.129004 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:59.128947    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2537f19c-480f-455b-ad08-654ff7d027a0-tmp-dir\") pod \"noten-model-ecd628-kserve-58d79d486c-dp6w6\" (UID: \"2537f19c-480f-455b-ad08-654ff7d027a0\") " pod="ai-tenant-e2e-noten-ecd628/noten-model-ecd628-kserve-58d79d486c-dp6w6"
Sep 05 08:05:59.229790 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:59.229757    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2537f19c-480f-455b-ad08-654ff7d027a0-model-cache\") pod \"noten-model-ecd628-kserve-58d79d486c-dp6w6\" (UID: \"2537f19c-480f-455b-ad08-654ff7d027a0\") " pod="ai-tenant-e2e-noten-ecd628/noten-model-ecd628-kserve-58d79d486c-dp6w6"
Sep 05 08:05:59.229995 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:59.229809    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2537f19c-480f-455b-ad08-654ff7d027a0-home\") pod \"noten-model-ecd628-kserve-58d79d486c-dp6w6\" (UID: \"2537f19c-480f-455b-ad08-654ff7d027a0\") " pod="ai-tenant-e2e-noten-ecd628/noten-model-ecd628-kserve-58d79d486c-dp6w6"
Sep 05 08:05:59.229995 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:59.229832    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2537f19c-480f-455b-ad08-654ff7d027a0-dshm\") pod \"noten-model-ecd628-kserve-58d79d486c-dp6w6\" (UID: \"2537f19c-480f-455b-ad08-654ff7d027a0\") " pod="ai-tenant-e2e-noten-ecd628/noten-model-ecd628-kserve-58d79d486c-dp6w6"
Sep 05 08:05:59.229995 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:59.229871    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2537f19c-480f-455b-ad08-654ff7d027a0-tls-certs\") pod \"noten-model-ecd628-kserve-58d79d486c-dp6w6\" (UID: \"2537f19c-480f-455b-ad08-654ff7d027a0\") " pod="ai-tenant-e2e-noten-ecd628/noten-model-ecd628-kserve-58d79d486c-dp6w6"
Sep 05 08:05:59.229995 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:59.229900    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvmb7\" (UniqueName: \"kubernetes.io/projected/2537f19c-480f-455b-ad08-654ff7d027a0-kube-api-access-dvmb7\") pod \"noten-model-ecd628-kserve-58d79d486c-dp6w6\" (UID: \"2537f19c-480f-455b-ad08-654ff7d027a0\") " pod="ai-tenant-e2e-noten-ecd628/noten-model-ecd628-kserve-58d79d486c-dp6w6"
Sep 05 08:05:59.229995 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:59.229930    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2537f19c-480f-455b-ad08-654ff7d027a0-tmp-dir\") pod \"noten-model-ecd628-kserve-58d79d486c-dp6w6\" (UID: \"2537f19c-480f-455b-ad08-654ff7d027a0\") " pod="ai-tenant-e2e-noten-ecd628/noten-model-ecd628-kserve-58d79d486c-dp6w6"
Sep 05 08:05:59.230271 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:59.230231    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2537f19c-480f-455b-ad08-654ff7d027a0-model-cache\") pod \"noten-model-ecd628-kserve-58d79d486c-dp6w6\" (UID: \"2537f19c-480f-455b-ad08-654ff7d027a0\") " pod="ai-tenant-e2e-noten-ecd628/noten-model-ecd628-kserve-58d79d486c-dp6w6"
Sep 05 08:05:59.230319 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:59.230263    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2537f19c-480f-455b-ad08-654ff7d027a0-home\") pod \"noten-model-ecd628-kserve-58d79d486c-dp6w6\" (UID: \"2537f19c-480f-455b-ad08-654ff7d027a0\") " pod="ai-tenant-e2e-noten-ecd628/noten-model-ecd628-kserve-58d79d486c-dp6w6"
Sep 05 08:05:59.230374 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:59.230355    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2537f19c-480f-455b-ad08-654ff7d027a0-tmp-dir\") pod \"noten-model-ecd628-kserve-58d79d486c-dp6w6\" (UID: \"2537f19c-480f-455b-ad08-654ff7d027a0\") " pod="ai-tenant-e2e-noten-ecd628/noten-model-ecd628-kserve-58d79d486c-dp6w6"
Sep 05 08:05:59.232291 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:59.232272    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2537f19c-480f-455b-ad08-654ff7d027a0-dshm\") pod \"noten-model-ecd628-kserve-58d79d486c-dp6w6\" (UID: \"2537f19c-480f-455b-ad08-654ff7d027a0\") " pod="ai-tenant-e2e-noten-ecd628/noten-model-ecd628-kserve-58d79d486c-dp6w6"
Sep 05 08:05:59.232636 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:59.232618    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2537f19c-480f-455b-ad08-654ff7d027a0-tls-certs\") pod \"noten-model-ecd628-kserve-58d79d486c-dp6w6\" (UID: \"2537f19c-480f-455b-ad08-654ff7d027a0\") " pod="ai-tenant-e2e-noten-ecd628/noten-model-ecd628-kserve-58d79d486c-dp6w6"
Sep 05 08:05:59.241634 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:59.241612    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvmb7\" (UniqueName: \"kubernetes.io/projected/2537f19c-480f-455b-ad08-654ff7d027a0-kube-api-access-dvmb7\") pod \"noten-model-ecd628-kserve-58d79d486c-dp6w6\" (UID: \"2537f19c-480f-455b-ad08-654ff7d027a0\") " pod="ai-tenant-e2e-noten-ecd628/noten-model-ecd628-kserve-58d79d486c-dp6w6"
Sep 05 08:05:59.311961 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:59.311931    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-ecd628/noten-model-ecd628-kserve-58d79d486c-dp6w6"
Sep 05 08:05:59.654501 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:59.654475    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-ecd628/noten-model-ecd628-kserve-58d79d486c-dp6w6"]
Sep 05 08:05:59.656458 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:05:59.656434    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2537f19c_480f_455b_ad08_654ff7d027a0.slice/crio-e0afd1eed32ba525f63d1687d28c58ec2a7acb445c14884effde8c0910ef6323 WatchSource:0}: Error finding container e0afd1eed32ba525f63d1687d28c58ec2a7acb445c14884effde8c0910ef6323: Status 404 returned error can't find the container with id e0afd1eed32ba525f63d1687d28c58ec2a7acb445c14884effde8c0910ef6323
Sep 05 08:05:59.713587 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:59.713552    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-ecd628/noten-model-ecd628-kserve-58d79d486c-dp6w6" event={"ID":"2537f19c-480f-455b-ad08-654ff7d027a0","Type":"ContainerStarted","Data":"e0afd1eed32ba525f63d1687d28c58ec2a7acb445c14884effde8c0910ef6323"}
Sep 05 08:05:59.715366 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:59.715340    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-61f3ac-d7m8x" event={"ID":"a6f52f10-2a6e-4d87-9d11-5b08c244eb6f","Type":"ContainerStarted","Data":"da649e8c566dd6fc2bc47b41c540751a6cb3a91781e96bdd85c76a875338cc74"}
Sep 05 08:05:59.715440 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:59.715427    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-61f3ac-d7m8x" event={"ID":"a6f52f10-2a6e-4d87-9d11-5b08c244eb6f","Type":"ContainerStarted","Data":"ba2af23882278620f1a514d7d6ed2742d9bda979ed4879e9bb4b0a79b8b04e7a"}
Sep 05 08:05:59.734238 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:59.734047    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-61f3ac-d7m8x" podStartSLOduration=1.7340297740000001 podStartE2EDuration="1.734029774s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:05:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:05:59.733042263 +0000 UTC m=+1062.503184630" watchObservedRunningTime="2026-09-05 08:05:59.734029774 +0000 UTC m=+1062.504172112"
Sep 05 08:05:59.880017 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:59.879983    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:05:59.996102 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:05:59.996017    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:00.081297 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:00.081265    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:00.241946 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:00.241912    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:00.722118 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:00.722082    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-ecd628/noten-model-ecd628-kserve-58d79d486c-dp6w6" event={"ID":"2537f19c-480f-455b-ad08-654ff7d027a0","Type":"ContainerStarted","Data":"b287e3599e12dfd866167e8097404643da0c54237f3dc62575d33067de120ece"}
Sep 05 08:06:00.723728 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:00.723703    2563 generic.go:356] "Generic (PLEG): container finished" podID="a6f52f10-2a6e-4d87-9d11-5b08c244eb6f" containerID="da649e8c566dd6fc2bc47b41c540751a6cb3a91781e96bdd85c76a875338cc74" exitCode=0
Sep 05 08:06:00.723857 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:00.723787    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-61f3ac-d7m8x" event={"ID":"a6f52f10-2a6e-4d87-9d11-5b08c244eb6f","Type":"ContainerDied","Data":"da649e8c566dd6fc2bc47b41c540751a6cb3a91781e96bdd85c76a875338cc74"}
Sep 05 08:06:00.739093 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:00.739044    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-ecd628/noten-model-ecd628-kserve-58d79d486c-dp6w6" podStartSLOduration=2.739028004 podStartE2EDuration="2.739028004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:05:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:06:00.738242692 +0000 UTC m=+1063.508385031" watchObservedRunningTime="2026-09-05 08:06:00.739028004 +0000 UTC m=+1063.509170341"
Sep 05 08:06:01.862228 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:01.862204    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-61f3ac-d7m8x"
Sep 05 08:06:01.952259 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:01.952217    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6f52f10-2a6e-4d87-9d11-5b08c244eb6f-maas-api-service-ca\") pod \"a6f52f10-2a6e-4d87-9d11-5b08c244eb6f\" (UID: \"a6f52f10-2a6e-4d87-9d11-5b08c244eb6f\") "
Sep 05 08:06:01.952591 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:01.952568    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6f52f10-2a6e-4d87-9d11-5b08c244eb6f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a6f52f10-2a6e-4d87-9d11-5b08c244eb6f" (UID: "a6f52f10-2a6e-4d87-9d11-5b08c244eb6f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:06:02.053190 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:02.053101    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6f52f10-2a6e-4d87-9d11-5b08c244eb6f-maas-api-service-ca\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:06:02.736376 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:02.736336    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-61f3ac-d7m8x" event={"ID":"a6f52f10-2a6e-4d87-9d11-5b08c244eb6f","Type":"ContainerDied","Data":"ba2af23882278620f1a514d7d6ed2742d9bda979ed4879e9bb4b0a79b8b04e7a"}
Sep 05 08:06:02.736376 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:02.736381    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ba2af23882278620f1a514d7d6ed2742d9bda979ed4879e9bb4b0a79b8b04e7a"
Sep 05 08:06:02.736584 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:02.736350    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-61f3ac-d7m8x"
Sep 05 08:06:03.624314 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:03.624284    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-ecd628/noten-model-ecd628-kserve-58d79d486c-dp6w6"]
Sep 05 08:06:03.652776 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:03.652744    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:03.756950 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:03.756922    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:03.988206 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:03.988121    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:04.100734 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:04.100695    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:06.028712 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:06.028679    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:06.098732 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:06.098700    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:06.954276 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:06.954243    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:07.281893 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:07.281793    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:07.678041 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:07.678009    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:09.110397 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:09.110311    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:09.185795 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:09.185740    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:09.312798 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:09.312761    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-ecd628/noten-model-ecd628-kserve-58d79d486c-dp6w6"
Sep 05 08:06:09.335494 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:09.335464    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:09.402634 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:09.402554    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-61f3ac-d7m8x"]
Sep 05 08:06:09.407458 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:09.407428    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-61f3ac-d7m8x"]
Sep 05 08:06:09.411983 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:09.411957    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:09.730615 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:09.730540    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6f52f10-2a6e-4d87-9d11-5b08c244eb6f" path="/var/lib/kubelet/pods/a6f52f10-2a6e-4d87-9d11-5b08c244eb6f/volumes"
Sep 05 08:06:09.753235 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:09.753209    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:10.377016 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:10.376985    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:10.469378 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:10.469342    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:11.552858 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:11.552806    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:11.633091 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:11.633057    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:11.726013 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:11.725977    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:12.050605 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:12.050576    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:12.151964 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:12.151930    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:15.928172 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:15.928136    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:16.029823 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:16.029776    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:16.207532 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:16.207455    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"]
Sep 05 08:06:16.207685 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:16.207666    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t" podUID="b3777fc1-aff3-44c1-8da4-0e0e59e45460" containerName="istio-proxy" containerID="cri-o://52bfd21651d0d5b97e039aa579311598693050c77c153ac285e329db4b1e055f" gracePeriod=30
Sep 05 08:06:16.302292 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:16.302263    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:16.781776 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:16.781733    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:17.124859 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:17.124804    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:17.316265 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:17.316236    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:18.644174 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:18.644112    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-ecd628/noten-model-ecd628-kserve-58d79d486c-dp6w6" podUID="2537f19c-480f-455b-ad08-654ff7d027a0" containerName="main" containerID="cri-o://b287e3599e12dfd866167e8097404643da0c54237f3dc62575d33067de120ece" gracePeriod=45
Sep 05 08:06:18.819101 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:18.819066    2563 generic.go:356] "Generic (PLEG): container finished" podID="2537f19c-480f-455b-ad08-654ff7d027a0" containerID="b287e3599e12dfd866167e8097404643da0c54237f3dc62575d33067de120ece" exitCode=0
Sep 05 08:06:18.819239 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:18.819144    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-ecd628/noten-model-ecd628-kserve-58d79d486c-dp6w6" event={"ID":"2537f19c-480f-455b-ad08-654ff7d027a0","Type":"ContainerDied","Data":"b287e3599e12dfd866167e8097404643da0c54237f3dc62575d33067de120ece"}
Sep 05 08:06:18.896829 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:18.896768    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-ecd628/noten-model-ecd628-kserve-58d79d486c-dp6w6"
Sep 05 08:06:18.913108 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:18.913086    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2537f19c-480f-455b-ad08-654ff7d027a0-tls-certs\") pod \"2537f19c-480f-455b-ad08-654ff7d027a0\" (UID: \"2537f19c-480f-455b-ad08-654ff7d027a0\") "
Sep 05 08:06:18.913224 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:18.913122    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2537f19c-480f-455b-ad08-654ff7d027a0-tmp-dir\") pod \"2537f19c-480f-455b-ad08-654ff7d027a0\" (UID: \"2537f19c-480f-455b-ad08-654ff7d027a0\") "
Sep 05 08:06:18.913224 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:18.913156    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2537f19c-480f-455b-ad08-654ff7d027a0-dshm\") pod \"2537f19c-480f-455b-ad08-654ff7d027a0\" (UID: \"2537f19c-480f-455b-ad08-654ff7d027a0\") "
Sep 05 08:06:18.913224 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:18.913171    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2537f19c-480f-455b-ad08-654ff7d027a0-home\") pod \"2537f19c-480f-455b-ad08-654ff7d027a0\" (UID: \"2537f19c-480f-455b-ad08-654ff7d027a0\") "
Sep 05 08:06:18.913224 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:18.913216    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2537f19c-480f-455b-ad08-654ff7d027a0-model-cache\") pod \"2537f19c-480f-455b-ad08-654ff7d027a0\" (UID: \"2537f19c-480f-455b-ad08-654ff7d027a0\") "
Sep 05 08:06:18.913444 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:18.913234    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dvmb7\" (UniqueName: \"kubernetes.io/projected/2537f19c-480f-455b-ad08-654ff7d027a0-kube-api-access-dvmb7\") pod \"2537f19c-480f-455b-ad08-654ff7d027a0\" (UID: \"2537f19c-480f-455b-ad08-654ff7d027a0\") "
Sep 05 08:06:18.913444 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:18.913382    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2537f19c-480f-455b-ad08-654ff7d027a0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2537f19c-480f-455b-ad08-654ff7d027a0" (UID: "2537f19c-480f-455b-ad08-654ff7d027a0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:06:18.913538 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:18.913467    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2537f19c-480f-455b-ad08-654ff7d027a0-home" (OuterVolumeSpecName: "home") pod "2537f19c-480f-455b-ad08-654ff7d027a0" (UID: "2537f19c-480f-455b-ad08-654ff7d027a0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:06:18.913538 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:18.913478    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2537f19c-480f-455b-ad08-654ff7d027a0-model-cache" (OuterVolumeSpecName: "model-cache") pod "2537f19c-480f-455b-ad08-654ff7d027a0" (UID: "2537f19c-480f-455b-ad08-654ff7d027a0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:06:18.913656 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:18.913639    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2537f19c-480f-455b-ad08-654ff7d027a0-home\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:06:18.913704 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:18.913662    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2537f19c-480f-455b-ad08-654ff7d027a0-model-cache\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:06:18.913704 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:18.913678    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2537f19c-480f-455b-ad08-654ff7d027a0-tmp-dir\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:06:18.915736 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:18.915693    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2537f19c-480f-455b-ad08-654ff7d027a0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2537f19c-480f-455b-ad08-654ff7d027a0" (UID: "2537f19c-480f-455b-ad08-654ff7d027a0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:06:18.915981 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:18.915956    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2537f19c-480f-455b-ad08-654ff7d027a0-kube-api-access-dvmb7" (OuterVolumeSpecName: "kube-api-access-dvmb7") pod "2537f19c-480f-455b-ad08-654ff7d027a0" (UID: "2537f19c-480f-455b-ad08-654ff7d027a0"). InnerVolumeSpecName "kube-api-access-dvmb7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:06:18.916058 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:18.915991    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2537f19c-480f-455b-ad08-654ff7d027a0-dshm" (OuterVolumeSpecName: "dshm") pod "2537f19c-480f-455b-ad08-654ff7d027a0" (UID: "2537f19c-480f-455b-ad08-654ff7d027a0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:06:19.014977 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:19.014943    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dvmb7\" (UniqueName: \"kubernetes.io/projected/2537f19c-480f-455b-ad08-654ff7d027a0-kube-api-access-dvmb7\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:06:19.014977 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:19.014973    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2537f19c-480f-455b-ad08-654ff7d027a0-tls-certs\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:06:19.014977 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:19.014984    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2537f19c-480f-455b-ad08-654ff7d027a0-dshm\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:06:19.826009 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:19.825983    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-ecd628/noten-model-ecd628-kserve-58d79d486c-dp6w6"
Sep 05 08:06:19.826415 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:19.825986    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-ecd628/noten-model-ecd628-kserve-58d79d486c-dp6w6" event={"ID":"2537f19c-480f-455b-ad08-654ff7d027a0","Type":"ContainerDied","Data":"e0afd1eed32ba525f63d1687d28c58ec2a7acb445c14884effde8c0910ef6323"}
Sep 05 08:06:19.826415 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:19.826103    2563 scope.go:117] "RemoveContainer" containerID="b287e3599e12dfd866167e8097404643da0c54237f3dc62575d33067de120ece"
Sep 05 08:06:19.843588 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:19.843564    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-ecd628/noten-model-ecd628-kserve-58d79d486c-dp6w6"]
Sep 05 08:06:19.847905 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:19.847883    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-ecd628/noten-model-ecd628-kserve-58d79d486c-dp6w6"]
Sep 05 08:06:20.371837 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:20.371791    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:20.460041 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:20.459982    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:21.466121 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.466099    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"
Sep 05 08:06:21.537167 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.537078    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3777fc1-aff3-44c1-8da4-0e0e59e45460-istio-data\") pod \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\" (UID: \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\") "
Sep 05 08:06:21.537167 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.537109    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3777fc1-aff3-44c1-8da4-0e0e59e45460-istio-envoy\") pod \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\" (UID: \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\") "
Sep 05 08:06:21.537167 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.537142    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3777fc1-aff3-44c1-8da4-0e0e59e45460-workload-certs\") pod \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\" (UID: \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\") "
Sep 05 08:06:21.537167 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.537163    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3777fc1-aff3-44c1-8da4-0e0e59e45460-istio-token\") pod \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\" (UID: \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\") "
Sep 05 08:06:21.537500 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.537206    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d5nqj\" (UniqueName: \"kubernetes.io/projected/b3777fc1-aff3-44c1-8da4-0e0e59e45460-kube-api-access-d5nqj\") pod \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\" (UID: \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\") "
Sep 05 08:06:21.537500 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.537233    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3777fc1-aff3-44c1-8da4-0e0e59e45460-workload-socket\") pod \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\" (UID: \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\") "
Sep 05 08:06:21.537500 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.537271    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3777fc1-aff3-44c1-8da4-0e0e59e45460-istio-podinfo\") pod \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\" (UID: \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\") "
Sep 05 08:06:21.537500 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.537296    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3777fc1-aff3-44c1-8da4-0e0e59e45460-istiod-ca-cert\") pod \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\" (UID: \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\") "
Sep 05 08:06:21.537500 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.537351    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3777fc1-aff3-44c1-8da4-0e0e59e45460-credential-socket\") pod \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\" (UID: \"b3777fc1-aff3-44c1-8da4-0e0e59e45460\") "
Sep 05 08:06:21.537500 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.537419    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3777fc1-aff3-44c1-8da4-0e0e59e45460-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b3777fc1-aff3-44c1-8da4-0e0e59e45460" (UID: "b3777fc1-aff3-44c1-8da4-0e0e59e45460"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:06:21.537500 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.537434    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3777fc1-aff3-44c1-8da4-0e0e59e45460-istio-data" (OuterVolumeSpecName: "istio-data") pod "b3777fc1-aff3-44c1-8da4-0e0e59e45460" (UID: "b3777fc1-aff3-44c1-8da4-0e0e59e45460"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:06:21.537879 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.537659    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3777fc1-aff3-44c1-8da4-0e0e59e45460-istio-data\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:06:21.537879 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.537681    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3777fc1-aff3-44c1-8da4-0e0e59e45460-workload-certs\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:06:21.537879 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.537694    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3777fc1-aff3-44c1-8da4-0e0e59e45460-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b3777fc1-aff3-44c1-8da4-0e0e59e45460" (UID: "b3777fc1-aff3-44c1-8da4-0e0e59e45460"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:06:21.537879 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.537748    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3777fc1-aff3-44c1-8da4-0e0e59e45460-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b3777fc1-aff3-44c1-8da4-0e0e59e45460" (UID: "b3777fc1-aff3-44c1-8da4-0e0e59e45460"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:06:21.537879 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.537836    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3777fc1-aff3-44c1-8da4-0e0e59e45460-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b3777fc1-aff3-44c1-8da4-0e0e59e45460" (UID: "b3777fc1-aff3-44c1-8da4-0e0e59e45460"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:06:21.539529 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.539502    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3777fc1-aff3-44c1-8da4-0e0e59e45460-kube-api-access-d5nqj" (OuterVolumeSpecName: "kube-api-access-d5nqj") pod "b3777fc1-aff3-44c1-8da4-0e0e59e45460" (UID: "b3777fc1-aff3-44c1-8da4-0e0e59e45460"). InnerVolumeSpecName "kube-api-access-d5nqj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:06:21.539636 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.539579    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3777fc1-aff3-44c1-8da4-0e0e59e45460-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b3777fc1-aff3-44c1-8da4-0e0e59e45460" (UID: "b3777fc1-aff3-44c1-8da4-0e0e59e45460"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:06:21.539962 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.539945    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3777fc1-aff3-44c1-8da4-0e0e59e45460-istio-token" (OuterVolumeSpecName: "istio-token") pod "b3777fc1-aff3-44c1-8da4-0e0e59e45460" (UID: "b3777fc1-aff3-44c1-8da4-0e0e59e45460"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:06:21.540020 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.539957    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b3777fc1-aff3-44c1-8da4-0e0e59e45460-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b3777fc1-aff3-44c1-8da4-0e0e59e45460" (UID: "b3777fc1-aff3-44c1-8da4-0e0e59e45460"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 05 08:06:21.638492 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.638463    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3777fc1-aff3-44c1-8da4-0e0e59e45460-istio-token\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:06:21.638492 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.638490    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5nqj\" (UniqueName: \"kubernetes.io/projected/b3777fc1-aff3-44c1-8da4-0e0e59e45460-kube-api-access-d5nqj\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:06:21.638689 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.638500    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3777fc1-aff3-44c1-8da4-0e0e59e45460-workload-socket\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:06:21.638689 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.638509    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3777fc1-aff3-44c1-8da4-0e0e59e45460-istio-podinfo\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:06:21.638689 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.638517    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3777fc1-aff3-44c1-8da4-0e0e59e45460-istiod-ca-cert\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:06:21.638689 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.638524    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3777fc1-aff3-44c1-8da4-0e0e59e45460-credential-socket\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:06:21.638689 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.638532    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3777fc1-aff3-44c1-8da4-0e0e59e45460-istio-envoy\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:06:21.654372 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.654346    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:21.730585 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.730551    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2537f19c-480f-455b-ad08-654ff7d027a0" path="/var/lib/kubelet/pods/2537f19c-480f-455b-ad08-654ff7d027a0/volumes"
Sep 05 08:06:21.753394 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.753368    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:21.837978 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.837944    2563 generic.go:356] "Generic (PLEG): container finished" podID="b3777fc1-aff3-44c1-8da4-0e0e59e45460" containerID="52bfd21651d0d5b97e039aa579311598693050c77c153ac285e329db4b1e055f" exitCode=0
Sep 05 08:06:21.838157 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.837991    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"
Sep 05 08:06:21.838157 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.838022    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t" event={"ID":"b3777fc1-aff3-44c1-8da4-0e0e59e45460","Type":"ContainerDied","Data":"52bfd21651d0d5b97e039aa579311598693050c77c153ac285e329db4b1e055f"}
Sep 05 08:06:21.838157 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.838079    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t" event={"ID":"b3777fc1-aff3-44c1-8da4-0e0e59e45460","Type":"ContainerDied","Data":"e8380b3588ba36578cdd0ec72b9cea7ebe6ccc736c2be7ac5bb68a0b455aa3dc"}
Sep 05 08:06:21.838157 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.838111    2563 scope.go:117] "RemoveContainer" containerID="52bfd21651d0d5b97e039aa579311598693050c77c153ac285e329db4b1e055f"
Sep 05 08:06:21.847197 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.847177    2563 scope.go:117] "RemoveContainer" containerID="cb9ebbec8e35b2a61c531ad77a2471db127dd41327051e775562f5f2d49d8d66"
Sep 05 08:06:21.853449 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.853424    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"]
Sep 05 08:06:21.855746 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.855729    2563 scope.go:117] "RemoveContainer" containerID="52bfd21651d0d5b97e039aa579311598693050c77c153ac285e329db4b1e055f"
Sep 05 08:06:21.856089 ip-10-0-133-14 kubenswrapper[2563]: E0905 08:06:21.856058    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52bfd21651d0d5b97e039aa579311598693050c77c153ac285e329db4b1e055f\": container with ID starting with 52bfd21651d0d5b97e039aa579311598693050c77c153ac285e329db4b1e055f not found: ID does not exist" containerID="52bfd21651d0d5b97e039aa579311598693050c77c153ac285e329db4b1e055f"
Sep 05 08:06:21.856200 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.856098    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52bfd21651d0d5b97e039aa579311598693050c77c153ac285e329db4b1e055f"} err="failed to get container status \"52bfd21651d0d5b97e039aa579311598693050c77c153ac285e329db4b1e055f\": rpc error: code = NotFound desc = could not find container \"52bfd21651d0d5b97e039aa579311598693050c77c153ac285e329db4b1e055f\": container with ID starting with 52bfd21651d0d5b97e039aa579311598693050c77c153ac285e329db4b1e055f not found: ID does not exist"
Sep 05 08:06:21.856200 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.856129    2563 scope.go:117] "RemoveContainer" containerID="cb9ebbec8e35b2a61c531ad77a2471db127dd41327051e775562f5f2d49d8d66"
Sep 05 08:06:21.856534 ip-10-0-133-14 kubenswrapper[2563]: E0905 08:06:21.856505    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb9ebbec8e35b2a61c531ad77a2471db127dd41327051e775562f5f2d49d8d66\": container with ID starting with cb9ebbec8e35b2a61c531ad77a2471db127dd41327051e775562f5f2d49d8d66 not found: ID does not exist" containerID="cb9ebbec8e35b2a61c531ad77a2471db127dd41327051e775562f5f2d49d8d66"
Sep 05 08:06:21.856633 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.856541    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb9ebbec8e35b2a61c531ad77a2471db127dd41327051e775562f5f2d49d8d66"} err="failed to get container status \"cb9ebbec8e35b2a61c531ad77a2471db127dd41327051e775562f5f2d49d8d66\": rpc error: code = NotFound desc = could not find container \"cb9ebbec8e35b2a61c531ad77a2471db127dd41327051e775562f5f2d49d8d66\": container with ID starting with cb9ebbec8e35b2a61c531ad77a2471db127dd41327051e775562f5f2d49d8d66 not found: ID does not exist"
Sep 05 08:06:21.858426 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:21.858409    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-61f3ac-openshift-default-76bdb6cdb4-zbt8t"]
Sep 05 08:06:22.185880 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:22.185769    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:22.293202 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:22.293172    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:22.394143 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:22.394109    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:23.729520 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:23.729475    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3777fc1-aff3-44c1-8da4-0e0e59e45460" path="/var/lib/kubelet/pods/b3777fc1-aff3-44c1-8da4-0e0e59e45460/volumes"
Sep 05 08:06:32.534243 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:32.534211    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:35.246900 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:35.246865    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:35.626255 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:35.626227    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:36.036367 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:36.036273    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:36.397193 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:36.397160    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:39.598943 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:39.598912    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:39.690596 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:39.690564    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:40.956004 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:40.955967    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:41.059430 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:41.059378    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:41.634174 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:41.634144    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:41.733429 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:41.733393    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:43.241356 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:43.241324    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:43.976810 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:43.976780    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:44.720330 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:44.720291    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:45.392239 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:45.392206    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:45.721180 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:45.721108    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:46.002889 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.002778    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"]
Sep 05 08:06:46.003192 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.003178    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3777fc1-aff3-44c1-8da4-0e0e59e45460" containerName="istio-proxy"
Sep 05 08:06:46.003192 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.003193    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3777fc1-aff3-44c1-8da4-0e0e59e45460" containerName="istio-proxy"
Sep 05 08:06:46.003324 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.003211    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6f52f10-2a6e-4d87-9d11-5b08c244eb6f" containerName="revoke-keys"
Sep 05 08:06:46.003324 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.003216    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6f52f10-2a6e-4d87-9d11-5b08c244eb6f" containerName="revoke-keys"
Sep 05 08:06:46.003324 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.003225    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3777fc1-aff3-44c1-8da4-0e0e59e45460" containerName="istio-proxy"
Sep 05 08:06:46.003324 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.003231    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3777fc1-aff3-44c1-8da4-0e0e59e45460" containerName="istio-proxy"
Sep 05 08:06:46.003324 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.003240    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2537f19c-480f-455b-ad08-654ff7d027a0" containerName="main"
Sep 05 08:06:46.003324 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.003248    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2537f19c-480f-455b-ad08-654ff7d027a0" containerName="main"
Sep 05 08:06:46.003324 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.003303    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3777fc1-aff3-44c1-8da4-0e0e59e45460" containerName="istio-proxy"
Sep 05 08:06:46.003324 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.003312    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6f52f10-2a6e-4d87-9d11-5b08c244eb6f" containerName="revoke-keys"
Sep 05 08:06:46.003324 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.003323    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2537f19c-480f-455b-ad08-654ff7d027a0" containerName="main"
Sep 05 08:06:46.006447 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.006425    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"
Sep 05 08:06:46.012911 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.012886    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-5ef325-openshift-default-dockercfg-sdg98\""
Sep 05 08:06:46.020113 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.020084    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"]
Sep 05 08:06:46.041493 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.041457    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:46.154325 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.154293    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-istiod-ca-cert\") pod \"e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t\" (UID: \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\") " pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"
Sep 05 08:06:46.154508 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.154341    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-workload-certs\") pod \"e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t\" (UID: \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\") " pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"
Sep 05 08:06:46.154508 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.154367    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-credential-socket\") pod \"e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t\" (UID: \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\") " pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"
Sep 05 08:06:46.154508 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.154472    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-istio-envoy\") pod \"e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t\" (UID: \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\") " pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"
Sep 05 08:06:46.154688 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.154506    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psnfd\" (UniqueName: \"kubernetes.io/projected/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-kube-api-access-psnfd\") pod \"e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t\" (UID: \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\") " pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"
Sep 05 08:06:46.154688 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.154536    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-istio-data\") pod \"e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t\" (UID: \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\") " pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"
Sep 05 08:06:46.154688 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.154571    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-workload-socket\") pod \"e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t\" (UID: \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\") " pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"
Sep 05 08:06:46.154688 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.154606    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-istio-podinfo\") pod \"e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t\" (UID: \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\") " pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"
Sep 05 08:06:46.154688 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.154644    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-istio-token\") pod \"e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t\" (UID: \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\") " pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"
Sep 05 08:06:46.256104 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.256015    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-istiod-ca-cert\") pod \"e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t\" (UID: \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\") " pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"
Sep 05 08:06:46.256104 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.256061    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-workload-certs\") pod \"e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t\" (UID: \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\") " pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"
Sep 05 08:06:46.256104 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.256085    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-credential-socket\") pod \"e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t\" (UID: \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\") " pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"
Sep 05 08:06:46.256403 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.256119    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-istio-envoy\") pod \"e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t\" (UID: \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\") " pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"
Sep 05 08:06:46.256403 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.256237    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psnfd\" (UniqueName: \"kubernetes.io/projected/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-kube-api-access-psnfd\") pod \"e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t\" (UID: \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\") " pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"
Sep 05 08:06:46.256403 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.256284    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-istio-data\") pod \"e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t\" (UID: \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\") " pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"
Sep 05 08:06:46.256980 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.256468    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-workload-socket\") pod \"e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t\" (UID: \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\") " pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"
Sep 05 08:06:46.257132 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.257024    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-istio-podinfo\") pod \"e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t\" (UID: \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\") " pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"
Sep 05 08:06:46.257132 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.257086    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-istio-token\") pod \"e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t\" (UID: \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\") " pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"
Sep 05 08:06:46.257250 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.257192    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-istio-data\") pod \"e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t\" (UID: \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\") " pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"
Sep 05 08:06:46.257250 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.257215    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-workload-socket\") pod \"e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t\" (UID: \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\") " pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"
Sep 05 08:06:46.257365 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.257290    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-workload-certs\") pod \"e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t\" (UID: \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\") " pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"
Sep 05 08:06:46.257537 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.257476    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-credential-socket\") pod \"e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t\" (UID: \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\") " pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"
Sep 05 08:06:46.263754 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.257950    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-istiod-ca-cert\") pod \"e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t\" (UID: \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\") " pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"
Sep 05 08:06:46.263754 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.259002    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-istio-envoy\") pod \"e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t\" (UID: \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\") " pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"
Sep 05 08:06:46.266876 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.266729    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-psnfd\" (UniqueName: \"kubernetes.io/projected/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-kube-api-access-psnfd\") pod \"e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t\" (UID: \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\") " pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"
Sep 05 08:06:46.269407 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.269383    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-istio-token\") pod \"e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t\" (UID: \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\") " pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"
Sep 05 08:06:46.269610 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.269591    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-istio-podinfo\") pod \"e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t\" (UID: \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\") " pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"
Sep 05 08:06:46.319506 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.319467    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"
Sep 05 08:06:46.410566 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.410522    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:46.606687 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.606665    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"]
Sep 05 08:06:46.608871 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:06:46.608828    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb72d3eee_5514_4bf7_ae01_a78e6c934bf3.slice/crio-9c6454b0f87711d4c214640aeb46b730ef8fad9cf684dcc86587fdddb9097a68 WatchSource:0}: Error finding container 9c6454b0f87711d4c214640aeb46b730ef8fad9cf684dcc86587fdddb9097a68: Status 404 returned error can't find the container with id 9c6454b0f87711d4c214640aeb46b730ef8fad9cf684dcc86587fdddb9097a68
Sep 05 08:06:46.611250 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.611219    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 05 08:06:46.611315 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.611285    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 05 08:06:46.611359 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.611313    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 05 08:06:46.791174 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.791137    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:46.968912 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.968814    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t" event={"ID":"b72d3eee-5514-4bf7-ae01-a78e6c934bf3","Type":"ContainerStarted","Data":"cf9f013bd65396c755ad6a3c35c761cc83b0a6dd1db95f1e0b51ca264100fb3b"}
Sep 05 08:06:46.968912 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.968874    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t" event={"ID":"b72d3eee-5514-4bf7-ae01-a78e6c934bf3","Type":"ContainerStarted","Data":"9c6454b0f87711d4c214640aeb46b730ef8fad9cf684dcc86587fdddb9097a68"}
Sep 05 08:06:46.995273 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:46.995218    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t" podStartSLOduration=1.995199651 podStartE2EDuration="1.995199651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:06:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:06:46.99476621 +0000 UTC m=+1109.764908567" watchObservedRunningTime="2026-09-05 08:06:46.995199651 +0000 UTC m=+1109.765341988"
Sep 05 08:06:47.320174 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:47.320140    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"
Sep 05 08:06:47.325212 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:47.325189    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"
Sep 05 08:06:47.974216 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:47.974186    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"
Sep 05 08:06:47.975079 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:47.975059    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"
Sep 05 08:06:49.028247 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:49.028214    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:49.124798 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:49.124766    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:50.260037 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:50.260003    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:50.359479 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:50.359444    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:50.427300 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:50.427265    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:50.782443 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:50.782411    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:06:50.882055 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:06:50.882019    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:07:03.237045 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:03.237012    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:07:04.016035 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:04.015983    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 05 08:07:04.016235 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:04.016055    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t" podUID="b72d3eee-5514-4bf7-ae01-a78e6c934bf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 05 08:07:04.485133 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:04.485099    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:07:05.163392 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:05.163355    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7847-140060157298128"]
Sep 05 08:07:05.163754 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:05.163742    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3777fc1-aff3-44c1-8da4-0e0e59e45460" containerName="istio-proxy"
Sep 05 08:07:05.167023 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:05.167007    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7847-140060157298128"
Sep 05 08:07:05.169493 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:05.169470    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-x4x7f\""
Sep 05 08:07:05.171522 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:05.171497    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7847-140060157298128"]
Sep 05 08:07:05.236934 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:05.236903    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbjk7\" (UniqueName: \"kubernetes.io/projected/30572788-4e21-4617-8d83-2e047a9ffe55-kube-api-access-pbjk7\") pod \"test-curl-7847-140060157298128\" (UID: \"30572788-4e21-4617-8d83-2e047a9ffe55\") " pod="opendatahub/test-curl-7847-140060157298128"
Sep 05 08:07:05.337787 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:05.337751    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbjk7\" (UniqueName: \"kubernetes.io/projected/30572788-4e21-4617-8d83-2e047a9ffe55-kube-api-access-pbjk7\") pod \"test-curl-7847-140060157298128\" (UID: \"30572788-4e21-4617-8d83-2e047a9ffe55\") " pod="opendatahub/test-curl-7847-140060157298128"
Sep 05 08:07:05.346734 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:05.346708    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbjk7\" (UniqueName: \"kubernetes.io/projected/30572788-4e21-4617-8d83-2e047a9ffe55-kube-api-access-pbjk7\") pod \"test-curl-7847-140060157298128\" (UID: \"30572788-4e21-4617-8d83-2e047a9ffe55\") " pod="opendatahub/test-curl-7847-140060157298128"
Sep 05 08:07:05.478950 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:05.478867    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7847-140060157298128"
Sep 05 08:07:05.753736 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:05.753711    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7847-140060157298128"]
Sep 05 08:07:05.755990 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:07:05.755964    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod30572788_4e21_4617_8d83_2e047a9ffe55.slice/crio-dd995d2bcd47c0e70ebc32d021ef168896c5783f08cc8530abf06c4ae3c8fe21 WatchSource:0}: Error finding container dd995d2bcd47c0e70ebc32d021ef168896c5783f08cc8530abf06c4ae3c8fe21: Status 404 returned error can't find the container with id dd995d2bcd47c0e70ebc32d021ef168896c5783f08cc8530abf06c4ae3c8fe21
Sep 05 08:07:05.757612 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:05.757591    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 05 08:07:06.068219 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:06.068181    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7847-140060157298128" event={"ID":"30572788-4e21-4617-8d83-2e047a9ffe55","Type":"ContainerStarted","Data":"dd995d2bcd47c0e70ebc32d021ef168896c5783f08cc8530abf06c4ae3c8fe21"}
Sep 05 08:07:06.069751 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:06.069725    2563 generic.go:356] "Generic (PLEG): container finished" podID="b72d3eee-5514-4bf7-ae01-a78e6c934bf3" containerID="cf9f013bd65396c755ad6a3c35c761cc83b0a6dd1db95f1e0b51ca264100fb3b" exitCode=0
Sep 05 08:07:06.069871 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:06.069771    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t" event={"ID":"b72d3eee-5514-4bf7-ae01-a78e6c934bf3","Type":"ContainerDied","Data":"cf9f013bd65396c755ad6a3c35c761cc83b0a6dd1db95f1e0b51ca264100fb3b"}
Sep 05 08:07:06.070178 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:06.070164    2563 scope.go:117] "RemoveContainer" containerID="cf9f013bd65396c755ad6a3c35c761cc83b0a6dd1db95f1e0b51ca264100fb3b"
Sep 05 08:07:06.071192 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:06.071160    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 05 08:07:06.071302 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:06.071232    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 05 08:07:06.071302 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:06.071266    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 05 08:07:07.077045 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:07.077011    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t" event={"ID":"b72d3eee-5514-4bf7-ae01-a78e6c934bf3","Type":"ContainerStarted","Data":"1e89a5fbf8e89b406c875f4939b1bd3578e576b804bac1d38ab27492462eed41"}
Sep 05 08:07:07.078470 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:07.078447    2563 generic.go:356] "Generic (PLEG): container finished" podID="30572788-4e21-4617-8d83-2e047a9ffe55" containerID="afb5359518b280975d7718c32d741c6d3b82fbfc281aa1be6e9f75f38ae80589" exitCode=0
Sep 05 08:07:07.078591 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:07.078533    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7847-140060157298128" event={"ID":"30572788-4e21-4617-8d83-2e047a9ffe55","Type":"ContainerDied","Data":"afb5359518b280975d7718c32d741c6d3b82fbfc281aa1be6e9f75f38ae80589"}
Sep 05 08:07:07.319925 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:07.319888    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"
Sep 05 08:07:07.320151 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:07.320130    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 05 08:07:07.320196 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:07.320176    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t" podUID="b72d3eee-5514-4bf7-ae01-a78e6c934bf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 05 08:07:08.256187 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:08.256164    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7847-140060157298128"
Sep 05 08:07:08.277437 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:08.277404    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7847-140060157298128_30572788-4e21-4617-8d83-2e047a9ffe55/test-curl-7847-140060157298128/0.log"
Sep 05 08:07:08.313951 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:08.305378    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7847-140060157298128"]
Sep 05 08:07:08.314666 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:08.314636    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7847-140060157298128"]
Sep 05 08:07:08.320390 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:08.320359    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 05 08:07:08.320510 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:08.320415    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t" podUID="b72d3eee-5514-4bf7-ae01-a78e6c934bf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 05 08:07:08.366525 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:08.366456    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pbjk7\" (UniqueName: \"kubernetes.io/projected/30572788-4e21-4617-8d83-2e047a9ffe55-kube-api-access-pbjk7\") pod \"30572788-4e21-4617-8d83-2e047a9ffe55\" (UID: \"30572788-4e21-4617-8d83-2e047a9ffe55\") "
Sep 05 08:07:08.368602 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:08.368571    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30572788-4e21-4617-8d83-2e047a9ffe55-kube-api-access-pbjk7" (OuterVolumeSpecName: "kube-api-access-pbjk7") pod "30572788-4e21-4617-8d83-2e047a9ffe55" (UID: "30572788-4e21-4617-8d83-2e047a9ffe55"). InnerVolumeSpecName "kube-api-access-pbjk7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:07:08.426260 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:08.426230    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7847-140060157298128"]
Sep 05 08:07:08.426585 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:08.426571    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30572788-4e21-4617-8d83-2e047a9ffe55" containerName="test-curl-7847-140060157298128"
Sep 05 08:07:08.426639 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:08.426586    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="30572788-4e21-4617-8d83-2e047a9ffe55" containerName="test-curl-7847-140060157298128"
Sep 05 08:07:08.426677 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:08.426653    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="30572788-4e21-4617-8d83-2e047a9ffe55" containerName="test-curl-7847-140060157298128"
Sep 05 08:07:08.428701 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:08.428680    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7847-140060157298128"
Sep 05 08:07:08.435940 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:08.435919    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7847-140060157298128"]
Sep 05 08:07:08.467117 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:08.467090    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7gqc\" (UniqueName: \"kubernetes.io/projected/f8e8d337-ff98-4a7a-bc9f-ed493fab15f6-kube-api-access-q7gqc\") pod \"test-curl-7847-140060157298128\" (UID: \"f8e8d337-ff98-4a7a-bc9f-ed493fab15f6\") " pod="opendatahub/test-curl-7847-140060157298128"
Sep 05 08:07:08.467219 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:08.467153    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pbjk7\" (UniqueName: \"kubernetes.io/projected/30572788-4e21-4617-8d83-2e047a9ffe55-kube-api-access-pbjk7\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:07:08.568619 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:08.568578    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7gqc\" (UniqueName: \"kubernetes.io/projected/f8e8d337-ff98-4a7a-bc9f-ed493fab15f6-kube-api-access-q7gqc\") pod \"test-curl-7847-140060157298128\" (UID: \"f8e8d337-ff98-4a7a-bc9f-ed493fab15f6\") " pod="opendatahub/test-curl-7847-140060157298128"
Sep 05 08:07:08.577414 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:08.577385    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7gqc\" (UniqueName: \"kubernetes.io/projected/f8e8d337-ff98-4a7a-bc9f-ed493fab15f6-kube-api-access-q7gqc\") pod \"test-curl-7847-140060157298128\" (UID: \"f8e8d337-ff98-4a7a-bc9f-ed493fab15f6\") " pod="opendatahub/test-curl-7847-140060157298128"
Sep 05 08:07:08.738328 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:08.738230    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7847-140060157298128"
Sep 05 08:07:09.038409 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:09.038379    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7847-140060157298128"]
Sep 05 08:07:09.040643 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:07:09.040617    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf8e8d337_ff98_4a7a_bc9f_ed493fab15f6.slice/crio-dbba13826de9877019880a8e919b078e61351982eb7a40ba5318d022219d2da3 WatchSource:0}: Error finding container dbba13826de9877019880a8e919b078e61351982eb7a40ba5318d022219d2da3: Status 404 returned error can't find the container with id dbba13826de9877019880a8e919b078e61351982eb7a40ba5318d022219d2da3
Sep 05 08:07:09.092589 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:09.092514    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7847-140060157298128" event={"ID":"f8e8d337-ff98-4a7a-bc9f-ed493fab15f6","Type":"ContainerStarted","Data":"dbba13826de9877019880a8e919b078e61351982eb7a40ba5318d022219d2da3"}
Sep 05 08:07:09.094199 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:09.094173    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dd995d2bcd47c0e70ebc32d021ef168896c5783f08cc8530abf06c4ae3c8fe21"
Sep 05 08:07:09.094336 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:09.094247    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7847-140060157298128"
Sep 05 08:07:09.115302 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:09.115254    2563 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7847-140060157298128" oldPodUID="30572788-4e21-4617-8d83-2e047a9ffe55" podUID="f8e8d337-ff98-4a7a-bc9f-ed493fab15f6"
Sep 05 08:07:09.320622 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:09.320586    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 05 08:07:09.321041 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:09.320643    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t" podUID="b72d3eee-5514-4bf7-ae01-a78e6c934bf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 05 08:07:09.729611 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:09.729527    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30572788-4e21-4617-8d83-2e047a9ffe55" path="/var/lib/kubelet/pods/30572788-4e21-4617-8d83-2e047a9ffe55/volumes"
Sep 05 08:07:10.103590 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:10.103545    2563 generic.go:356] "Generic (PLEG): container finished" podID="f8e8d337-ff98-4a7a-bc9f-ed493fab15f6" containerID="2fe6f9a944e74441d0716640ce23650fb967b3315667e8367615ea0fa3aeb0b2" exitCode=0
Sep 05 08:07:10.103802 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:10.103637    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7847-140060157298128" event={"ID":"f8e8d337-ff98-4a7a-bc9f-ed493fab15f6","Type":"ContainerDied","Data":"2fe6f9a944e74441d0716640ce23650fb967b3315667e8367615ea0fa3aeb0b2"}
Sep 05 08:07:10.320491 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:10.320457    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 05 08:07:10.320662 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:10.320515    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t" podUID="b72d3eee-5514-4bf7-ae01-a78e6c934bf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 05 08:07:11.224779 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:11.224736    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:07:11.270523 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:11.270496    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7847-140060157298128"
Sep 05 08:07:11.312721 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:11.312690    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7847-140060157298128_f8e8d337-ff98-4a7a-bc9f-ed493fab15f6/test-curl-7847-140060157298128/0.log"
Sep 05 08:07:11.320656 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:11.320624    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 05 08:07:11.321047 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:11.320703    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t" podUID="b72d3eee-5514-4bf7-ae01-a78e6c934bf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 05 08:07:11.336963 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:11.336932    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7847-140060157298128"]
Sep 05 08:07:11.343984 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:11.343958    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7847-140060157298128"]
Sep 05 08:07:11.414836 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:11.414717    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q7gqc\" (UniqueName: \"kubernetes.io/projected/f8e8d337-ff98-4a7a-bc9f-ed493fab15f6-kube-api-access-q7gqc\") pod \"f8e8d337-ff98-4a7a-bc9f-ed493fab15f6\" (UID: \"f8e8d337-ff98-4a7a-bc9f-ed493fab15f6\") "
Sep 05 08:07:11.417078 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:11.417041    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8e8d337-ff98-4a7a-bc9f-ed493fab15f6-kube-api-access-q7gqc" (OuterVolumeSpecName: "kube-api-access-q7gqc") pod "f8e8d337-ff98-4a7a-bc9f-ed493fab15f6" (UID: "f8e8d337-ff98-4a7a-bc9f-ed493fab15f6"). InnerVolumeSpecName "kube-api-access-q7gqc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:07:11.464717 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:11.464682    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7847-140060157298704"]
Sep 05 08:07:11.465075 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:11.465060    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8e8d337-ff98-4a7a-bc9f-ed493fab15f6" containerName="test-curl-7847-140060157298128"
Sep 05 08:07:11.465133 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:11.465077    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8e8d337-ff98-4a7a-bc9f-ed493fab15f6" containerName="test-curl-7847-140060157298128"
Sep 05 08:07:11.465174 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:11.465151    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8e8d337-ff98-4a7a-bc9f-ed493fab15f6" containerName="test-curl-7847-140060157298128"
Sep 05 08:07:11.467291 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:11.467275    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7847-140060157298704"
Sep 05 08:07:11.477867 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:11.477829    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7847-140060157298704"]
Sep 05 08:07:11.515972 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:11.515929    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68lph\" (UniqueName: \"kubernetes.io/projected/f49c1e03-e49f-4efd-b1ad-ba76143edba2-kube-api-access-68lph\") pod \"test-curl-7847-140060157298704\" (UID: \"f49c1e03-e49f-4efd-b1ad-ba76143edba2\") " pod="opendatahub/test-curl-7847-140060157298704"
Sep 05 08:07:11.516150 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:11.516011    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7gqc\" (UniqueName: \"kubernetes.io/projected/f8e8d337-ff98-4a7a-bc9f-ed493fab15f6-kube-api-access-q7gqc\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:07:11.617447 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:11.617407    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68lph\" (UniqueName: \"kubernetes.io/projected/f49c1e03-e49f-4efd-b1ad-ba76143edba2-kube-api-access-68lph\") pod \"test-curl-7847-140060157298704\" (UID: \"f49c1e03-e49f-4efd-b1ad-ba76143edba2\") " pod="opendatahub/test-curl-7847-140060157298704"
Sep 05 08:07:11.625513 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:11.625480    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68lph\" (UniqueName: \"kubernetes.io/projected/f49c1e03-e49f-4efd-b1ad-ba76143edba2-kube-api-access-68lph\") pod \"test-curl-7847-140060157298704\" (UID: \"f49c1e03-e49f-4efd-b1ad-ba76143edba2\") " pod="opendatahub/test-curl-7847-140060157298704"
Sep 05 08:07:11.729259 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:11.729175    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8e8d337-ff98-4a7a-bc9f-ed493fab15f6" path="/var/lib/kubelet/pods/f8e8d337-ff98-4a7a-bc9f-ed493fab15f6/volumes"
Sep 05 08:07:11.776651 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:11.776626    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7847-140060157298704"
Sep 05 08:07:12.070052 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:12.070022    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7847-140060157298704"]
Sep 05 08:07:12.072810 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:07:12.072780    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf49c1e03_e49f_4efd_b1ad_ba76143edba2.slice/crio-c72ff3f45d2edf5e29f75d07964ec136e57cdd143cdbd658187a786eed7c4268 WatchSource:0}: Error finding container c72ff3f45d2edf5e29f75d07964ec136e57cdd143cdbd658187a786eed7c4268: Status 404 returned error can't find the container with id c72ff3f45d2edf5e29f75d07964ec136e57cdd143cdbd658187a786eed7c4268
Sep 05 08:07:12.117593 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:12.117565    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7847-140060157298128"
Sep 05 08:07:12.117806 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:12.117613    2563 scope.go:117] "RemoveContainer" containerID="2fe6f9a944e74441d0716640ce23650fb967b3315667e8367615ea0fa3aeb0b2"
Sep 05 08:07:12.119247 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:12.119221    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7847-140060157298704" event={"ID":"f49c1e03-e49f-4efd-b1ad-ba76143edba2","Type":"ContainerStarted","Data":"c72ff3f45d2edf5e29f75d07964ec136e57cdd143cdbd658187a786eed7c4268"}
Sep 05 08:07:12.335096 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:12.335011    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"
Sep 05 08:07:12.335566 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:12.335518    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"
Sep 05 08:07:12.336445 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:12.336425    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"
Sep 05 08:07:13.127971 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:13.127932    2563 generic.go:356] "Generic (PLEG): container finished" podID="f49c1e03-e49f-4efd-b1ad-ba76143edba2" containerID="ff7392003bc46a58db41c79e546fd2d3a8b9de7777947f10aa797847c5b8c04c" exitCode=0
Sep 05 08:07:13.128139 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:13.128024    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7847-140060157298704" event={"ID":"f49c1e03-e49f-4efd-b1ad-ba76143edba2","Type":"ContainerDied","Data":"ff7392003bc46a58db41c79e546fd2d3a8b9de7777947f10aa797847c5b8c04c"}
Sep 05 08:07:14.065958 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:14.065920    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:07:14.269302 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:14.269271    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7847-140060157298704"
Sep 05 08:07:14.288430 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:14.288404    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7847-140060157298704_f49c1e03-e49f-4efd-b1ad-ba76143edba2/test-curl-7847-140060157298704/0.log"
Sep 05 08:07:14.310174 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:14.310130    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7847-140060157298704"]
Sep 05 08:07:14.313253 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:14.313229    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7847-140060157298704"]
Sep 05 08:07:14.344040 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:14.343979    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-68lph\" (UniqueName: \"kubernetes.io/projected/f49c1e03-e49f-4efd-b1ad-ba76143edba2-kube-api-access-68lph\") pod \"f49c1e03-e49f-4efd-b1ad-ba76143edba2\" (UID: \"f49c1e03-e49f-4efd-b1ad-ba76143edba2\") "
Sep 05 08:07:14.346196 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:14.346166    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f49c1e03-e49f-4efd-b1ad-ba76143edba2-kube-api-access-68lph" (OuterVolumeSpecName: "kube-api-access-68lph") pod "f49c1e03-e49f-4efd-b1ad-ba76143edba2" (UID: "f49c1e03-e49f-4efd-b1ad-ba76143edba2"). InnerVolumeSpecName "kube-api-access-68lph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:07:14.442302 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:14.442270    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7847-140060157298704"]
Sep 05 08:07:14.442645 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:14.442632    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f49c1e03-e49f-4efd-b1ad-ba76143edba2" containerName="test-curl-7847-140060157298704"
Sep 05 08:07:14.442704 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:14.442646    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f49c1e03-e49f-4efd-b1ad-ba76143edba2" containerName="test-curl-7847-140060157298704"
Sep 05 08:07:14.442738 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:14.442708    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f49c1e03-e49f-4efd-b1ad-ba76143edba2" containerName="test-curl-7847-140060157298704"
Sep 05 08:07:14.444661 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:14.444645    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7847-140060157298704"
Sep 05 08:07:14.445080 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:14.445064    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68lph\" (UniqueName: \"kubernetes.io/projected/f49c1e03-e49f-4efd-b1ad-ba76143edba2-kube-api-access-68lph\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:07:14.452405 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:14.452373    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7847-140060157298704"]
Sep 05 08:07:14.545488 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:14.545455    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjrbf\" (UniqueName: \"kubernetes.io/projected/0ab2ab52-35b6-4001-889b-820905dce93d-kube-api-access-sjrbf\") pod \"test-curl-7847-140060157298704\" (UID: \"0ab2ab52-35b6-4001-889b-820905dce93d\") " pod="opendatahub/test-curl-7847-140060157298704"
Sep 05 08:07:14.646380 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:14.646292    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjrbf\" (UniqueName: \"kubernetes.io/projected/0ab2ab52-35b6-4001-889b-820905dce93d-kube-api-access-sjrbf\") pod \"test-curl-7847-140060157298704\" (UID: \"0ab2ab52-35b6-4001-889b-820905dce93d\") " pod="opendatahub/test-curl-7847-140060157298704"
Sep 05 08:07:14.654630 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:14.654603    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjrbf\" (UniqueName: \"kubernetes.io/projected/0ab2ab52-35b6-4001-889b-820905dce93d-kube-api-access-sjrbf\") pod \"test-curl-7847-140060157298704\" (UID: \"0ab2ab52-35b6-4001-889b-820905dce93d\") " pod="opendatahub/test-curl-7847-140060157298704"
Sep 05 08:07:14.754399 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:14.754368    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7847-140060157298704"
Sep 05 08:07:15.026394 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:15.026319    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7847-140060157298704"]
Sep 05 08:07:15.029126 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:07:15.029096    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0ab2ab52_35b6_4001_889b_820905dce93d.slice/crio-719b169a29e2963799db6f7d5c90257b22953cfa38b4839dd4de7afdd8ea6e3d WatchSource:0}: Error finding container 719b169a29e2963799db6f7d5c90257b22953cfa38b4839dd4de7afdd8ea6e3d: Status 404 returned error can't find the container with id 719b169a29e2963799db6f7d5c90257b22953cfa38b4839dd4de7afdd8ea6e3d
Sep 05 08:07:15.139480 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:15.139448    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7847-140060157298704" event={"ID":"0ab2ab52-35b6-4001-889b-820905dce93d","Type":"ContainerStarted","Data":"719b169a29e2963799db6f7d5c90257b22953cfa38b4839dd4de7afdd8ea6e3d"}
Sep 05 08:07:15.140687 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:15.140656    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c72ff3f45d2edf5e29f75d07964ec136e57cdd143cdbd658187a786eed7c4268"
Sep 05 08:07:15.140687 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:15.140668    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7847-140060157298704"
Sep 05 08:07:15.151512 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:15.151487    2563 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7847-140060157298704" oldPodUID="f49c1e03-e49f-4efd-b1ad-ba76143edba2" podUID="0ab2ab52-35b6-4001-889b-820905dce93d"
Sep 05 08:07:15.730077 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:15.730044    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f49c1e03-e49f-4efd-b1ad-ba76143edba2" path="/var/lib/kubelet/pods/f49c1e03-e49f-4efd-b1ad-ba76143edba2/volumes"
Sep 05 08:07:16.147516 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:16.147481    2563 generic.go:356] "Generic (PLEG): container finished" podID="0ab2ab52-35b6-4001-889b-820905dce93d" containerID="b0de25e702fbf4b0897b564c2c078e90df9fd1b977e6e7b243bd1c06260a1362" exitCode=0
Sep 05 08:07:16.147956 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:16.147556    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7847-140060157298704" event={"ID":"0ab2ab52-35b6-4001-889b-820905dce93d","Type":"ContainerDied","Data":"b0de25e702fbf4b0897b564c2c078e90df9fd1b977e6e7b243bd1c06260a1362"}
Sep 05 08:07:17.278038 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:17.278015    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7847-140060157298704"
Sep 05 08:07:17.298727 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:17.298700    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7847-140060157298704_0ab2ab52-35b6-4001-889b-820905dce93d/test-curl-7847-140060157298704/0.log"
Sep 05 08:07:17.322241 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:17.322214    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7847-140060157298704"]
Sep 05 08:07:17.324537 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:17.324503    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7847-140060157298704"]
Sep 05 08:07:17.371336 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:17.371312    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sjrbf\" (UniqueName: \"kubernetes.io/projected/0ab2ab52-35b6-4001-889b-820905dce93d-kube-api-access-sjrbf\") pod \"0ab2ab52-35b6-4001-889b-820905dce93d\" (UID: \"0ab2ab52-35b6-4001-889b-820905dce93d\") "
Sep 05 08:07:17.373499 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:17.373474    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ab2ab52-35b6-4001-889b-820905dce93d-kube-api-access-sjrbf" (OuterVolumeSpecName: "kube-api-access-sjrbf") pod "0ab2ab52-35b6-4001-889b-820905dce93d" (UID: "0ab2ab52-35b6-4001-889b-820905dce93d"). InnerVolumeSpecName "kube-api-access-sjrbf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:07:17.472354 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:17.472270    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjrbf\" (UniqueName: \"kubernetes.io/projected/0ab2ab52-35b6-4001-889b-820905dce93d-kube-api-access-sjrbf\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:07:17.730154 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:17.730074    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ab2ab52-35b6-4001-889b-820905dce93d" path="/var/lib/kubelet/pods/0ab2ab52-35b6-4001-889b-820905dce93d/volumes"
Sep 05 08:07:17.776349 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:17.776324    2563 scope.go:117] "RemoveContainer" containerID="bd995adc34e4714cd1f58add31c327027c456c85cb2f70605adc671fbd23e5fd"
Sep 05 08:07:17.965077 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:17.965041    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:07:18.158999 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.158968    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7847-140060157298704"
Sep 05 08:07:18.159181 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.159024    2563 scope.go:117] "RemoveContainer" containerID="b0de25e702fbf4b0897b564c2c078e90df9fd1b977e6e7b243bd1c06260a1362"
Sep 05 08:07:18.270942 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.270909    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"]
Sep 05 08:07:18.271288 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.271274    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ab2ab52-35b6-4001-889b-820905dce93d" containerName="test-curl-7847-140060157298704"
Sep 05 08:07:18.271348 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.271291    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ab2ab52-35b6-4001-889b-820905dce93d" containerName="test-curl-7847-140060157298704"
Sep 05 08:07:18.271384 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.271350    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ab2ab52-35b6-4001-889b-820905dce93d" containerName="test-curl-7847-140060157298704"
Sep 05 08:07:18.274627 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.274610    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"
Sep 05 08:07:18.277971 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.277952    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-4b62745d-openshift-default-dockercfg-qbww9\""
Sep 05 08:07:18.294694 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.294667    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"]
Sep 05 08:07:18.302315 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.302293    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:07:18.381042 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.381006    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/262bcc62-34d3-4363-9189-6a8ab3d96838-istiod-ca-cert\") pod \"e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5\" (UID: \"262bcc62-34d3-4363-9189-6a8ab3d96838\") " pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"
Sep 05 08:07:18.381042 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.381051    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7s6b\" (UniqueName: \"kubernetes.io/projected/262bcc62-34d3-4363-9189-6a8ab3d96838-kube-api-access-k7s6b\") pod \"e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5\" (UID: \"262bcc62-34d3-4363-9189-6a8ab3d96838\") " pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"
Sep 05 08:07:18.381266 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.381085    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/262bcc62-34d3-4363-9189-6a8ab3d96838-workload-socket\") pod \"e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5\" (UID: \"262bcc62-34d3-4363-9189-6a8ab3d96838\") " pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"
Sep 05 08:07:18.381266 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.381174    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/262bcc62-34d3-4363-9189-6a8ab3d96838-istio-envoy\") pod \"e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5\" (UID: \"262bcc62-34d3-4363-9189-6a8ab3d96838\") " pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"
Sep 05 08:07:18.381266 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.381234    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/262bcc62-34d3-4363-9189-6a8ab3d96838-credential-socket\") pod \"e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5\" (UID: \"262bcc62-34d3-4363-9189-6a8ab3d96838\") " pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"
Sep 05 08:07:18.381266 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.381251    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/262bcc62-34d3-4363-9189-6a8ab3d96838-istio-data\") pod \"e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5\" (UID: \"262bcc62-34d3-4363-9189-6a8ab3d96838\") " pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"
Sep 05 08:07:18.381415 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.381268    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/262bcc62-34d3-4363-9189-6a8ab3d96838-istio-podinfo\") pod \"e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5\" (UID: \"262bcc62-34d3-4363-9189-6a8ab3d96838\") " pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"
Sep 05 08:07:18.381415 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.381293    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/262bcc62-34d3-4363-9189-6a8ab3d96838-workload-certs\") pod \"e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5\" (UID: \"262bcc62-34d3-4363-9189-6a8ab3d96838\") " pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"
Sep 05 08:07:18.381415 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.381369    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/262bcc62-34d3-4363-9189-6a8ab3d96838-istio-token\") pod \"e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5\" (UID: \"262bcc62-34d3-4363-9189-6a8ab3d96838\") " pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"
Sep 05 08:07:18.429237 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.429158    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:07:18.481861 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.481819    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/262bcc62-34d3-4363-9189-6a8ab3d96838-workload-certs\") pod \"e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5\" (UID: \"262bcc62-34d3-4363-9189-6a8ab3d96838\") " pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"
Sep 05 08:07:18.482003 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.481883    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/262bcc62-34d3-4363-9189-6a8ab3d96838-istio-token\") pod \"e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5\" (UID: \"262bcc62-34d3-4363-9189-6a8ab3d96838\") " pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"
Sep 05 08:07:18.482003 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.481931    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/262bcc62-34d3-4363-9189-6a8ab3d96838-istiod-ca-cert\") pod \"e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5\" (UID: \"262bcc62-34d3-4363-9189-6a8ab3d96838\") " pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"
Sep 05 08:07:18.482109 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.482088    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7s6b\" (UniqueName: \"kubernetes.io/projected/262bcc62-34d3-4363-9189-6a8ab3d96838-kube-api-access-k7s6b\") pod \"e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5\" (UID: \"262bcc62-34d3-4363-9189-6a8ab3d96838\") " pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"
Sep 05 08:07:18.482163 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.482137    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/262bcc62-34d3-4363-9189-6a8ab3d96838-workload-socket\") pod \"e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5\" (UID: \"262bcc62-34d3-4363-9189-6a8ab3d96838\") " pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"
Sep 05 08:07:18.482230 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.482208    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/262bcc62-34d3-4363-9189-6a8ab3d96838-istio-envoy\") pod \"e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5\" (UID: \"262bcc62-34d3-4363-9189-6a8ab3d96838\") " pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"
Sep 05 08:07:18.482230 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.482219    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/262bcc62-34d3-4363-9189-6a8ab3d96838-workload-certs\") pod \"e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5\" (UID: \"262bcc62-34d3-4363-9189-6a8ab3d96838\") " pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"
Sep 05 08:07:18.482353 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.482291    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/262bcc62-34d3-4363-9189-6a8ab3d96838-credential-socket\") pod \"e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5\" (UID: \"262bcc62-34d3-4363-9189-6a8ab3d96838\") " pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"
Sep 05 08:07:18.482353 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.482319    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/262bcc62-34d3-4363-9189-6a8ab3d96838-istio-data\") pod \"e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5\" (UID: \"262bcc62-34d3-4363-9189-6a8ab3d96838\") " pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"
Sep 05 08:07:18.482353 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.482349    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/262bcc62-34d3-4363-9189-6a8ab3d96838-istio-podinfo\") pod \"e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5\" (UID: \"262bcc62-34d3-4363-9189-6a8ab3d96838\") " pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"
Sep 05 08:07:18.482519 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.482447    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/262bcc62-34d3-4363-9189-6a8ab3d96838-workload-socket\") pod \"e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5\" (UID: \"262bcc62-34d3-4363-9189-6a8ab3d96838\") " pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"
Sep 05 08:07:18.482652 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.482635    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/262bcc62-34d3-4363-9189-6a8ab3d96838-istiod-ca-cert\") pod \"e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5\" (UID: \"262bcc62-34d3-4363-9189-6a8ab3d96838\") " pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"
Sep 05 08:07:18.482696 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.482656    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/262bcc62-34d3-4363-9189-6a8ab3d96838-istio-data\") pod \"e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5\" (UID: \"262bcc62-34d3-4363-9189-6a8ab3d96838\") " pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"
Sep 05 08:07:18.482696 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.482675    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/262bcc62-34d3-4363-9189-6a8ab3d96838-credential-socket\") pod \"e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5\" (UID: \"262bcc62-34d3-4363-9189-6a8ab3d96838\") " pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"
Sep 05 08:07:18.484613 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.484594    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/262bcc62-34d3-4363-9189-6a8ab3d96838-istio-envoy\") pod \"e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5\" (UID: \"262bcc62-34d3-4363-9189-6a8ab3d96838\") " pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"
Sep 05 08:07:18.484930 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.484910    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/262bcc62-34d3-4363-9189-6a8ab3d96838-istio-podinfo\") pod \"e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5\" (UID: \"262bcc62-34d3-4363-9189-6a8ab3d96838\") " pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"
Sep 05 08:07:18.496156 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.496133    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7s6b\" (UniqueName: \"kubernetes.io/projected/262bcc62-34d3-4363-9189-6a8ab3d96838-kube-api-access-k7s6b\") pod \"e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5\" (UID: \"262bcc62-34d3-4363-9189-6a8ab3d96838\") " pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"
Sep 05 08:07:18.496814 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.496799    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/262bcc62-34d3-4363-9189-6a8ab3d96838-istio-token\") pod \"e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5\" (UID: \"262bcc62-34d3-4363-9189-6a8ab3d96838\") " pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"
Sep 05 08:07:18.585299 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.585252    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"
Sep 05 08:07:18.863756 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.863729    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"]
Sep 05 08:07:18.865717 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:07:18.865692    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod262bcc62_34d3_4363_9189_6a8ab3d96838.slice/crio-eb3298e73bcc82d37dc7e0250b1bebc8edd4df7c9caa2befa6b80997dc001aa1 WatchSource:0}: Error finding container eb3298e73bcc82d37dc7e0250b1bebc8edd4df7c9caa2befa6b80997dc001aa1: Status 404 returned error can't find the container with id eb3298e73bcc82d37dc7e0250b1bebc8edd4df7c9caa2befa6b80997dc001aa1
Sep 05 08:07:18.868066 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.868034    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 05 08:07:18.868141 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.868098    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 05 08:07:18.868141 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:18.868127    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 05 08:07:19.167662 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:19.167564    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5" event={"ID":"262bcc62-34d3-4363-9189-6a8ab3d96838","Type":"ContainerStarted","Data":"8ec9078a44fd31009410e261853266ec6c2c8d65864274ee31ead39fbc327db5"}
Sep 05 08:07:19.167662 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:19.167635    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5" event={"ID":"262bcc62-34d3-4363-9189-6a8ab3d96838","Type":"ContainerStarted","Data":"eb3298e73bcc82d37dc7e0250b1bebc8edd4df7c9caa2befa6b80997dc001aa1"}
Sep 05 08:07:19.191432 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:19.191382    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5" podStartSLOduration=1.1913666250000001 podStartE2EDuration="1.191366625s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:07:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:07:19.189576098 +0000 UTC m=+1141.959718435" watchObservedRunningTime="2026-09-05 08:07:19.191366625 +0000 UTC m=+1141.961508961"
Sep 05 08:07:19.586291 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:19.586259    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"
Sep 05 08:07:19.591346 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:19.591325    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"
Sep 05 08:07:20.174095 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:20.174062    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"
Sep 05 08:07:20.175177 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:20.175154    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"
Sep 05 08:07:21.131357 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:21.131324    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:07:21.238809 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:21.238781    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:07:22.271380 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:22.271347    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:07:22.370357 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:22.370321    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:07:22.482874 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:22.482823    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:07:22.898704 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:22.898672    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:07:22.992103 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:22.992076    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:07:23.042613 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:23.042582    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4b62745d-69fc6f6f5c-thkhg"]
Sep 05 08:07:23.045165 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:23.045151    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4b62745d-69fc6f6f5c-thkhg"
Sep 05 08:07:23.049436 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:23.049418    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-4b62745d\""
Sep 05 08:07:23.049566 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:23.049420    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-4b62745d-dockercfg-26s5t\""
Sep 05 08:07:23.082822 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:23.082794    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4b62745d-69fc6f6f5c-thkhg"]
Sep 05 08:07:23.129798 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:23.129761    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5ae26f8-33c5-40d8-9eb7-0dd9c07bdc03-plugins-config-volume\") pod \"payload-processing-e2e-mt-4b62745d-69fc6f6f5c-thkhg\" (UID: \"a5ae26f8-33c5-40d8-9eb7-0dd9c07bdc03\") " pod="openshift-ingress/payload-processing-e2e-mt-4b62745d-69fc6f6f5c-thkhg"
Sep 05 08:07:23.129995 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:23.129926    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zd8bb\" (UniqueName: \"kubernetes.io/projected/a5ae26f8-33c5-40d8-9eb7-0dd9c07bdc03-kube-api-access-zd8bb\") pod \"payload-processing-e2e-mt-4b62745d-69fc6f6f5c-thkhg\" (UID: \"a5ae26f8-33c5-40d8-9eb7-0dd9c07bdc03\") " pod="openshift-ingress/payload-processing-e2e-mt-4b62745d-69fc6f6f5c-thkhg"
Sep 05 08:07:23.231589 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:23.231493    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5ae26f8-33c5-40d8-9eb7-0dd9c07bdc03-plugins-config-volume\") pod \"payload-processing-e2e-mt-4b62745d-69fc6f6f5c-thkhg\" (UID: \"a5ae26f8-33c5-40d8-9eb7-0dd9c07bdc03\") " pod="openshift-ingress/payload-processing-e2e-mt-4b62745d-69fc6f6f5c-thkhg"
Sep 05 08:07:23.231589 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:23.231585    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zd8bb\" (UniqueName: \"kubernetes.io/projected/a5ae26f8-33c5-40d8-9eb7-0dd9c07bdc03-kube-api-access-zd8bb\") pod \"payload-processing-e2e-mt-4b62745d-69fc6f6f5c-thkhg\" (UID: \"a5ae26f8-33c5-40d8-9eb7-0dd9c07bdc03\") " pod="openshift-ingress/payload-processing-e2e-mt-4b62745d-69fc6f6f5c-thkhg"
Sep 05 08:07:23.232312 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:23.232281    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5ae26f8-33c5-40d8-9eb7-0dd9c07bdc03-plugins-config-volume\") pod \"payload-processing-e2e-mt-4b62745d-69fc6f6f5c-thkhg\" (UID: \"a5ae26f8-33c5-40d8-9eb7-0dd9c07bdc03\") " pod="openshift-ingress/payload-processing-e2e-mt-4b62745d-69fc6f6f5c-thkhg"
Sep 05 08:07:23.240956 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:23.240925    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zd8bb\" (UniqueName: \"kubernetes.io/projected/a5ae26f8-33c5-40d8-9eb7-0dd9c07bdc03-kube-api-access-zd8bb\") pod \"payload-processing-e2e-mt-4b62745d-69fc6f6f5c-thkhg\" (UID: \"a5ae26f8-33c5-40d8-9eb7-0dd9c07bdc03\") " pod="openshift-ingress/payload-processing-e2e-mt-4b62745d-69fc6f6f5c-thkhg"
Sep 05 08:07:23.355339 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:23.355309    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4b62745d-69fc6f6f5c-thkhg"
Sep 05 08:07:23.630806 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:23.630778    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4b62745d-69fc6f6f5c-thkhg"]
Sep 05 08:07:23.632745 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:07:23.632720    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5ae26f8_33c5_40d8_9eb7_0dd9c07bdc03.slice/crio-43619d7853351b5b477adad93c101710d9070bcf6f437be0b4d53d573a64d89c WatchSource:0}: Error finding container 43619d7853351b5b477adad93c101710d9070bcf6f437be0b4d53d573a64d89c: Status 404 returned error can't find the container with id 43619d7853351b5b477adad93c101710d9070bcf6f437be0b4d53d573a64d89c
Sep 05 08:07:24.201515 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:24.201480    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4b62745d-69fc6f6f5c-thkhg" event={"ID":"a5ae26f8-33c5-40d8-9eb7-0dd9c07bdc03","Type":"ContainerStarted","Data":"43619d7853351b5b477adad93c101710d9070bcf6f437be0b4d53d573a64d89c"}
Sep 05 08:07:26.219111 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:26.219067    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4b62745d-69fc6f6f5c-thkhg" event={"ID":"a5ae26f8-33c5-40d8-9eb7-0dd9c07bdc03","Type":"ContainerStarted","Data":"781cbe7c60cc7e1f4b63c9806dc65359117305b092f19b648ab82c421c251c40"}
Sep 05 08:07:26.219577 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:26.219211    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-4b62745d-69fc6f6f5c-thkhg"
Sep 05 08:07:26.259688 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:26.259639    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-4b62745d-69fc6f6f5c-thkhg" podStartSLOduration=1.603964881 podStartE2EDuration="3.259621908s" totalImagesPullingTime="1.655657027s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:07:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:07:26.257354756 +0000 UTC m=+1149.027497120" watchObservedRunningTime="2026-09-05 08:07:26.259621908 +0000 UTC m=+1149.029764245"
Sep 05 08:07:27.486986 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:27.486950    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-8c2e20/auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl"]
Sep 05 08:07:27.489447 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:27.489426    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-8c2e20/auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl"
Sep 05 08:07:27.492353 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:27.492329    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-8c2e20\"/\"openshift-service-ca.crt\""
Sep 05 08:07:27.492475 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:27.492447    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-8c2e20\"/\"auth-test-model-8c2e20-kserve-self-signed-certs\""
Sep 05 08:07:27.493068 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:27.493051    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-8c2e20\"/\"kube-root-ca.crt\""
Sep 05 08:07:27.493159 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:27.493107    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-8c2e20\"/\"default-dockercfg-ddq4m\""
Sep 05 08:07:27.505360 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:27.505338    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-8c2e20/auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl"]
Sep 05 08:07:27.522438 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:27.522398    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:07:27.676002 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:27.675964    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16510468-4894-4bc4-b7cb-884a86f17437-tmp-dir\") pod \"auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl\" (UID: \"16510468-4894-4bc4-b7cb-884a86f17437\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl"
Sep 05 08:07:27.676184 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:27.676023    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16510468-4894-4bc4-b7cb-884a86f17437-home\") pod \"auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl\" (UID: \"16510468-4894-4bc4-b7cb-884a86f17437\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl"
Sep 05 08:07:27.676184 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:27.676074    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16510468-4894-4bc4-b7cb-884a86f17437-tls-certs\") pod \"auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl\" (UID: \"16510468-4894-4bc4-b7cb-884a86f17437\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl"
Sep 05 08:07:27.676184 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:27.676095    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16510468-4894-4bc4-b7cb-884a86f17437-dshm\") pod \"auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl\" (UID: \"16510468-4894-4bc4-b7cb-884a86f17437\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl"
Sep 05 08:07:27.676184 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:27.676148    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ll8s2\" (UniqueName: \"kubernetes.io/projected/16510468-4894-4bc4-b7cb-884a86f17437-kube-api-access-ll8s2\") pod \"auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl\" (UID: \"16510468-4894-4bc4-b7cb-884a86f17437\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl"
Sep 05 08:07:27.676319 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:27.676209    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16510468-4894-4bc4-b7cb-884a86f17437-model-cache\") pod \"auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl\" (UID: \"16510468-4894-4bc4-b7cb-884a86f17437\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl"
Sep 05 08:07:27.777776 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:27.777693    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16510468-4894-4bc4-b7cb-884a86f17437-model-cache\") pod \"auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl\" (UID: \"16510468-4894-4bc4-b7cb-884a86f17437\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl"
Sep 05 08:07:27.777776 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:27.777751    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16510468-4894-4bc4-b7cb-884a86f17437-tmp-dir\") pod \"auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl\" (UID: \"16510468-4894-4bc4-b7cb-884a86f17437\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl"
Sep 05 08:07:27.777997 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:27.777783    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16510468-4894-4bc4-b7cb-884a86f17437-home\") pod \"auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl\" (UID: \"16510468-4894-4bc4-b7cb-884a86f17437\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl"
Sep 05 08:07:27.777997 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:27.777803    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16510468-4894-4bc4-b7cb-884a86f17437-tls-certs\") pod \"auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl\" (UID: \"16510468-4894-4bc4-b7cb-884a86f17437\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl"
Sep 05 08:07:27.777997 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:27.777819    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16510468-4894-4bc4-b7cb-884a86f17437-dshm\") pod \"auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl\" (UID: \"16510468-4894-4bc4-b7cb-884a86f17437\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl"
Sep 05 08:07:27.777997 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:27.777863    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ll8s2\" (UniqueName: \"kubernetes.io/projected/16510468-4894-4bc4-b7cb-884a86f17437-kube-api-access-ll8s2\") pod \"auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl\" (UID: \"16510468-4894-4bc4-b7cb-884a86f17437\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl"
Sep 05 08:07:27.778192 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:27.778171    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16510468-4894-4bc4-b7cb-884a86f17437-model-cache\") pod \"auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl\" (UID: \"16510468-4894-4bc4-b7cb-884a86f17437\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl"
Sep 05 08:07:27.778227 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:27.778215    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16510468-4894-4bc4-b7cb-884a86f17437-tmp-dir\") pod \"auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl\" (UID: \"16510468-4894-4bc4-b7cb-884a86f17437\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl"
Sep 05 08:07:27.778307 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:27.778249    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16510468-4894-4bc4-b7cb-884a86f17437-home\") pod \"auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl\" (UID: \"16510468-4894-4bc4-b7cb-884a86f17437\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl"
Sep 05 08:07:27.780204 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:27.780183    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16510468-4894-4bc4-b7cb-884a86f17437-dshm\") pod \"auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl\" (UID: \"16510468-4894-4bc4-b7cb-884a86f17437\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl"
Sep 05 08:07:27.780607 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:27.780583    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16510468-4894-4bc4-b7cb-884a86f17437-tls-certs\") pod \"auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl\" (UID: \"16510468-4894-4bc4-b7cb-884a86f17437\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl"
Sep 05 08:07:27.786457 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:27.786437    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ll8s2\" (UniqueName: \"kubernetes.io/projected/16510468-4894-4bc4-b7cb-884a86f17437-kube-api-access-ll8s2\") pod \"auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl\" (UID: \"16510468-4894-4bc4-b7cb-884a86f17437\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl"
Sep 05 08:07:27.798440 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:27.798418    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-8c2e20/auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl"
Sep 05 08:07:28.087430 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:28.087403    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-8c2e20/auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl"]
Sep 05 08:07:28.089783 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:07:28.089758    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod16510468_4894_4bc4_b7cb_884a86f17437.slice/crio-fc6a9b3b69b80efd4780c8c818cce5d31a93f82cb2ac4d17b3cbb010548dc330 WatchSource:0}: Error finding container fc6a9b3b69b80efd4780c8c818cce5d31a93f82cb2ac4d17b3cbb010548dc330: Status 404 returned error can't find the container with id fc6a9b3b69b80efd4780c8c818cce5d31a93f82cb2ac4d17b3cbb010548dc330
Sep 05 08:07:28.231107 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:28.231070    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-8c2e20/auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl" event={"ID":"16510468-4894-4bc4-b7cb-884a86f17437","Type":"ContainerStarted","Data":"f689d72a14cad082515cc3b92fff671a916b8cf538dbd3a0ca1a0000f8a3a41e"}
Sep 05 08:07:28.231250 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:28.231123    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-8c2e20/auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl" event={"ID":"16510468-4894-4bc4-b7cb-884a86f17437","Type":"ContainerStarted","Data":"fc6a9b3b69b80efd4780c8c818cce5d31a93f82cb2ac4d17b3cbb010548dc330"}
Sep 05 08:07:28.250811 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:28.250761    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-8c2e20/auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl" podStartSLOduration=1.250743672 podStartE2EDuration="1.250743672s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:07:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:07:28.250274354 +0000 UTC m=+1151.020416712" watchObservedRunningTime="2026-09-05 08:07:28.250743672 +0000 UTC m=+1151.020886009"
Sep 05 08:07:37.227080 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:37.227048    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-4b62745d-69fc6f6f5c-thkhg"
Sep 05 08:07:37.799546 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:37.799508    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-8c2e20/auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl"
Sep 05 08:07:37.799717 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:37.799564    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-8c2e20/auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl"
Sep 05 08:07:37.817899 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:37.817874    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-8c2e20/auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl"
Sep 05 08:07:38.296179 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:38.296141    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-8c2e20/auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl"
Sep 05 08:07:39.291775 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:39.291740    2563 generic.go:356] "Generic (PLEG): container finished" podID="262bcc62-34d3-4363-9189-6a8ab3d96838" containerID="8ec9078a44fd31009410e261853266ec6c2c8d65864274ee31ead39fbc327db5" exitCode=0
Sep 05 08:07:39.292003 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:39.291814    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5" event={"ID":"262bcc62-34d3-4363-9189-6a8ab3d96838","Type":"ContainerDied","Data":"8ec9078a44fd31009410e261853266ec6c2c8d65864274ee31ead39fbc327db5"}
Sep 05 08:07:39.292274 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:39.292258    2563 scope.go:117] "RemoveContainer" containerID="8ec9078a44fd31009410e261853266ec6c2c8d65864274ee31ead39fbc327db5"
Sep 05 08:07:39.293341 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:39.293314    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 05 08:07:39.293402 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:39.293383    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 05 08:07:39.293442 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:39.293419    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 05 08:07:40.313557 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:40.313518    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5" event={"ID":"262bcc62-34d3-4363-9189-6a8ab3d96838","Type":"ContainerStarted","Data":"a8b559f36542b6e927a722e2317f8ca2366a09d0d3774129b98719a6dad56a51"}
Sep 05 08:07:40.585767 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:40.585680    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"
Sep 05 08:07:40.586016 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:40.585936    2563 patch_prober.go:28] interesting pod/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 05 08:07:40.586016 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:40.585997    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5" podUID="262bcc62-34d3-4363-9189-6a8ab3d96838" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 05 08:07:41.586541 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:41.586499    2563 patch_prober.go:28] interesting pod/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 05 08:07:41.587088 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:41.586571    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5" podUID="262bcc62-34d3-4363-9189-6a8ab3d96838" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 05 08:07:42.586449 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:42.586410    2563 patch_prober.go:28] interesting pod/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 05 08:07:42.586736 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:42.586710    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5" podUID="262bcc62-34d3-4363-9189-6a8ab3d96838" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 05 08:07:43.586549 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:43.586511    2563 patch_prober.go:28] interesting pod/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 05 08:07:43.586834 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:43.586806    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5" podUID="262bcc62-34d3-4363-9189-6a8ab3d96838" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 05 08:07:44.586058 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:44.586020    2563 patch_prober.go:28] interesting pod/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 05 08:07:44.586343 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:44.586315    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5" podUID="262bcc62-34d3-4363-9189-6a8ab3d96838" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 05 08:07:45.736812 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:45.736766    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"
Sep 05 08:07:45.737294 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:45.737151    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"
Sep 05 08:07:45.738111 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:45.738092    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"
Sep 05 08:07:46.148622 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:46.148589    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:07:56.969574 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:56.969541    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:07:57.065731 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:57.065697    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:07:57.606929 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:57.606898    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-5ef325/auth-test-model-5ef325-kserve-845c94757f-hjr9t"]
Sep 05 08:07:57.609954 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:57.609931    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-5ef325/auth-test-model-5ef325-kserve-845c94757f-hjr9t"
Sep 05 08:07:57.613591 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:57.613566    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-5ef325\"/\"kube-root-ca.crt\""
Sep 05 08:07:57.614374 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:57.614359    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-5ef325\"/\"default-dockercfg-fqjc2\""
Sep 05 08:07:57.614554 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:57.614542    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-5ef325\"/\"openshift-service-ca.crt\""
Sep 05 08:07:57.614657 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:57.614636    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-5ef325\"/\"auth-test-model-5ef325-kserve-self-signed-certs\""
Sep 05 08:07:57.624457 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:57.624436    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-5ef325/auth-test-model-5ef325-kserve-845c94757f-hjr9t"]
Sep 05 08:07:57.647682 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:57.647657    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:07:57.753373 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:57.753340    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/429b258a-caaf-4cef-aa49-fb975d367945-tls-certs\") pod \"auth-test-model-5ef325-kserve-845c94757f-hjr9t\" (UID: \"429b258a-caaf-4cef-aa49-fb975d367945\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/auth-test-model-5ef325-kserve-845c94757f-hjr9t"
Sep 05 08:07:57.753373 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:57.753370    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/429b258a-caaf-4cef-aa49-fb975d367945-home\") pod \"auth-test-model-5ef325-kserve-845c94757f-hjr9t\" (UID: \"429b258a-caaf-4cef-aa49-fb975d367945\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/auth-test-model-5ef325-kserve-845c94757f-hjr9t"
Sep 05 08:07:57.753571 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:57.753389    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/429b258a-caaf-4cef-aa49-fb975d367945-tmp-dir\") pod \"auth-test-model-5ef325-kserve-845c94757f-hjr9t\" (UID: \"429b258a-caaf-4cef-aa49-fb975d367945\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/auth-test-model-5ef325-kserve-845c94757f-hjr9t"
Sep 05 08:07:57.753571 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:57.753451    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5zn8\" (UniqueName: \"kubernetes.io/projected/429b258a-caaf-4cef-aa49-fb975d367945-kube-api-access-t5zn8\") pod \"auth-test-model-5ef325-kserve-845c94757f-hjr9t\" (UID: \"429b258a-caaf-4cef-aa49-fb975d367945\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/auth-test-model-5ef325-kserve-845c94757f-hjr9t"
Sep 05 08:07:57.753571 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:57.753499    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/429b258a-caaf-4cef-aa49-fb975d367945-dshm\") pod \"auth-test-model-5ef325-kserve-845c94757f-hjr9t\" (UID: \"429b258a-caaf-4cef-aa49-fb975d367945\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/auth-test-model-5ef325-kserve-845c94757f-hjr9t"
Sep 05 08:07:57.753571 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:57.753543    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/429b258a-caaf-4cef-aa49-fb975d367945-model-cache\") pod \"auth-test-model-5ef325-kserve-845c94757f-hjr9t\" (UID: \"429b258a-caaf-4cef-aa49-fb975d367945\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/auth-test-model-5ef325-kserve-845c94757f-hjr9t"
Sep 05 08:07:57.854715 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:57.854677    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/429b258a-caaf-4cef-aa49-fb975d367945-model-cache\") pod \"auth-test-model-5ef325-kserve-845c94757f-hjr9t\" (UID: \"429b258a-caaf-4cef-aa49-fb975d367945\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/auth-test-model-5ef325-kserve-845c94757f-hjr9t"
Sep 05 08:07:57.854949 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:57.854773    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/429b258a-caaf-4cef-aa49-fb975d367945-tls-certs\") pod \"auth-test-model-5ef325-kserve-845c94757f-hjr9t\" (UID: \"429b258a-caaf-4cef-aa49-fb975d367945\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/auth-test-model-5ef325-kserve-845c94757f-hjr9t"
Sep 05 08:07:57.854949 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:57.854799    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/429b258a-caaf-4cef-aa49-fb975d367945-home\") pod \"auth-test-model-5ef325-kserve-845c94757f-hjr9t\" (UID: \"429b258a-caaf-4cef-aa49-fb975d367945\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/auth-test-model-5ef325-kserve-845c94757f-hjr9t"
Sep 05 08:07:57.854949 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:57.854824    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/429b258a-caaf-4cef-aa49-fb975d367945-tmp-dir\") pod \"auth-test-model-5ef325-kserve-845c94757f-hjr9t\" (UID: \"429b258a-caaf-4cef-aa49-fb975d367945\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/auth-test-model-5ef325-kserve-845c94757f-hjr9t"
Sep 05 08:07:57.854949 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:57.854880    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5zn8\" (UniqueName: \"kubernetes.io/projected/429b258a-caaf-4cef-aa49-fb975d367945-kube-api-access-t5zn8\") pod \"auth-test-model-5ef325-kserve-845c94757f-hjr9t\" (UID: \"429b258a-caaf-4cef-aa49-fb975d367945\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/auth-test-model-5ef325-kserve-845c94757f-hjr9t"
Sep 05 08:07:57.854949 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:57.854943    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/429b258a-caaf-4cef-aa49-fb975d367945-dshm\") pod \"auth-test-model-5ef325-kserve-845c94757f-hjr9t\" (UID: \"429b258a-caaf-4cef-aa49-fb975d367945\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/auth-test-model-5ef325-kserve-845c94757f-hjr9t"
Sep 05 08:07:57.855248 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:57.855223    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/429b258a-caaf-4cef-aa49-fb975d367945-model-cache\") pod \"auth-test-model-5ef325-kserve-845c94757f-hjr9t\" (UID: \"429b258a-caaf-4cef-aa49-fb975d367945\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/auth-test-model-5ef325-kserve-845c94757f-hjr9t"
Sep 05 08:07:57.855316 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:57.855235    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/429b258a-caaf-4cef-aa49-fb975d367945-tmp-dir\") pod \"auth-test-model-5ef325-kserve-845c94757f-hjr9t\" (UID: \"429b258a-caaf-4cef-aa49-fb975d367945\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/auth-test-model-5ef325-kserve-845c94757f-hjr9t"
Sep 05 08:07:57.855316 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:57.855286    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/429b258a-caaf-4cef-aa49-fb975d367945-home\") pod \"auth-test-model-5ef325-kserve-845c94757f-hjr9t\" (UID: \"429b258a-caaf-4cef-aa49-fb975d367945\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/auth-test-model-5ef325-kserve-845c94757f-hjr9t"
Sep 05 08:07:57.857368 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:57.857322    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/429b258a-caaf-4cef-aa49-fb975d367945-dshm\") pod \"auth-test-model-5ef325-kserve-845c94757f-hjr9t\" (UID: \"429b258a-caaf-4cef-aa49-fb975d367945\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/auth-test-model-5ef325-kserve-845c94757f-hjr9t"
Sep 05 08:07:57.857541 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:57.857523    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/429b258a-caaf-4cef-aa49-fb975d367945-tls-certs\") pod \"auth-test-model-5ef325-kserve-845c94757f-hjr9t\" (UID: \"429b258a-caaf-4cef-aa49-fb975d367945\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/auth-test-model-5ef325-kserve-845c94757f-hjr9t"
Sep 05 08:07:57.863633 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:57.863610    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5zn8\" (UniqueName: \"kubernetes.io/projected/429b258a-caaf-4cef-aa49-fb975d367945-kube-api-access-t5zn8\") pod \"auth-test-model-5ef325-kserve-845c94757f-hjr9t\" (UID: \"429b258a-caaf-4cef-aa49-fb975d367945\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/auth-test-model-5ef325-kserve-845c94757f-hjr9t"
Sep 05 08:07:57.919181 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:57.919149    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-5ef325/auth-test-model-5ef325-kserve-845c94757f-hjr9t"
Sep 05 08:07:58.194484 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:58.194456    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-5ef325/auth-test-model-5ef325-kserve-845c94757f-hjr9t"]
Sep 05 08:07:58.197115 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:07:58.197091    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod429b258a_caaf_4cef_aa49_fb975d367945.slice/crio-8fa903a800cf7d1cc7532dd19fe3f211a43a3515452594f2be81a570ef3ca767 WatchSource:0}: Error finding container 8fa903a800cf7d1cc7532dd19fe3f211a43a3515452594f2be81a570ef3ca767: Status 404 returned error can't find the container with id 8fa903a800cf7d1cc7532dd19fe3f211a43a3515452594f2be81a570ef3ca767
Sep 05 08:07:58.409227 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:58.409191    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-5ef325/auth-test-model-5ef325-kserve-845c94757f-hjr9t" event={"ID":"429b258a-caaf-4cef-aa49-fb975d367945","Type":"ContainerStarted","Data":"37a0125a3c2dfcbd291a766e7ab7e13041b2289389ed4986b0a10c424abd2041"}
Sep 05 08:07:58.409405 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:58.409233    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-5ef325/auth-test-model-5ef325-kserve-845c94757f-hjr9t" event={"ID":"429b258a-caaf-4cef-aa49-fb975d367945","Type":"ContainerStarted","Data":"8fa903a800cf7d1cc7532dd19fe3f211a43a3515452594f2be81a570ef3ca767"}
Sep 05 08:07:58.432266 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:58.432212    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-5ef325/auth-test-model-5ef325-kserve-845c94757f-hjr9t" podStartSLOduration=1.432198057 podStartE2EDuration="1.432198057s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:07:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:07:58.430443342 +0000 UTC m=+1181.200585692" watchObservedRunningTime="2026-09-05 08:07:58.432198057 +0000 UTC m=+1181.202340395"
Sep 05 08:07:59.609607 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:07:59.609571    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:08:02.604558 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:02.604519    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-4b62745d/e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s"]
Sep 05 08:08:02.607083 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:02.607058    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-4b62745d/e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s"
Sep 05 08:08:02.612900 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:02.612880    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-4b62745d\"/\"kube-root-ca.crt\""
Sep 05 08:08:02.613552 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:02.613212    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-4b62745d\"/\"e2e-lifecycle-model-4b62745d-kserve-self-signed-certs\""
Sep 05 08:08:02.613552 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:02.613226    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-4b62745d\"/\"default-dockercfg-g9d9k\""
Sep 05 08:08:02.613552 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:02.613238    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-4b62745d\"/\"openshift-service-ca.crt\""
Sep 05 08:08:02.617667 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:02.617639    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-4b62745d/e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s"]
Sep 05 08:08:02.702630 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:02.702595    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1cb7513-5db3-4857-8a1c-751c5df2158b-tls-certs\") pod \"e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s\" (UID: \"d1cb7513-5db3-4857-8a1c-751c5df2158b\") " pod="ai-tenant-e2e-mt-4b62745d/e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s"
Sep 05 08:08:02.702811 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:02.702672    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1cb7513-5db3-4857-8a1c-751c5df2158b-model-cache\") pod \"e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s\" (UID: \"d1cb7513-5db3-4857-8a1c-751c5df2158b\") " pod="ai-tenant-e2e-mt-4b62745d/e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s"
Sep 05 08:08:02.702811 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:02.702691    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1cb7513-5db3-4857-8a1c-751c5df2158b-tmp-dir\") pod \"e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s\" (UID: \"d1cb7513-5db3-4857-8a1c-751c5df2158b\") " pod="ai-tenant-e2e-mt-4b62745d/e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s"
Sep 05 08:08:02.702811 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:02.702708    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1cb7513-5db3-4857-8a1c-751c5df2158b-dshm\") pod \"e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s\" (UID: \"d1cb7513-5db3-4857-8a1c-751c5df2158b\") " pod="ai-tenant-e2e-mt-4b62745d/e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s"
Sep 05 08:08:02.702811 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:02.702726    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcrzx\" (UniqueName: \"kubernetes.io/projected/d1cb7513-5db3-4857-8a1c-751c5df2158b-kube-api-access-wcrzx\") pod \"e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s\" (UID: \"d1cb7513-5db3-4857-8a1c-751c5df2158b\") " pod="ai-tenant-e2e-mt-4b62745d/e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s"
Sep 05 08:08:02.702811 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:02.702779    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1cb7513-5db3-4857-8a1c-751c5df2158b-home\") pod \"e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s\" (UID: \"d1cb7513-5db3-4857-8a1c-751c5df2158b\") " pod="ai-tenant-e2e-mt-4b62745d/e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s"
Sep 05 08:08:02.728743 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:02.728711    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:08:02.804154 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:02.804123    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcrzx\" (UniqueName: \"kubernetes.io/projected/d1cb7513-5db3-4857-8a1c-751c5df2158b-kube-api-access-wcrzx\") pod \"e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s\" (UID: \"d1cb7513-5db3-4857-8a1c-751c5df2158b\") " pod="ai-tenant-e2e-mt-4b62745d/e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s"
Sep 05 08:08:02.804360 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:02.804171    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1cb7513-5db3-4857-8a1c-751c5df2158b-home\") pod \"e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s\" (UID: \"d1cb7513-5db3-4857-8a1c-751c5df2158b\") " pod="ai-tenant-e2e-mt-4b62745d/e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s"
Sep 05 08:08:02.804360 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:02.804241    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1cb7513-5db3-4857-8a1c-751c5df2158b-tls-certs\") pod \"e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s\" (UID: \"d1cb7513-5db3-4857-8a1c-751c5df2158b\") " pod="ai-tenant-e2e-mt-4b62745d/e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s"
Sep 05 08:08:02.804360 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:02.804344    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1cb7513-5db3-4857-8a1c-751c5df2158b-model-cache\") pod \"e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s\" (UID: \"d1cb7513-5db3-4857-8a1c-751c5df2158b\") " pod="ai-tenant-e2e-mt-4b62745d/e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s"
Sep 05 08:08:02.804615 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:02.804483    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1cb7513-5db3-4857-8a1c-751c5df2158b-tmp-dir\") pod \"e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s\" (UID: \"d1cb7513-5db3-4857-8a1c-751c5df2158b\") " pod="ai-tenant-e2e-mt-4b62745d/e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s"
Sep 05 08:08:02.804615 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:02.804534    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1cb7513-5db3-4857-8a1c-751c5df2158b-dshm\") pod \"e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s\" (UID: \"d1cb7513-5db3-4857-8a1c-751c5df2158b\") " pod="ai-tenant-e2e-mt-4b62745d/e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s"
Sep 05 08:08:02.804726 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:02.804677    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1cb7513-5db3-4857-8a1c-751c5df2158b-home\") pod \"e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s\" (UID: \"d1cb7513-5db3-4857-8a1c-751c5df2158b\") " pod="ai-tenant-e2e-mt-4b62745d/e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s"
Sep 05 08:08:02.804785 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:02.804729    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1cb7513-5db3-4857-8a1c-751c5df2158b-model-cache\") pod \"e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s\" (UID: \"d1cb7513-5db3-4857-8a1c-751c5df2158b\") " pod="ai-tenant-e2e-mt-4b62745d/e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s"
Sep 05 08:08:02.804933 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:02.804909    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1cb7513-5db3-4857-8a1c-751c5df2158b-tmp-dir\") pod \"e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s\" (UID: \"d1cb7513-5db3-4857-8a1c-751c5df2158b\") " pod="ai-tenant-e2e-mt-4b62745d/e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s"
Sep 05 08:08:02.806861 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:02.806822    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1cb7513-5db3-4857-8a1c-751c5df2158b-dshm\") pod \"e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s\" (UID: \"d1cb7513-5db3-4857-8a1c-751c5df2158b\") " pod="ai-tenant-e2e-mt-4b62745d/e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s"
Sep 05 08:08:02.806953 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:02.806916    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1cb7513-5db3-4857-8a1c-751c5df2158b-tls-certs\") pod \"e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s\" (UID: \"d1cb7513-5db3-4857-8a1c-751c5df2158b\") " pod="ai-tenant-e2e-mt-4b62745d/e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s"
Sep 05 08:08:02.812664 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:02.812646    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcrzx\" (UniqueName: \"kubernetes.io/projected/d1cb7513-5db3-4857-8a1c-751c5df2158b-kube-api-access-wcrzx\") pod \"e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s\" (UID: \"d1cb7513-5db3-4857-8a1c-751c5df2158b\") " pod="ai-tenant-e2e-mt-4b62745d/e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s"
Sep 05 08:08:02.918051 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:02.917960    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-4b62745d/e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s"
Sep 05 08:08:03.192887 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:03.192853    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-4b62745d/e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s"]
Sep 05 08:08:03.194682 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:08:03.194652    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd1cb7513_5db3_4857_8a1c_751c5df2158b.slice/crio-5aae96afa8c319f436c98143f94697f2657e80d906b71aab190b232e0e658747 WatchSource:0}: Error finding container 5aae96afa8c319f436c98143f94697f2657e80d906b71aab190b232e0e658747: Status 404 returned error can't find the container with id 5aae96afa8c319f436c98143f94697f2657e80d906b71aab190b232e0e658747
Sep 05 08:08:03.445306 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:03.445207    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-4b62745d/e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s" event={"ID":"d1cb7513-5db3-4857-8a1c-751c5df2158b","Type":"ContainerStarted","Data":"fde1389f048c1b412b04eaf3d8ac31343d036df74ef0dffbb0106f0b25e995ed"}
Sep 05 08:08:03.445306 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:03.445253    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-4b62745d/e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s" event={"ID":"d1cb7513-5db3-4857-8a1c-751c5df2158b","Type":"ContainerStarted","Data":"5aae96afa8c319f436c98143f94697f2657e80d906b71aab190b232e0e658747"}
Sep 05 08:08:03.476983 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:03.476925    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-4b62745d/e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s" podStartSLOduration=1.476906341 podStartE2EDuration="1.476906341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:08:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:08:03.474530491 +0000 UTC m=+1186.244672844" watchObservedRunningTime="2026-09-05 08:08:03.476906341 +0000 UTC m=+1186.247048714"
Sep 05 08:08:07.919262 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:07.919221    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-5ef325/auth-test-model-5ef325-kserve-845c94757f-hjr9t"
Sep 05 08:08:07.919674 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:07.919294    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-5ef325/auth-test-model-5ef325-kserve-845c94757f-hjr9t"
Sep 05 08:08:07.931733 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:07.931709    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-5ef325/auth-test-model-5ef325-kserve-845c94757f-hjr9t"
Sep 05 08:08:08.484756 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:08.484727    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-5ef325/auth-test-model-5ef325-kserve-845c94757f-hjr9t"
Sep 05 08:08:12.918284 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:12.918248    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-4b62745d/e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s"
Sep 05 08:08:12.918284 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:12.918292    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-4b62745d/e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s"
Sep 05 08:08:12.934178 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:12.934146    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-4b62745d/e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s"
Sep 05 08:08:13.509635 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:13.509604    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-4b62745d/e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s"
Sep 05 08:08:17.738458 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:17.738431    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2bzp8_569cffae-1c5c-4b02-af57-9ca53c455e4c/ovn-acl-logging/0.log"
Sep 05 08:08:17.740171 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:17.740150    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2bzp8_569cffae-1c5c-4b02-af57-9ca53c455e4c/ovn-acl-logging/0.log"
Sep 05 08:08:17.809319 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:17.809292    2563 scope.go:117] "RemoveContainer" containerID="4d697d461f5ba22743c7c41e7f421385bd2cc784f6a0fa0bdedcd4759985114c"
Sep 05 08:08:17.817581 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:17.817561    2563 scope.go:117] "RemoveContainer" containerID="3cc4f6737994b42efa3eec658dfc64ccfdff35a54c112bab7007d834aa86458f"
Sep 05 08:08:24.182458 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:24.182405    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:08:31.266291 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:31.266255    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4b62745d-kl6tn"]
Sep 05 08:08:31.270762 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:31.270743    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4b62745d-kl6tn"
Sep 05 08:08:31.273376 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:31.273352    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mpmmp\""
Sep 05 08:08:31.273509 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:31.273397    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 05 08:08:31.280947 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:31.280918    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4b62745d-kl6tn"]
Sep 05 08:08:31.371279 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:31.371247    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:08:31.379604 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:31.379569    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3949d1f-57a5-40d1-9e75-c5c4b786b8c1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-4b62745d-kl6tn\" (UID: \"f3949d1f-57a5-40d1-9e75-c5c4b786b8c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4b62745d-kl6tn"
Sep 05 08:08:31.480190 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:31.480159    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3949d1f-57a5-40d1-9e75-c5c4b786b8c1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-4b62745d-kl6tn\" (UID: \"f3949d1f-57a5-40d1-9e75-c5c4b786b8c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4b62745d-kl6tn"
Sep 05 08:08:31.480809 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:31.480781    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3949d1f-57a5-40d1-9e75-c5c4b786b8c1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-4b62745d-kl6tn\" (UID: \"f3949d1f-57a5-40d1-9e75-c5c4b786b8c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4b62745d-kl6tn"
Sep 05 08:08:31.582411 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:31.582365    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4b62745d-kl6tn"
Sep 05 08:08:31.852575 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:31.852547    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4b62745d-kl6tn"]
Sep 05 08:08:31.855430 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:08:31.855403    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3949d1f_57a5_40d1_9e75_c5c4b786b8c1.slice/crio-ace6c2dd5ae93bbc8964f3c6f9000cd1ad304cf093d8a23dca8ea23fddaf4097 WatchSource:0}: Error finding container ace6c2dd5ae93bbc8964f3c6f9000cd1ad304cf093d8a23dca8ea23fddaf4097: Status 404 returned error can't find the container with id ace6c2dd5ae93bbc8964f3c6f9000cd1ad304cf093d8a23dca8ea23fddaf4097
Sep 05 08:08:32.604862 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:32.604808    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4b62745d-kl6tn" event={"ID":"f3949d1f-57a5-40d1-9e75-c5c4b786b8c1","Type":"ContainerStarted","Data":"51ed61e87a980070ff3f54b4a10c818d3a927719d7c6e16f76f110aade645081"}
Sep 05 08:08:32.605334 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:32.604899    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4b62745d-kl6tn" event={"ID":"f3949d1f-57a5-40d1-9e75-c5c4b786b8c1","Type":"ContainerStarted","Data":"ace6c2dd5ae93bbc8964f3c6f9000cd1ad304cf093d8a23dca8ea23fddaf4097"}
Sep 05 08:08:32.624787 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:32.624691    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4b62745d-kl6tn" podStartSLOduration=1.624669283 podStartE2EDuration="1.624669283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:08:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:08:32.620943697 +0000 UTC m=+1215.391086066" watchObservedRunningTime="2026-09-05 08:08:32.624669283 +0000 UTC m=+1215.394811621"
Sep 05 08:08:33.611651 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:33.611618    2563 generic.go:356] "Generic (PLEG): container finished" podID="f3949d1f-57a5-40d1-9e75-c5c4b786b8c1" containerID="51ed61e87a980070ff3f54b4a10c818d3a927719d7c6e16f76f110aade645081" exitCode=0
Sep 05 08:08:33.612047 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:33.611695    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4b62745d-kl6tn" event={"ID":"f3949d1f-57a5-40d1-9e75-c5c4b786b8c1","Type":"ContainerDied","Data":"51ed61e87a980070ff3f54b4a10c818d3a927719d7c6e16f76f110aade645081"}
Sep 05 08:08:34.752421 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:34.752393    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4b62745d-kl6tn"
Sep 05 08:08:34.811917 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:34.811889    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3949d1f-57a5-40d1-9e75-c5c4b786b8c1-maas-api-service-ca\") pod \"f3949d1f-57a5-40d1-9e75-c5c4b786b8c1\" (UID: \"f3949d1f-57a5-40d1-9e75-c5c4b786b8c1\") "
Sep 05 08:08:34.812225 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:34.812200    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3949d1f-57a5-40d1-9e75-c5c4b786b8c1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f3949d1f-57a5-40d1-9e75-c5c4b786b8c1" (UID: "f3949d1f-57a5-40d1-9e75-c5c4b786b8c1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:08:34.913372 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:34.913289    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3949d1f-57a5-40d1-9e75-c5c4b786b8c1-maas-api-service-ca\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:08:35.623963 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:35.623934    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4b62745d-kl6tn"
Sep 05 08:08:35.624139 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:35.623981    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4b62745d-kl6tn" event={"ID":"f3949d1f-57a5-40d1-9e75-c5c4b786b8c1","Type":"ContainerDied","Data":"ace6c2dd5ae93bbc8964f3c6f9000cd1ad304cf093d8a23dca8ea23fddaf4097"}
Sep 05 08:08:35.624139 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:35.624027    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ace6c2dd5ae93bbc8964f3c6f9000cd1ad304cf093d8a23dca8ea23fddaf4097"
Sep 05 08:08:46.873832 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:46.873795    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:08:46.904184 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:46.904151    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4b62745d-69fc6f6f5c-thkhg"]
Sep 05 08:08:47.122440 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:47.122398    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4b62745d-kl6tn"]
Sep 05 08:08:47.126874 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:47.126792    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4b62745d-kl6tn"]
Sep 05 08:08:47.699194 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:47.699150    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-4b62745d-69fc6f6f5c-thkhg" podUID="a5ae26f8-33c5-40d8-9eb7-0dd9c07bdc03" containerName="payload-processing" containerID="cri-o://781cbe7c60cc7e1f4b63c9806dc65359117305b092f19b648ab82c421c251c40" gracePeriod=30
Sep 05 08:08:47.730151 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:47.730118    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3949d1f-57a5-40d1-9e75-c5c4b786b8c1" path="/var/lib/kubelet/pods/f3949d1f-57a5-40d1-9e75-c5c4b786b8c1/volumes"
Sep 05 08:08:47.958105 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:47.958043    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4b62745d-69fc6f6f5c-thkhg"
Sep 05 08:08:48.036833 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:48.036805    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zd8bb\" (UniqueName: \"kubernetes.io/projected/a5ae26f8-33c5-40d8-9eb7-0dd9c07bdc03-kube-api-access-zd8bb\") pod \"a5ae26f8-33c5-40d8-9eb7-0dd9c07bdc03\" (UID: \"a5ae26f8-33c5-40d8-9eb7-0dd9c07bdc03\") "
Sep 05 08:08:48.036988 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:48.036964    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5ae26f8-33c5-40d8-9eb7-0dd9c07bdc03-plugins-config-volume\") pod \"a5ae26f8-33c5-40d8-9eb7-0dd9c07bdc03\" (UID: \"a5ae26f8-33c5-40d8-9eb7-0dd9c07bdc03\") "
Sep 05 08:08:48.037333 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:48.037301    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5ae26f8-33c5-40d8-9eb7-0dd9c07bdc03-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a5ae26f8-33c5-40d8-9eb7-0dd9c07bdc03" (UID: "a5ae26f8-33c5-40d8-9eb7-0dd9c07bdc03"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:08:48.039032 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:48.039003    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5ae26f8-33c5-40d8-9eb7-0dd9c07bdc03-kube-api-access-zd8bb" (OuterVolumeSpecName: "kube-api-access-zd8bb") pod "a5ae26f8-33c5-40d8-9eb7-0dd9c07bdc03" (UID: "a5ae26f8-33c5-40d8-9eb7-0dd9c07bdc03"). InnerVolumeSpecName "kube-api-access-zd8bb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:08:48.058488 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:48.058464    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:08:48.137591 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:48.137566    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zd8bb\" (UniqueName: \"kubernetes.io/projected/a5ae26f8-33c5-40d8-9eb7-0dd9c07bdc03-kube-api-access-zd8bb\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:08:48.137591 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:48.137589    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5ae26f8-33c5-40d8-9eb7-0dd9c07bdc03-plugins-config-volume\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:08:48.705307 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:48.705271    2563 generic.go:356] "Generic (PLEG): container finished" podID="a5ae26f8-33c5-40d8-9eb7-0dd9c07bdc03" containerID="781cbe7c60cc7e1f4b63c9806dc65359117305b092f19b648ab82c421c251c40" exitCode=0
Sep 05 08:08:48.705474 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:48.705313    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4b62745d-69fc6f6f5c-thkhg"
Sep 05 08:08:48.705474 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:48.705357    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4b62745d-69fc6f6f5c-thkhg" event={"ID":"a5ae26f8-33c5-40d8-9eb7-0dd9c07bdc03","Type":"ContainerDied","Data":"781cbe7c60cc7e1f4b63c9806dc65359117305b092f19b648ab82c421c251c40"}
Sep 05 08:08:48.705474 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:48.705408    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4b62745d-69fc6f6f5c-thkhg" event={"ID":"a5ae26f8-33c5-40d8-9eb7-0dd9c07bdc03","Type":"ContainerDied","Data":"43619d7853351b5b477adad93c101710d9070bcf6f437be0b4d53d573a64d89c"}
Sep 05 08:08:48.705474 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:48.705430    2563 scope.go:117] "RemoveContainer" containerID="781cbe7c60cc7e1f4b63c9806dc65359117305b092f19b648ab82c421c251c40"
Sep 05 08:08:48.721711 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:48.721694    2563 scope.go:117] "RemoveContainer" containerID="781cbe7c60cc7e1f4b63c9806dc65359117305b092f19b648ab82c421c251c40"
Sep 05 08:08:48.721983 ip-10-0-133-14 kubenswrapper[2563]: E0905 08:08:48.721965    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"781cbe7c60cc7e1f4b63c9806dc65359117305b092f19b648ab82c421c251c40\": container with ID starting with 781cbe7c60cc7e1f4b63c9806dc65359117305b092f19b648ab82c421c251c40 not found: ID does not exist" containerID="781cbe7c60cc7e1f4b63c9806dc65359117305b092f19b648ab82c421c251c40"
Sep 05 08:08:48.722074 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:48.721989    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"781cbe7c60cc7e1f4b63c9806dc65359117305b092f19b648ab82c421c251c40"} err="failed to get container status \"781cbe7c60cc7e1f4b63c9806dc65359117305b092f19b648ab82c421c251c40\": rpc error: code = NotFound desc = could not find container \"781cbe7c60cc7e1f4b63c9806dc65359117305b092f19b648ab82c421c251c40\": container with ID starting with 781cbe7c60cc7e1f4b63c9806dc65359117305b092f19b648ab82c421c251c40 not found: ID does not exist"
Sep 05 08:08:48.730702 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:48.730680    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4b62745d-69fc6f6f5c-thkhg"]
Sep 05 08:08:48.732725 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:48.732708    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4b62745d-69fc6f6f5c-thkhg"]
Sep 05 08:08:49.730323 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:49.730283    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5ae26f8-33c5-40d8-9eb7-0dd9c07bdc03" path="/var/lib/kubelet/pods/a5ae26f8-33c5-40d8-9eb7-0dd9c07bdc03/volumes"
Sep 05 08:08:50.646350 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:50.646308    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:08:52.447594 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:52.447552    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:08:55.628503 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:55.628467    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-4b62745d/e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s"]
Sep 05 08:08:55.676938 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:08:55.676900    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:09:10.649010 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:10.648900    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-4b62745d/e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s" podUID="d1cb7513-5db3-4857-8a1c-751c5df2158b" containerName="main" containerID="cri-o://fde1389f048c1b412b04eaf3d8ac31343d036df74ef0dffbb0106f0b25e995ed" gracePeriod=45
Sep 05 08:09:10.822517 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:10.822467    2563 generic.go:356] "Generic (PLEG): container finished" podID="d1cb7513-5db3-4857-8a1c-751c5df2158b" containerID="fde1389f048c1b412b04eaf3d8ac31343d036df74ef0dffbb0106f0b25e995ed" exitCode=0
Sep 05 08:09:10.822689 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:10.822549    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-4b62745d/e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s" event={"ID":"d1cb7513-5db3-4857-8a1c-751c5df2158b","Type":"ContainerDied","Data":"fde1389f048c1b412b04eaf3d8ac31343d036df74ef0dffbb0106f0b25e995ed"}
Sep 05 08:09:10.902427 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:10.902366    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-4b62745d/e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s"
Sep 05 08:09:10.959746 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:10.959715    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1cb7513-5db3-4857-8a1c-751c5df2158b-dshm\") pod \"d1cb7513-5db3-4857-8a1c-751c5df2158b\" (UID: \"d1cb7513-5db3-4857-8a1c-751c5df2158b\") "
Sep 05 08:09:10.959951 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:10.959782    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1cb7513-5db3-4857-8a1c-751c5df2158b-model-cache\") pod \"d1cb7513-5db3-4857-8a1c-751c5df2158b\" (UID: \"d1cb7513-5db3-4857-8a1c-751c5df2158b\") "
Sep 05 08:09:10.959951 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:10.959822    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1cb7513-5db3-4857-8a1c-751c5df2158b-tls-certs\") pod \"d1cb7513-5db3-4857-8a1c-751c5df2158b\" (UID: \"d1cb7513-5db3-4857-8a1c-751c5df2158b\") "
Sep 05 08:09:10.959951 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:10.959867    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1cb7513-5db3-4857-8a1c-751c5df2158b-home\") pod \"d1cb7513-5db3-4857-8a1c-751c5df2158b\" (UID: \"d1cb7513-5db3-4857-8a1c-751c5df2158b\") "
Sep 05 08:09:10.959951 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:10.959910    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wcrzx\" (UniqueName: \"kubernetes.io/projected/d1cb7513-5db3-4857-8a1c-751c5df2158b-kube-api-access-wcrzx\") pod \"d1cb7513-5db3-4857-8a1c-751c5df2158b\" (UID: \"d1cb7513-5db3-4857-8a1c-751c5df2158b\") "
Sep 05 08:09:10.960176 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:10.959979    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1cb7513-5db3-4857-8a1c-751c5df2158b-tmp-dir\") pod \"d1cb7513-5db3-4857-8a1c-751c5df2158b\" (UID: \"d1cb7513-5db3-4857-8a1c-751c5df2158b\") "
Sep 05 08:09:10.960176 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:10.960106    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1cb7513-5db3-4857-8a1c-751c5df2158b-model-cache" (OuterVolumeSpecName: "model-cache") pod "d1cb7513-5db3-4857-8a1c-751c5df2158b" (UID: "d1cb7513-5db3-4857-8a1c-751c5df2158b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:09:10.960176 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:10.960151    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1cb7513-5db3-4857-8a1c-751c5df2158b-home" (OuterVolumeSpecName: "home") pod "d1cb7513-5db3-4857-8a1c-751c5df2158b" (UID: "d1cb7513-5db3-4857-8a1c-751c5df2158b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:09:10.960359 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:10.960335    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1cb7513-5db3-4857-8a1c-751c5df2158b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d1cb7513-5db3-4857-8a1c-751c5df2158b" (UID: "d1cb7513-5db3-4857-8a1c-751c5df2158b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:09:10.960452 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:10.960434    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1cb7513-5db3-4857-8a1c-751c5df2158b-model-cache\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:09:10.960511 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:10.960458    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1cb7513-5db3-4857-8a1c-751c5df2158b-home\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:09:10.962566 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:10.962539    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1cb7513-5db3-4857-8a1c-751c5df2158b-kube-api-access-wcrzx" (OuterVolumeSpecName: "kube-api-access-wcrzx") pod "d1cb7513-5db3-4857-8a1c-751c5df2158b" (UID: "d1cb7513-5db3-4857-8a1c-751c5df2158b"). InnerVolumeSpecName "kube-api-access-wcrzx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:09:10.962739 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:10.962717    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1cb7513-5db3-4857-8a1c-751c5df2158b-dshm" (OuterVolumeSpecName: "dshm") pod "d1cb7513-5db3-4857-8a1c-751c5df2158b" (UID: "d1cb7513-5db3-4857-8a1c-751c5df2158b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:09:10.962739 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:10.962724    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1cb7513-5db3-4857-8a1c-751c5df2158b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d1cb7513-5db3-4857-8a1c-751c5df2158b" (UID: "d1cb7513-5db3-4857-8a1c-751c5df2158b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:09:11.061831 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:11.061797    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1cb7513-5db3-4857-8a1c-751c5df2158b-dshm\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:09:11.061831 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:11.061825    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1cb7513-5db3-4857-8a1c-751c5df2158b-tls-certs\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:09:11.061831 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:11.061835    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wcrzx\" (UniqueName: \"kubernetes.io/projected/d1cb7513-5db3-4857-8a1c-751c5df2158b-kube-api-access-wcrzx\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:09:11.062084 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:11.061869    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1cb7513-5db3-4857-8a1c-751c5df2158b-tmp-dir\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:09:11.828783 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:11.828743    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-4b62745d/e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s" event={"ID":"d1cb7513-5db3-4857-8a1c-751c5df2158b","Type":"ContainerDied","Data":"5aae96afa8c319f436c98143f94697f2657e80d906b71aab190b232e0e658747"}
Sep 05 08:09:11.829216 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:11.828797    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-4b62745d/e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s"
Sep 05 08:09:11.829216 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:11.828807    2563 scope.go:117] "RemoveContainer" containerID="fde1389f048c1b412b04eaf3d8ac31343d036df74ef0dffbb0106f0b25e995ed"
Sep 05 08:09:11.846395 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:11.846369    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-4b62745d/e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s"]
Sep 05 08:09:11.849827 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:11.849800    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-4b62745d/e2e-lifecycle-model-4b62745d-kserve-5c4bd9899-qwh5s"]
Sep 05 08:09:13.729790 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:13.729746    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1cb7513-5db3-4857-8a1c-751c5df2158b" path="/var/lib/kubelet/pods/d1cb7513-5db3-4857-8a1c-751c5df2158b/volumes"
Sep 05 08:09:17.843759 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:17.843726    2563 scope.go:117] "RemoveContainer" containerID="5d9db85608f713c0681bb29fdeb381ef244e821789552374198890e1b684d5d0"
Sep 05 08:09:17.852348 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:17.852330    2563 scope.go:117] "RemoveContainer" containerID="86556cd8b6737014c0b107dd15164bc684a04c6e0e8366fa43d95145f4413415"
Sep 05 08:09:17.861513 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:17.861442    2563 scope.go:117] "RemoveContainer" containerID="cc9875edd326bcb00e4f49bdb36c4a29200c0029e01402c9d35f7e711a2de3b3"
Sep 05 08:09:31.892255 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:31.892220    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:09:32.938075 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:32.938046    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:09:37.724479 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:37.724441    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:09:52.169279 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:52.169239    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:09:52.230465 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:52.230432    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:09:52.400653 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:52.400617    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"]
Sep 05 08:09:52.400928 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:52.400903    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5" podUID="262bcc62-34d3-4363-9189-6a8ab3d96838" containerName="istio-proxy" containerID="cri-o://a8b559f36542b6e927a722e2317f8ca2366a09d0d3774129b98719a6dad56a51" gracePeriod=30
Sep 05 08:09:52.420829 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:52.420751    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:09:53.726460 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:53.726427    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:09:53.834859 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:53.834816    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:09:53.927682 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:53.927650    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:09:54.537556 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:54.537523    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:09:54.637269 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:54.637235    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:09:54.740513 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:54.740483    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:09:55.958380 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:55.958340    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:09:57.651522 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:57.651492    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"
Sep 05 08:09:57.817769 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:57.817739    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/262bcc62-34d3-4363-9189-6a8ab3d96838-istio-podinfo\") pod \"262bcc62-34d3-4363-9189-6a8ab3d96838\" (UID: \"262bcc62-34d3-4363-9189-6a8ab3d96838\") "
Sep 05 08:09:57.817970 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:57.817792    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/262bcc62-34d3-4363-9189-6a8ab3d96838-credential-socket\") pod \"262bcc62-34d3-4363-9189-6a8ab3d96838\" (UID: \"262bcc62-34d3-4363-9189-6a8ab3d96838\") "
Sep 05 08:09:57.817970 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:57.817818    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/262bcc62-34d3-4363-9189-6a8ab3d96838-istio-data\") pod \"262bcc62-34d3-4363-9189-6a8ab3d96838\" (UID: \"262bcc62-34d3-4363-9189-6a8ab3d96838\") "
Sep 05 08:09:57.817970 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:57.817922    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/262bcc62-34d3-4363-9189-6a8ab3d96838-workload-socket\") pod \"262bcc62-34d3-4363-9189-6a8ab3d96838\" (UID: \"262bcc62-34d3-4363-9189-6a8ab3d96838\") "
Sep 05 08:09:57.818130 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:57.817972    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/262bcc62-34d3-4363-9189-6a8ab3d96838-istiod-ca-cert\") pod \"262bcc62-34d3-4363-9189-6a8ab3d96838\" (UID: \"262bcc62-34d3-4363-9189-6a8ab3d96838\") "
Sep 05 08:09:57.818130 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:57.818021    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/262bcc62-34d3-4363-9189-6a8ab3d96838-istio-token\") pod \"262bcc62-34d3-4363-9189-6a8ab3d96838\" (UID: \"262bcc62-34d3-4363-9189-6a8ab3d96838\") "
Sep 05 08:09:57.818130 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:57.818067    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/262bcc62-34d3-4363-9189-6a8ab3d96838-istio-envoy\") pod \"262bcc62-34d3-4363-9189-6a8ab3d96838\" (UID: \"262bcc62-34d3-4363-9189-6a8ab3d96838\") "
Sep 05 08:09:57.818130 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:57.818094    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/262bcc62-34d3-4363-9189-6a8ab3d96838-istio-data" (OuterVolumeSpecName: "istio-data") pod "262bcc62-34d3-4363-9189-6a8ab3d96838" (UID: "262bcc62-34d3-4363-9189-6a8ab3d96838"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:09:57.818325 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:57.818111    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/262bcc62-34d3-4363-9189-6a8ab3d96838-workload-certs\") pod \"262bcc62-34d3-4363-9189-6a8ab3d96838\" (UID: \"262bcc62-34d3-4363-9189-6a8ab3d96838\") "
Sep 05 08:09:57.818325 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:57.818137    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/262bcc62-34d3-4363-9189-6a8ab3d96838-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "262bcc62-34d3-4363-9189-6a8ab3d96838" (UID: "262bcc62-34d3-4363-9189-6a8ab3d96838"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:09:57.818325 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:57.818171    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k7s6b\" (UniqueName: \"kubernetes.io/projected/262bcc62-34d3-4363-9189-6a8ab3d96838-kube-api-access-k7s6b\") pod \"262bcc62-34d3-4363-9189-6a8ab3d96838\" (UID: \"262bcc62-34d3-4363-9189-6a8ab3d96838\") "
Sep 05 08:09:57.818492 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:57.818392    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/262bcc62-34d3-4363-9189-6a8ab3d96838-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "262bcc62-34d3-4363-9189-6a8ab3d96838" (UID: "262bcc62-34d3-4363-9189-6a8ab3d96838"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:09:57.818492 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:57.818415    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/262bcc62-34d3-4363-9189-6a8ab3d96838-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "262bcc62-34d3-4363-9189-6a8ab3d96838" (UID: "262bcc62-34d3-4363-9189-6a8ab3d96838"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:09:57.818492 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:57.818478    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/262bcc62-34d3-4363-9189-6a8ab3d96838-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "262bcc62-34d3-4363-9189-6a8ab3d96838" (UID: "262bcc62-34d3-4363-9189-6a8ab3d96838"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:09:57.818655 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:57.818593    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/262bcc62-34d3-4363-9189-6a8ab3d96838-workload-certs\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:09:57.818655 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:57.818614    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/262bcc62-34d3-4363-9189-6a8ab3d96838-credential-socket\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:09:57.818655 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:57.818636    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/262bcc62-34d3-4363-9189-6a8ab3d96838-istio-data\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:09:57.818655 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:57.818654    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/262bcc62-34d3-4363-9189-6a8ab3d96838-workload-socket\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:09:57.819059 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:57.818667    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/262bcc62-34d3-4363-9189-6a8ab3d96838-istiod-ca-cert\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:09:57.820443 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:57.820417    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/262bcc62-34d3-4363-9189-6a8ab3d96838-kube-api-access-k7s6b" (OuterVolumeSpecName: "kube-api-access-k7s6b") pod "262bcc62-34d3-4363-9189-6a8ab3d96838" (UID: "262bcc62-34d3-4363-9189-6a8ab3d96838"). InnerVolumeSpecName "kube-api-access-k7s6b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:09:57.820583 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:57.820442    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/262bcc62-34d3-4363-9189-6a8ab3d96838-istio-token" (OuterVolumeSpecName: "istio-token") pod "262bcc62-34d3-4363-9189-6a8ab3d96838" (UID: "262bcc62-34d3-4363-9189-6a8ab3d96838"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:09:57.820675 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:57.820655    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/262bcc62-34d3-4363-9189-6a8ab3d96838-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "262bcc62-34d3-4363-9189-6a8ab3d96838" (UID: "262bcc62-34d3-4363-9189-6a8ab3d96838"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:09:57.820716 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:57.820671    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/262bcc62-34d3-4363-9189-6a8ab3d96838-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "262bcc62-34d3-4363-9189-6a8ab3d96838" (UID: "262bcc62-34d3-4363-9189-6a8ab3d96838"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 05 08:09:57.919462 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:57.919433    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/262bcc62-34d3-4363-9189-6a8ab3d96838-istio-token\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:09:57.919462 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:57.919457    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/262bcc62-34d3-4363-9189-6a8ab3d96838-istio-envoy\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:09:57.919462 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:57.919467    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7s6b\" (UniqueName: \"kubernetes.io/projected/262bcc62-34d3-4363-9189-6a8ab3d96838-kube-api-access-k7s6b\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:09:57.919700 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:57.919477    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/262bcc62-34d3-4363-9189-6a8ab3d96838-istio-podinfo\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:09:58.077641 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:58.077557    2563 generic.go:356] "Generic (PLEG): container finished" podID="262bcc62-34d3-4363-9189-6a8ab3d96838" containerID="a8b559f36542b6e927a722e2317f8ca2366a09d0d3774129b98719a6dad56a51" exitCode=0
Sep 05 08:09:58.077641 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:58.077607    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"
Sep 05 08:09:58.077641 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:58.077623    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5" event={"ID":"262bcc62-34d3-4363-9189-6a8ab3d96838","Type":"ContainerDied","Data":"a8b559f36542b6e927a722e2317f8ca2366a09d0d3774129b98719a6dad56a51"}
Sep 05 08:09:58.077929 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:58.077666    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5" event={"ID":"262bcc62-34d3-4363-9189-6a8ab3d96838","Type":"ContainerDied","Data":"eb3298e73bcc82d37dc7e0250b1bebc8edd4df7c9caa2befa6b80997dc001aa1"}
Sep 05 08:09:58.077929 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:58.077688    2563 scope.go:117] "RemoveContainer" containerID="a8b559f36542b6e927a722e2317f8ca2366a09d0d3774129b98719a6dad56a51"
Sep 05 08:09:58.087251 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:58.087231    2563 scope.go:117] "RemoveContainer" containerID="8ec9078a44fd31009410e261853266ec6c2c8d65864274ee31ead39fbc327db5"
Sep 05 08:09:58.095199 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:58.095183    2563 scope.go:117] "RemoveContainer" containerID="a8b559f36542b6e927a722e2317f8ca2366a09d0d3774129b98719a6dad56a51"
Sep 05 08:09:58.095440 ip-10-0-133-14 kubenswrapper[2563]: E0905 08:09:58.095421    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8b559f36542b6e927a722e2317f8ca2366a09d0d3774129b98719a6dad56a51\": container with ID starting with a8b559f36542b6e927a722e2317f8ca2366a09d0d3774129b98719a6dad56a51 not found: ID does not exist" containerID="a8b559f36542b6e927a722e2317f8ca2366a09d0d3774129b98719a6dad56a51"
Sep 05 08:09:58.095488 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:58.095449    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8b559f36542b6e927a722e2317f8ca2366a09d0d3774129b98719a6dad56a51"} err="failed to get container status \"a8b559f36542b6e927a722e2317f8ca2366a09d0d3774129b98719a6dad56a51\": rpc error: code = NotFound desc = could not find container \"a8b559f36542b6e927a722e2317f8ca2366a09d0d3774129b98719a6dad56a51\": container with ID starting with a8b559f36542b6e927a722e2317f8ca2366a09d0d3774129b98719a6dad56a51 not found: ID does not exist"
Sep 05 08:09:58.095488 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:58.095466    2563 scope.go:117] "RemoveContainer" containerID="8ec9078a44fd31009410e261853266ec6c2c8d65864274ee31ead39fbc327db5"
Sep 05 08:09:58.095697 ip-10-0-133-14 kubenswrapper[2563]: E0905 08:09:58.095681    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ec9078a44fd31009410e261853266ec6c2c8d65864274ee31ead39fbc327db5\": container with ID starting with 8ec9078a44fd31009410e261853266ec6c2c8d65864274ee31ead39fbc327db5 not found: ID does not exist" containerID="8ec9078a44fd31009410e261853266ec6c2c8d65864274ee31ead39fbc327db5"
Sep 05 08:09:58.095752 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:58.095701    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ec9078a44fd31009410e261853266ec6c2c8d65864274ee31ead39fbc327db5"} err="failed to get container status \"8ec9078a44fd31009410e261853266ec6c2c8d65864274ee31ead39fbc327db5\": rpc error: code = NotFound desc = could not find container \"8ec9078a44fd31009410e261853266ec6c2c8d65864274ee31ead39fbc327db5\": container with ID starting with 8ec9078a44fd31009410e261853266ec6c2c8d65864274ee31ead39fbc327db5 not found: ID does not exist"
Sep 05 08:09:58.099669 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:58.099647    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"]
Sep 05 08:09:58.106316 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:58.106295    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-4b62745d-openshift-default-d99f9d5b4-w6hh5"]
Sep 05 08:09:59.730942 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:09:59.730906    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="262bcc62-34d3-4363-9189-6a8ab3d96838" path="/var/lib/kubelet/pods/262bcc62-34d3-4363-9189-6a8ab3d96838/volumes"
Sep 05 08:10:00.307107 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:00.307072    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-7850-2b2b89"]
Sep 05 08:10:00.307447 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:00.307435    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1cb7513-5db3-4857-8a1c-751c5df2158b" containerName="main"
Sep 05 08:10:00.307493 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:00.307450    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1cb7513-5db3-4857-8a1c-751c5df2158b" containerName="main"
Sep 05 08:10:00.307493 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:00.307464    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="262bcc62-34d3-4363-9189-6a8ab3d96838" containerName="istio-proxy"
Sep 05 08:10:00.307493 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:00.307471    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="262bcc62-34d3-4363-9189-6a8ab3d96838" containerName="istio-proxy"
Sep 05 08:10:00.307493 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:00.307478    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3949d1f-57a5-40d1-9e75-c5c4b786b8c1" containerName="revoke-keys"
Sep 05 08:10:00.307493 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:00.307484    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3949d1f-57a5-40d1-9e75-c5c4b786b8c1" containerName="revoke-keys"
Sep 05 08:10:00.307651 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:00.307494    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5ae26f8-33c5-40d8-9eb7-0dd9c07bdc03" containerName="payload-processing"
Sep 05 08:10:00.307651 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:00.307504    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5ae26f8-33c5-40d8-9eb7-0dd9c07bdc03" containerName="payload-processing"
Sep 05 08:10:00.307651 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:00.307512    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="262bcc62-34d3-4363-9189-6a8ab3d96838" containerName="istio-proxy"
Sep 05 08:10:00.307651 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:00.307518    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="262bcc62-34d3-4363-9189-6a8ab3d96838" containerName="istio-proxy"
Sep 05 08:10:00.307651 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:00.307588    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1cb7513-5db3-4857-8a1c-751c5df2158b" containerName="main"
Sep 05 08:10:00.307651 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:00.307596    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5ae26f8-33c5-40d8-9eb7-0dd9c07bdc03" containerName="payload-processing"
Sep 05 08:10:00.307651 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:00.307604    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="262bcc62-34d3-4363-9189-6a8ab3d96838" containerName="istio-proxy"
Sep 05 08:10:00.307651 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:00.307611    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="262bcc62-34d3-4363-9189-6a8ab3d96838" containerName="istio-proxy"
Sep 05 08:10:00.307651 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:00.307616    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3949d1f-57a5-40d1-9e75-c5c4b786b8c1" containerName="revoke-keys"
Sep 05 08:10:00.312045 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:00.312029    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7850-2b2b89"
Sep 05 08:10:00.314592 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:00.314560    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-x4x7f\""
Sep 05 08:10:00.316246 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:00.316221    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7850-2b2b89"]
Sep 05 08:10:00.440930 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:00.440895    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lp2pw\" (UniqueName: \"kubernetes.io/projected/c1e0f34a-3ae1-424f-83ed-2d955390a6c8-kube-api-access-lp2pw\") pod \"mt-curl-7850-2b2b89\" (UID: \"c1e0f34a-3ae1-424f-83ed-2d955390a6c8\") " pod="opendatahub/mt-curl-7850-2b2b89"
Sep 05 08:10:00.541835 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:00.541798    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lp2pw\" (UniqueName: \"kubernetes.io/projected/c1e0f34a-3ae1-424f-83ed-2d955390a6c8-kube-api-access-lp2pw\") pod \"mt-curl-7850-2b2b89\" (UID: \"c1e0f34a-3ae1-424f-83ed-2d955390a6c8\") " pod="opendatahub/mt-curl-7850-2b2b89"
Sep 05 08:10:00.549352 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:00.549331    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lp2pw\" (UniqueName: \"kubernetes.io/projected/c1e0f34a-3ae1-424f-83ed-2d955390a6c8-kube-api-access-lp2pw\") pod \"mt-curl-7850-2b2b89\" (UID: \"c1e0f34a-3ae1-424f-83ed-2d955390a6c8\") " pod="opendatahub/mt-curl-7850-2b2b89"
Sep 05 08:10:00.623683 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:00.623603    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7850-2b2b89"
Sep 05 08:10:00.892077 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:00.892002    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7850-2b2b89"]
Sep 05 08:10:00.895248 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:10:00.895210    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc1e0f34a_3ae1_424f_83ed_2d955390a6c8.slice/crio-8e0b0f4a0448981d55a593823858ed95c984871288e412e887612c0de2e1cd23 WatchSource:0}: Error finding container 8e0b0f4a0448981d55a593823858ed95c984871288e412e887612c0de2e1cd23: Status 404 returned error can't find the container with id 8e0b0f4a0448981d55a593823858ed95c984871288e412e887612c0de2e1cd23
Sep 05 08:10:01.097271 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:01.097231    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7850-2b2b89" event={"ID":"c1e0f34a-3ae1-424f-83ed-2d955390a6c8","Type":"ContainerStarted","Data":"8e0b0f4a0448981d55a593823858ed95c984871288e412e887612c0de2e1cd23"}
Sep 05 08:10:01.921894 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:01.921770    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:10:02.022088 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:02.022055    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:10:02.103759 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:02.103722    2563 generic.go:356] "Generic (PLEG): container finished" podID="c1e0f34a-3ae1-424f-83ed-2d955390a6c8" containerID="5b182561d8723c72e30ab23011cb4dc2fd92a39cbf33c614caf2aaf969e45a1c" exitCode=0
Sep 05 08:10:02.103944 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:02.103777    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7850-2b2b89" event={"ID":"c1e0f34a-3ae1-424f-83ed-2d955390a6c8","Type":"ContainerDied","Data":"5b182561d8723c72e30ab23011cb4dc2fd92a39cbf33c614caf2aaf969e45a1c"}
Sep 05 08:10:02.601785 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:02.601737    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:10:03.239442 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:03.239412    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7850-2b2b89"
Sep 05 08:10:03.262649 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:03.262616    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_mt-curl-7850-2b2b89_c1e0f34a-3ae1-424f-83ed-2d955390a6c8/mt-curl-7850-2b2b89/0.log"
Sep 05 08:10:03.285724 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:03.285693    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-7850-2b2b89"]
Sep 05 08:10:03.288623 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:03.288601    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-7850-2b2b89"]
Sep 05 08:10:03.370429 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:03.370361    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lp2pw\" (UniqueName: \"kubernetes.io/projected/c1e0f34a-3ae1-424f-83ed-2d955390a6c8-kube-api-access-lp2pw\") pod \"c1e0f34a-3ae1-424f-83ed-2d955390a6c8\" (UID: \"c1e0f34a-3ae1-424f-83ed-2d955390a6c8\") "
Sep 05 08:10:03.372509 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:03.372480    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1e0f34a-3ae1-424f-83ed-2d955390a6c8-kube-api-access-lp2pw" (OuterVolumeSpecName: "kube-api-access-lp2pw") pod "c1e0f34a-3ae1-424f-83ed-2d955390a6c8" (UID: "c1e0f34a-3ae1-424f-83ed-2d955390a6c8"). InnerVolumeSpecName "kube-api-access-lp2pw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:10:03.471879 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:03.471832    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lp2pw\" (UniqueName: \"kubernetes.io/projected/c1e0f34a-3ae1-424f-83ed-2d955390a6c8-kube-api-access-lp2pw\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:10:03.738455 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:03.738368    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1e0f34a-3ae1-424f-83ed-2d955390a6c8" path="/var/lib/kubelet/pods/c1e0f34a-3ae1-424f-83ed-2d955390a6c8/volumes"
Sep 05 08:10:04.114759 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:04.114725    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7850-2b2b89"
Sep 05 08:10:04.114992 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:04.114736    2563 scope.go:117] "RemoveContainer" containerID="5b182561d8723c72e30ab23011cb4dc2fd92a39cbf33c614caf2aaf969e45a1c"
Sep 05 08:10:04.689420 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:04.689391    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-8c2e20/auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl"]
Sep 05 08:10:04.713522 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:04.713491    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:10:05.600671 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:05.600640    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-5ef325/auth-test-model-5ef325-kserve-845c94757f-hjr9t"]
Sep 05 08:10:05.617518 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:05.617490    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:10:08.871734 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:08.871700    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:10:08.975436 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:08.975402    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:10:09.072057 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:09.072028    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:10:19.598641 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:19.598609    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:10:19.709756 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:19.709697    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-8c2e20/auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl" podUID="16510468-4894-4bc4-b7cb-884a86f17437" containerName="main" containerID="cri-o://f689d72a14cad082515cc3b92fff671a916b8cf538dbd3a0ca1a0000f8a3a41e" gracePeriod=45
Sep 05 08:10:19.836564 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:19.836533    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:10:19.968414 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:19.968381    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-8c2e20/auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl"
Sep 05 08:10:20.018131 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.018100    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16510468-4894-4bc4-b7cb-884a86f17437-dshm\") pod \"16510468-4894-4bc4-b7cb-884a86f17437\" (UID: \"16510468-4894-4bc4-b7cb-884a86f17437\") "
Sep 05 08:10:20.018302 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.018175    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16510468-4894-4bc4-b7cb-884a86f17437-model-cache\") pod \"16510468-4894-4bc4-b7cb-884a86f17437\" (UID: \"16510468-4894-4bc4-b7cb-884a86f17437\") "
Sep 05 08:10:20.018302 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.018195    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ll8s2\" (UniqueName: \"kubernetes.io/projected/16510468-4894-4bc4-b7cb-884a86f17437-kube-api-access-ll8s2\") pod \"16510468-4894-4bc4-b7cb-884a86f17437\" (UID: \"16510468-4894-4bc4-b7cb-884a86f17437\") "
Sep 05 08:10:20.018302 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.018237    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16510468-4894-4bc4-b7cb-884a86f17437-tmp-dir\") pod \"16510468-4894-4bc4-b7cb-884a86f17437\" (UID: \"16510468-4894-4bc4-b7cb-884a86f17437\") "
Sep 05 08:10:20.018302 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.018296    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16510468-4894-4bc4-b7cb-884a86f17437-home\") pod \"16510468-4894-4bc4-b7cb-884a86f17437\" (UID: \"16510468-4894-4bc4-b7cb-884a86f17437\") "
Sep 05 08:10:20.018540 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.018317    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16510468-4894-4bc4-b7cb-884a86f17437-tls-certs\") pod \"16510468-4894-4bc4-b7cb-884a86f17437\" (UID: \"16510468-4894-4bc4-b7cb-884a86f17437\") "
Sep 05 08:10:20.018540 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.018465    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16510468-4894-4bc4-b7cb-884a86f17437-model-cache" (OuterVolumeSpecName: "model-cache") pod "16510468-4894-4bc4-b7cb-884a86f17437" (UID: "16510468-4894-4bc4-b7cb-884a86f17437"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:10:20.018700 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.018546    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16510468-4894-4bc4-b7cb-884a86f17437-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "16510468-4894-4bc4-b7cb-884a86f17437" (UID: "16510468-4894-4bc4-b7cb-884a86f17437"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:10:20.018700 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.018564    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16510468-4894-4bc4-b7cb-884a86f17437-home" (OuterVolumeSpecName: "home") pod "16510468-4894-4bc4-b7cb-884a86f17437" (UID: "16510468-4894-4bc4-b7cb-884a86f17437"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:10:20.018700 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.018640    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16510468-4894-4bc4-b7cb-884a86f17437-model-cache\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:10:20.018700 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.018655    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16510468-4894-4bc4-b7cb-884a86f17437-tmp-dir\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:10:20.020657 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.020633    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16510468-4894-4bc4-b7cb-884a86f17437-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "16510468-4894-4bc4-b7cb-884a86f17437" (UID: "16510468-4894-4bc4-b7cb-884a86f17437"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:10:20.020937 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.020921    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16510468-4894-4bc4-b7cb-884a86f17437-dshm" (OuterVolumeSpecName: "dshm") pod "16510468-4894-4bc4-b7cb-884a86f17437" (UID: "16510468-4894-4bc4-b7cb-884a86f17437"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:10:20.021007 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.020967    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16510468-4894-4bc4-b7cb-884a86f17437-kube-api-access-ll8s2" (OuterVolumeSpecName: "kube-api-access-ll8s2") pod "16510468-4894-4bc4-b7cb-884a86f17437" (UID: "16510468-4894-4bc4-b7cb-884a86f17437"). InnerVolumeSpecName "kube-api-access-ll8s2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:10:20.120081 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.120047    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ll8s2\" (UniqueName: \"kubernetes.io/projected/16510468-4894-4bc4-b7cb-884a86f17437-kube-api-access-ll8s2\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:10:20.120081 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.120077    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16510468-4894-4bc4-b7cb-884a86f17437-home\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:10:20.120081 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.120086    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16510468-4894-4bc4-b7cb-884a86f17437-tls-certs\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:10:20.120319 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.120095    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16510468-4894-4bc4-b7cb-884a86f17437-dshm\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:10:20.197284 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.197193    2563 generic.go:356] "Generic (PLEG): container finished" podID="16510468-4894-4bc4-b7cb-884a86f17437" containerID="f689d72a14cad082515cc3b92fff671a916b8cf538dbd3a0ca1a0000f8a3a41e" exitCode=0
Sep 05 08:10:20.197284 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.197240    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-8c2e20/auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl" event={"ID":"16510468-4894-4bc4-b7cb-884a86f17437","Type":"ContainerDied","Data":"f689d72a14cad082515cc3b92fff671a916b8cf538dbd3a0ca1a0000f8a3a41e"}
Sep 05 08:10:20.197284 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.197254    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-8c2e20/auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl"
Sep 05 08:10:20.197284 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.197272    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-8c2e20/auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl" event={"ID":"16510468-4894-4bc4-b7cb-884a86f17437","Type":"ContainerDied","Data":"fc6a9b3b69b80efd4780c8c818cce5d31a93f82cb2ac4d17b3cbb010548dc330"}
Sep 05 08:10:20.197284 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.197290    2563 scope.go:117] "RemoveContainer" containerID="f689d72a14cad082515cc3b92fff671a916b8cf538dbd3a0ca1a0000f8a3a41e"
Sep 05 08:10:20.206656 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.206639    2563 scope.go:117] "RemoveContainer" containerID="f689d72a14cad082515cc3b92fff671a916b8cf538dbd3a0ca1a0000f8a3a41e"
Sep 05 08:10:20.206976 ip-10-0-133-14 kubenswrapper[2563]: E0905 08:10:20.206957    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f689d72a14cad082515cc3b92fff671a916b8cf538dbd3a0ca1a0000f8a3a41e\": container with ID starting with f689d72a14cad082515cc3b92fff671a916b8cf538dbd3a0ca1a0000f8a3a41e not found: ID does not exist" containerID="f689d72a14cad082515cc3b92fff671a916b8cf538dbd3a0ca1a0000f8a3a41e"
Sep 05 08:10:20.207039 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.206984    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f689d72a14cad082515cc3b92fff671a916b8cf538dbd3a0ca1a0000f8a3a41e"} err="failed to get container status \"f689d72a14cad082515cc3b92fff671a916b8cf538dbd3a0ca1a0000f8a3a41e\": rpc error: code = NotFound desc = could not find container \"f689d72a14cad082515cc3b92fff671a916b8cf538dbd3a0ca1a0000f8a3a41e\": container with ID starting with f689d72a14cad082515cc3b92fff671a916b8cf538dbd3a0ca1a0000f8a3a41e not found: ID does not exist"
Sep 05 08:10:20.217566 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.217542    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-8c2e20/auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl"]
Sep 05 08:10:20.221475 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.221456    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-8c2e20/auth-test-model-8c2e20-kserve-5b6b69b9f5-pg5tl"]
Sep 05 08:10:20.621434 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.621374    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-5ef325/auth-test-model-5ef325-kserve-845c94757f-hjr9t" podUID="429b258a-caaf-4cef-aa49-fb975d367945" containerName="main" containerID="cri-o://37a0125a3c2dfcbd291a766e7ab7e13041b2289389ed4986b0a10c424abd2041" gracePeriod=45
Sep 05 08:10:20.878350 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.878290    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-5ef325/auth-test-model-5ef325-kserve-845c94757f-hjr9t"
Sep 05 08:10:20.929007 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.928970    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/429b258a-caaf-4cef-aa49-fb975d367945-home\") pod \"429b258a-caaf-4cef-aa49-fb975d367945\" (UID: \"429b258a-caaf-4cef-aa49-fb975d367945\") "
Sep 05 08:10:20.929200 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.929038    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/429b258a-caaf-4cef-aa49-fb975d367945-dshm\") pod \"429b258a-caaf-4cef-aa49-fb975d367945\" (UID: \"429b258a-caaf-4cef-aa49-fb975d367945\") "
Sep 05 08:10:20.929200 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.929095    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/429b258a-caaf-4cef-aa49-fb975d367945-tmp-dir\") pod \"429b258a-caaf-4cef-aa49-fb975d367945\" (UID: \"429b258a-caaf-4cef-aa49-fb975d367945\") "
Sep 05 08:10:20.929200 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.929131    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/429b258a-caaf-4cef-aa49-fb975d367945-tls-certs\") pod \"429b258a-caaf-4cef-aa49-fb975d367945\" (UID: \"429b258a-caaf-4cef-aa49-fb975d367945\") "
Sep 05 08:10:20.929200 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.929192    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t5zn8\" (UniqueName: \"kubernetes.io/projected/429b258a-caaf-4cef-aa49-fb975d367945-kube-api-access-t5zn8\") pod \"429b258a-caaf-4cef-aa49-fb975d367945\" (UID: \"429b258a-caaf-4cef-aa49-fb975d367945\") "
Sep 05 08:10:20.929426 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.929283    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/429b258a-caaf-4cef-aa49-fb975d367945-home" (OuterVolumeSpecName: "home") pod "429b258a-caaf-4cef-aa49-fb975d367945" (UID: "429b258a-caaf-4cef-aa49-fb975d367945"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:10:20.929426 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.929316    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/429b258a-caaf-4cef-aa49-fb975d367945-model-cache\") pod \"429b258a-caaf-4cef-aa49-fb975d367945\" (UID: \"429b258a-caaf-4cef-aa49-fb975d367945\") "
Sep 05 08:10:20.929426 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.929410    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/429b258a-caaf-4cef-aa49-fb975d367945-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "429b258a-caaf-4cef-aa49-fb975d367945" (UID: "429b258a-caaf-4cef-aa49-fb975d367945"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:10:20.929659 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.929633    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/429b258a-caaf-4cef-aa49-fb975d367945-model-cache" (OuterVolumeSpecName: "model-cache") pod "429b258a-caaf-4cef-aa49-fb975d367945" (UID: "429b258a-caaf-4cef-aa49-fb975d367945"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:10:20.929728 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.929709    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/429b258a-caaf-4cef-aa49-fb975d367945-home\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:10:20.929786 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.929735    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/429b258a-caaf-4cef-aa49-fb975d367945-tmp-dir\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:10:20.931516 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.931490    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/429b258a-caaf-4cef-aa49-fb975d367945-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "429b258a-caaf-4cef-aa49-fb975d367945" (UID: "429b258a-caaf-4cef-aa49-fb975d367945"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:10:20.931516 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.931494    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/429b258a-caaf-4cef-aa49-fb975d367945-dshm" (OuterVolumeSpecName: "dshm") pod "429b258a-caaf-4cef-aa49-fb975d367945" (UID: "429b258a-caaf-4cef-aa49-fb975d367945"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:10:20.931697 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:20.931610    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/429b258a-caaf-4cef-aa49-fb975d367945-kube-api-access-t5zn8" (OuterVolumeSpecName: "kube-api-access-t5zn8") pod "429b258a-caaf-4cef-aa49-fb975d367945" (UID: "429b258a-caaf-4cef-aa49-fb975d367945"). InnerVolumeSpecName "kube-api-access-t5zn8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:10:21.030712 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.030679    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t5zn8\" (UniqueName: \"kubernetes.io/projected/429b258a-caaf-4cef-aa49-fb975d367945-kube-api-access-t5zn8\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:10:21.030712 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.030706    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/429b258a-caaf-4cef-aa49-fb975d367945-model-cache\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:10:21.030712 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.030715    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/429b258a-caaf-4cef-aa49-fb975d367945-dshm\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:10:21.030966 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.030723    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/429b258a-caaf-4cef-aa49-fb975d367945-tls-certs\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:10:21.204177 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.204086    2563 generic.go:356] "Generic (PLEG): container finished" podID="429b258a-caaf-4cef-aa49-fb975d367945" containerID="37a0125a3c2dfcbd291a766e7ab7e13041b2289389ed4986b0a10c424abd2041" exitCode=0
Sep 05 08:10:21.204177 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.204134    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-5ef325/auth-test-model-5ef325-kserve-845c94757f-hjr9t"
Sep 05 08:10:21.204177 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.204150    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-5ef325/auth-test-model-5ef325-kserve-845c94757f-hjr9t" event={"ID":"429b258a-caaf-4cef-aa49-fb975d367945","Type":"ContainerDied","Data":"37a0125a3c2dfcbd291a766e7ab7e13041b2289389ed4986b0a10c424abd2041"}
Sep 05 08:10:21.204177 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.204180    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-5ef325/auth-test-model-5ef325-kserve-845c94757f-hjr9t" event={"ID":"429b258a-caaf-4cef-aa49-fb975d367945","Type":"ContainerDied","Data":"8fa903a800cf7d1cc7532dd19fe3f211a43a3515452594f2be81a570ef3ca767"}
Sep 05 08:10:21.204512 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.204197    2563 scope.go:117] "RemoveContainer" containerID="37a0125a3c2dfcbd291a766e7ab7e13041b2289389ed4986b0a10c424abd2041"
Sep 05 08:10:21.213449 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.213430    2563 scope.go:117] "RemoveContainer" containerID="37a0125a3c2dfcbd291a766e7ab7e13041b2289389ed4986b0a10c424abd2041"
Sep 05 08:10:21.213727 ip-10-0-133-14 kubenswrapper[2563]: E0905 08:10:21.213708    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37a0125a3c2dfcbd291a766e7ab7e13041b2289389ed4986b0a10c424abd2041\": container with ID starting with 37a0125a3c2dfcbd291a766e7ab7e13041b2289389ed4986b0a10c424abd2041 not found: ID does not exist" containerID="37a0125a3c2dfcbd291a766e7ab7e13041b2289389ed4986b0a10c424abd2041"
Sep 05 08:10:21.213772 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.213735    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37a0125a3c2dfcbd291a766e7ab7e13041b2289389ed4986b0a10c424abd2041"} err="failed to get container status \"37a0125a3c2dfcbd291a766e7ab7e13041b2289389ed4986b0a10c424abd2041\": rpc error: code = NotFound desc = could not find container \"37a0125a3c2dfcbd291a766e7ab7e13041b2289389ed4986b0a10c424abd2041\": container with ID starting with 37a0125a3c2dfcbd291a766e7ab7e13041b2289389ed4986b0a10c424abd2041 not found: ID does not exist"
Sep 05 08:10:21.231120 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.231089    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-5ef325/auth-test-model-5ef325-kserve-845c94757f-hjr9t"]
Sep 05 08:10:21.232689 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.232667    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-5ef325/auth-test-model-5ef325-kserve-845c94757f-hjr9t"]
Sep 05 08:10:21.497623 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.497540    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-8c2e20/sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg"]
Sep 05 08:10:21.497967 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.497953    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1e0f34a-3ae1-424f-83ed-2d955390a6c8" containerName="mt-curl-7850-2b2b89"
Sep 05 08:10:21.498016 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.497970    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1e0f34a-3ae1-424f-83ed-2d955390a6c8" containerName="mt-curl-7850-2b2b89"
Sep 05 08:10:21.498016 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.497988    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="429b258a-caaf-4cef-aa49-fb975d367945" containerName="main"
Sep 05 08:10:21.498016 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.497994    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="429b258a-caaf-4cef-aa49-fb975d367945" containerName="main"
Sep 05 08:10:21.498016 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.498001    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16510468-4894-4bc4-b7cb-884a86f17437" containerName="main"
Sep 05 08:10:21.498016 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.498006    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="16510468-4894-4bc4-b7cb-884a86f17437" containerName="main"
Sep 05 08:10:21.498166 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.498057    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1e0f34a-3ae1-424f-83ed-2d955390a6c8" containerName="mt-curl-7850-2b2b89"
Sep 05 08:10:21.498166 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.498068    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="429b258a-caaf-4cef-aa49-fb975d367945" containerName="main"
Sep 05 08:10:21.498166 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.498078    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="16510468-4894-4bc4-b7cb-884a86f17437" containerName="main"
Sep 05 08:10:21.502235 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.502204    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-8c2e20/sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg"
Sep 05 08:10:21.504618 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.504593    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-8c2e20\"/\"kube-root-ca.crt\""
Sep 05 08:10:21.505267 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.505250    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-8c2e20\"/\"default-dockercfg-ddq4m\""
Sep 05 08:10:21.505395 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.505290    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-8c2e20\"/\"openshift-service-ca.crt\""
Sep 05 08:10:21.505395 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.505282    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-8c2e20\"/\"sub-test-model-8c2e20-kserve-self-signed-certs\""
Sep 05 08:10:21.513181 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.513159    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-8c2e20/sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg"]
Sep 05 08:10:21.522828 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.522796    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:10:21.536220 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.536196    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23ff45e4-cfba-4b8f-a52f-3714dc19eed2-tmp-dir\") pod \"sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg\" (UID: \"23ff45e4-cfba-4b8f-a52f-3714dc19eed2\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg"
Sep 05 08:10:21.536331 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.536226    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d72jv\" (UniqueName: \"kubernetes.io/projected/23ff45e4-cfba-4b8f-a52f-3714dc19eed2-kube-api-access-d72jv\") pod \"sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg\" (UID: \"23ff45e4-cfba-4b8f-a52f-3714dc19eed2\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg"
Sep 05 08:10:21.536331 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.536251    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23ff45e4-cfba-4b8f-a52f-3714dc19eed2-home\") pod \"sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg\" (UID: \"23ff45e4-cfba-4b8f-a52f-3714dc19eed2\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg"
Sep 05 08:10:21.536331 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.536316    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23ff45e4-cfba-4b8f-a52f-3714dc19eed2-tls-certs\") pod \"sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg\" (UID: \"23ff45e4-cfba-4b8f-a52f-3714dc19eed2\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg"
Sep 05 08:10:21.536446 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.536362    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23ff45e4-cfba-4b8f-a52f-3714dc19eed2-model-cache\") pod \"sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg\" (UID: \"23ff45e4-cfba-4b8f-a52f-3714dc19eed2\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg"
Sep 05 08:10:21.536519 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.536496    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23ff45e4-cfba-4b8f-a52f-3714dc19eed2-dshm\") pod \"sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg\" (UID: \"23ff45e4-cfba-4b8f-a52f-3714dc19eed2\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg"
Sep 05 08:10:21.637135 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.637102    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23ff45e4-cfba-4b8f-a52f-3714dc19eed2-dshm\") pod \"sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg\" (UID: \"23ff45e4-cfba-4b8f-a52f-3714dc19eed2\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg"
Sep 05 08:10:21.637544 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.637145    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23ff45e4-cfba-4b8f-a52f-3714dc19eed2-tmp-dir\") pod \"sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg\" (UID: \"23ff45e4-cfba-4b8f-a52f-3714dc19eed2\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg"
Sep 05 08:10:21.637544 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.637161    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d72jv\" (UniqueName: \"kubernetes.io/projected/23ff45e4-cfba-4b8f-a52f-3714dc19eed2-kube-api-access-d72jv\") pod \"sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg\" (UID: \"23ff45e4-cfba-4b8f-a52f-3714dc19eed2\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg"
Sep 05 08:10:21.637544 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.637184    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23ff45e4-cfba-4b8f-a52f-3714dc19eed2-home\") pod \"sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg\" (UID: \"23ff45e4-cfba-4b8f-a52f-3714dc19eed2\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg"
Sep 05 08:10:21.637544 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.637220    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23ff45e4-cfba-4b8f-a52f-3714dc19eed2-tls-certs\") pod \"sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg\" (UID: \"23ff45e4-cfba-4b8f-a52f-3714dc19eed2\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg"
Sep 05 08:10:21.637544 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.637267    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23ff45e4-cfba-4b8f-a52f-3714dc19eed2-model-cache\") pod \"sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg\" (UID: \"23ff45e4-cfba-4b8f-a52f-3714dc19eed2\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg"
Sep 05 08:10:21.637794 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.637568    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23ff45e4-cfba-4b8f-a52f-3714dc19eed2-tmp-dir\") pod \"sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg\" (UID: \"23ff45e4-cfba-4b8f-a52f-3714dc19eed2\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg"
Sep 05 08:10:21.637794 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.637630    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23ff45e4-cfba-4b8f-a52f-3714dc19eed2-model-cache\") pod \"sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg\" (UID: \"23ff45e4-cfba-4b8f-a52f-3714dc19eed2\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg"
Sep 05 08:10:21.637794 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.637651    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23ff45e4-cfba-4b8f-a52f-3714dc19eed2-home\") pod \"sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg\" (UID: \"23ff45e4-cfba-4b8f-a52f-3714dc19eed2\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg"
Sep 05 08:10:21.639381 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.639363    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23ff45e4-cfba-4b8f-a52f-3714dc19eed2-dshm\") pod \"sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg\" (UID: \"23ff45e4-cfba-4b8f-a52f-3714dc19eed2\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg"
Sep 05 08:10:21.639764 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.639741    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23ff45e4-cfba-4b8f-a52f-3714dc19eed2-tls-certs\") pod \"sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg\" (UID: \"23ff45e4-cfba-4b8f-a52f-3714dc19eed2\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg"
Sep 05 08:10:21.645804 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.645780    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d72jv\" (UniqueName: \"kubernetes.io/projected/23ff45e4-cfba-4b8f-a52f-3714dc19eed2-kube-api-access-d72jv\") pod \"sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg\" (UID: \"23ff45e4-cfba-4b8f-a52f-3714dc19eed2\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg"
Sep 05 08:10:21.730637 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.730602    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16510468-4894-4bc4-b7cb-884a86f17437" path="/var/lib/kubelet/pods/16510468-4894-4bc4-b7cb-884a86f17437/volumes"
Sep 05 08:10:21.731012 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.730999    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="429b258a-caaf-4cef-aa49-fb975d367945" path="/var/lib/kubelet/pods/429b258a-caaf-4cef-aa49-fb975d367945/volumes"
Sep 05 08:10:21.813897 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:21.813866    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-8c2e20/sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg"
Sep 05 08:10:22.092236 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:22.092017    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-8c2e20/sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg"]
Sep 05 08:10:22.094940 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:10:22.094908    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod23ff45e4_cfba_4b8f_a52f_3714dc19eed2.slice/crio-1aac26f490399d55f34e4b954997fda8e1b9ebef4e7b25ac70ddf21c5a6a0359 WatchSource:0}: Error finding container 1aac26f490399d55f34e4b954997fda8e1b9ebef4e7b25ac70ddf21c5a6a0359: Status 404 returned error can't find the container with id 1aac26f490399d55f34e4b954997fda8e1b9ebef4e7b25ac70ddf21c5a6a0359
Sep 05 08:10:22.210499 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:22.210456    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-8c2e20/sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg" event={"ID":"23ff45e4-cfba-4b8f-a52f-3714dc19eed2","Type":"ContainerStarted","Data":"264c2635e07b5145561c0b9c9a020d2622516f45e540697ec482aa067a61c0b8"}
Sep 05 08:10:22.210712 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:22.210508    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-8c2e20/sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg" event={"ID":"23ff45e4-cfba-4b8f-a52f-3714dc19eed2","Type":"ContainerStarted","Data":"1aac26f490399d55f34e4b954997fda8e1b9ebef4e7b25ac70ddf21c5a6a0359"}
Sep 05 08:10:22.226033 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:22.225990    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-8c2e20/sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg" podStartSLOduration=1.225974225 podStartE2EDuration="1.225974225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:10:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:10:22.225135278 +0000 UTC m=+1324.995277639" watchObservedRunningTime="2026-09-05 08:10:22.225974225 +0000 UTC m=+1324.996116561"
Sep 05 08:10:31.813963 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:31.813934    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-8c2e20/sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg"
Sep 05 08:10:31.814372 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:31.813979    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-8c2e20/sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg"
Sep 05 08:10:31.826574 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:31.826548    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-8c2e20/sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg"
Sep 05 08:10:32.272742 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:32.272652    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-8c2e20/sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg"
Sep 05 08:10:40.776714 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:40.776620    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:10:43.055168 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:43.055134    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:10:46.634685 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:46.634654    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:10:46.697505 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:46.697458    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:10:46.995076 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:46.994981    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:10:47.304728 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:47.304701    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"]
Sep 05 08:10:47.308153 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:47.308132    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"
Sep 05 08:10:47.310701 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:47.310682    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-e171e4-openshift-default-dockercfg-ws6kt\""
Sep 05 08:10:47.319615 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:47.319596    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"]
Sep 05 08:10:47.331122 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:47.331092    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:10:47.483752 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:47.483709    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:10:47.486006 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:47.485977    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5f08494-b053-4b39-ba19-317ec4148ffa-istio-token\") pod \"e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8\" (UID: \"d5f08494-b053-4b39-ba19-317ec4148ffa\") " pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"
Sep 05 08:10:47.486144 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:47.486024    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szj9m\" (UniqueName: \"kubernetes.io/projected/d5f08494-b053-4b39-ba19-317ec4148ffa-kube-api-access-szj9m\") pod \"e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8\" (UID: \"d5f08494-b053-4b39-ba19-317ec4148ffa\") " pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"
Sep 05 08:10:47.486144 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:47.486073    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5f08494-b053-4b39-ba19-317ec4148ffa-istio-envoy\") pod \"e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8\" (UID: \"d5f08494-b053-4b39-ba19-317ec4148ffa\") " pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"
Sep 05 08:10:47.486144 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:47.486099    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5f08494-b053-4b39-ba19-317ec4148ffa-istio-podinfo\") pod \"e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8\" (UID: \"d5f08494-b053-4b39-ba19-317ec4148ffa\") " pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"
Sep 05 08:10:47.486144 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:47.486143    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5f08494-b053-4b39-ba19-317ec4148ffa-workload-certs\") pod \"e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8\" (UID: \"d5f08494-b053-4b39-ba19-317ec4148ffa\") " pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"
Sep 05 08:10:47.486302 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:47.486197    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5f08494-b053-4b39-ba19-317ec4148ffa-workload-socket\") pod \"e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8\" (UID: \"d5f08494-b053-4b39-ba19-317ec4148ffa\") " pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"
Sep 05 08:10:47.486302 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:47.486225    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5f08494-b053-4b39-ba19-317ec4148ffa-istiod-ca-cert\") pod \"e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8\" (UID: \"d5f08494-b053-4b39-ba19-317ec4148ffa\") " pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"
Sep 05 08:10:47.486302 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:47.486248    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5f08494-b053-4b39-ba19-317ec4148ffa-credential-socket\") pod \"e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8\" (UID: \"d5f08494-b053-4b39-ba19-317ec4148ffa\") " pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"
Sep 05 08:10:47.486302 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:47.486271    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5f08494-b053-4b39-ba19-317ec4148ffa-istio-data\") pod \"e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8\" (UID: \"d5f08494-b053-4b39-ba19-317ec4148ffa\") " pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"
Sep 05 08:10:47.587190 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:47.587099    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5f08494-b053-4b39-ba19-317ec4148ffa-workload-socket\") pod \"e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8\" (UID: \"d5f08494-b053-4b39-ba19-317ec4148ffa\") " pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"
Sep 05 08:10:47.587190 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:47.587143    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5f08494-b053-4b39-ba19-317ec4148ffa-istiod-ca-cert\") pod \"e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8\" (UID: \"d5f08494-b053-4b39-ba19-317ec4148ffa\") " pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"
Sep 05 08:10:47.587190 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:47.587174    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5f08494-b053-4b39-ba19-317ec4148ffa-credential-socket\") pod \"e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8\" (UID: \"d5f08494-b053-4b39-ba19-317ec4148ffa\") " pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"
Sep 05 08:10:47.587481 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:47.587202    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5f08494-b053-4b39-ba19-317ec4148ffa-istio-data\") pod \"e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8\" (UID: \"d5f08494-b053-4b39-ba19-317ec4148ffa\") " pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"
Sep 05 08:10:47.587481 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:47.587231    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5f08494-b053-4b39-ba19-317ec4148ffa-istio-token\") pod \"e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8\" (UID: \"d5f08494-b053-4b39-ba19-317ec4148ffa\") " pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"
Sep 05 08:10:47.587481 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:47.587276    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szj9m\" (UniqueName: \"kubernetes.io/projected/d5f08494-b053-4b39-ba19-317ec4148ffa-kube-api-access-szj9m\") pod \"e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8\" (UID: \"d5f08494-b053-4b39-ba19-317ec4148ffa\") " pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"
Sep 05 08:10:47.587481 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:47.587308    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5f08494-b053-4b39-ba19-317ec4148ffa-istio-envoy\") pod \"e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8\" (UID: \"d5f08494-b053-4b39-ba19-317ec4148ffa\") " pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"
Sep 05 08:10:47.587481 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:47.587346    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5f08494-b053-4b39-ba19-317ec4148ffa-istio-podinfo\") pod \"e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8\" (UID: \"d5f08494-b053-4b39-ba19-317ec4148ffa\") " pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"
Sep 05 08:10:47.587481 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:47.587376    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5f08494-b053-4b39-ba19-317ec4148ffa-workload-certs\") pod \"e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8\" (UID: \"d5f08494-b053-4b39-ba19-317ec4148ffa\") " pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"
Sep 05 08:10:47.587789 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:47.587570    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5f08494-b053-4b39-ba19-317ec4148ffa-credential-socket\") pod \"e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8\" (UID: \"d5f08494-b053-4b39-ba19-317ec4148ffa\") " pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"
Sep 05 08:10:47.587789 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:47.587667    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5f08494-b053-4b39-ba19-317ec4148ffa-workload-socket\") pod \"e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8\" (UID: \"d5f08494-b053-4b39-ba19-317ec4148ffa\") " pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"
Sep 05 08:10:47.587789 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:47.587725    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5f08494-b053-4b39-ba19-317ec4148ffa-workload-certs\") pod \"e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8\" (UID: \"d5f08494-b053-4b39-ba19-317ec4148ffa\") " pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"
Sep 05 08:10:47.588002 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:47.587929    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5f08494-b053-4b39-ba19-317ec4148ffa-istiod-ca-cert\") pod \"e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8\" (UID: \"d5f08494-b053-4b39-ba19-317ec4148ffa\") " pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"
Sep 05 08:10:47.588055 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:47.588005    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5f08494-b053-4b39-ba19-317ec4148ffa-istio-data\") pod \"e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8\" (UID: \"d5f08494-b053-4b39-ba19-317ec4148ffa\") " pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"
Sep 05 08:10:47.589927 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:47.589904    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5f08494-b053-4b39-ba19-317ec4148ffa-istio-envoy\") pod \"e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8\" (UID: \"d5f08494-b053-4b39-ba19-317ec4148ffa\") " pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"
Sep 05 08:10:47.590020 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:47.589994    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5f08494-b053-4b39-ba19-317ec4148ffa-istio-podinfo\") pod \"e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8\" (UID: \"d5f08494-b053-4b39-ba19-317ec4148ffa\") " pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"
Sep 05 08:10:47.595630 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:47.595610    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5f08494-b053-4b39-ba19-317ec4148ffa-istio-token\") pod \"e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8\" (UID: \"d5f08494-b053-4b39-ba19-317ec4148ffa\") " pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"
Sep 05 08:10:47.595837 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:47.595814    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szj9m\" (UniqueName: \"kubernetes.io/projected/d5f08494-b053-4b39-ba19-317ec4148ffa-kube-api-access-szj9m\") pod \"e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8\" (UID: \"d5f08494-b053-4b39-ba19-317ec4148ffa\") " pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"
Sep 05 08:10:47.618724 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:47.618700    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"
Sep 05 08:10:47.894033 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:47.893932    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"]
Sep 05 08:10:47.896270 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:10:47.896246    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5f08494_b053_4b39_ba19_317ec4148ffa.slice/crio-1ca096a79758fdf83e4d4653e6f100ae5a4945041d0602c7055c2ea1c8fd1b10 WatchSource:0}: Error finding container 1ca096a79758fdf83e4d4653e6f100ae5a4945041d0602c7055c2ea1c8fd1b10: Status 404 returned error can't find the container with id 1ca096a79758fdf83e4d4653e6f100ae5a4945041d0602c7055c2ea1c8fd1b10
Sep 05 08:10:47.898635 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:47.898594    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 05 08:10:47.898709 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:47.898686    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 05 08:10:47.898750 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:47.898729    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 05 08:10:48.345356 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:48.345318    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8" event={"ID":"d5f08494-b053-4b39-ba19-317ec4148ffa","Type":"ContainerStarted","Data":"3410f9900af9992f126de30f6c9280c2243604a7005f7a3edc79a07001e70ee1"}
Sep 05 08:10:48.345529 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:48.345373    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8" event={"ID":"d5f08494-b053-4b39-ba19-317ec4148ffa","Type":"ContainerStarted","Data":"1ca096a79758fdf83e4d4653e6f100ae5a4945041d0602c7055c2ea1c8fd1b10"}
Sep 05 08:10:48.365403 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:48.365360    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8" podStartSLOduration=1.365345473 podStartE2EDuration="1.365345473s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:10:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:10:48.363595972 +0000 UTC m=+1351.133738329" watchObservedRunningTime="2026-09-05 08:10:48.365345473 +0000 UTC m=+1351.135487809"
Sep 05 08:10:48.619466 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:48.619386    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"
Sep 05 08:10:48.624802 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:48.624781    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"
Sep 05 08:10:49.350121 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:49.350092    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"
Sep 05 08:10:49.351171 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:49.351146    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"
Sep 05 08:10:50.297220 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:50.297181    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:10:50.391773 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:50.391741    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:10:51.437339 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:51.437303    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:10:51.542152 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:51.542119    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:10:51.909838 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:51.909803    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:10:52.008317 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:52.008285    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:10:52.029738 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:52.029706    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-e171e4-8dd49c65d-srsv2"]
Sep 05 08:10:52.033282 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:52.033264    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-e171e4-8dd49c65d-srsv2"
Sep 05 08:10:52.035920 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:52.035896    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-e171e4-dockercfg-82kmm\""
Sep 05 08:10:52.036053 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:52.035904    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-e171e4\""
Sep 05 08:10:52.044529 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:52.044508    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-e171e4-8dd49c65d-srsv2"]
Sep 05 08:10:52.107602 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:52.107542    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:10:52.121968 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:52.121944    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42e6fe07-85c0-4203-8cfc-2648e7af301e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-e171e4-8dd49c65d-srsv2\" (UID: \"42e6fe07-85c0-4203-8cfc-2648e7af301e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-e171e4-8dd49c65d-srsv2"
Sep 05 08:10:52.122080 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:52.122061    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsmk5\" (UniqueName: \"kubernetes.io/projected/42e6fe07-85c0-4203-8cfc-2648e7af301e-kube-api-access-rsmk5\") pod \"payload-pre-processing-e2e-api-a-e171e4-8dd49c65d-srsv2\" (UID: \"42e6fe07-85c0-4203-8cfc-2648e7af301e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-e171e4-8dd49c65d-srsv2"
Sep 05 08:10:52.223186 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:52.223090    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsmk5\" (UniqueName: \"kubernetes.io/projected/42e6fe07-85c0-4203-8cfc-2648e7af301e-kube-api-access-rsmk5\") pod \"payload-pre-processing-e2e-api-a-e171e4-8dd49c65d-srsv2\" (UID: \"42e6fe07-85c0-4203-8cfc-2648e7af301e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-e171e4-8dd49c65d-srsv2"
Sep 05 08:10:52.223186 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:52.223154    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42e6fe07-85c0-4203-8cfc-2648e7af301e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-e171e4-8dd49c65d-srsv2\" (UID: \"42e6fe07-85c0-4203-8cfc-2648e7af301e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-e171e4-8dd49c65d-srsv2"
Sep 05 08:10:52.223811 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:52.223792    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42e6fe07-85c0-4203-8cfc-2648e7af301e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-e171e4-8dd49c65d-srsv2\" (UID: \"42e6fe07-85c0-4203-8cfc-2648e7af301e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-e171e4-8dd49c65d-srsv2"
Sep 05 08:10:52.231225 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:52.231202    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsmk5\" (UniqueName: \"kubernetes.io/projected/42e6fe07-85c0-4203-8cfc-2648e7af301e-kube-api-access-rsmk5\") pod \"payload-pre-processing-e2e-api-a-e171e4-8dd49c65d-srsv2\" (UID: \"42e6fe07-85c0-4203-8cfc-2648e7af301e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-e171e4-8dd49c65d-srsv2"
Sep 05 08:10:52.344697 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:52.344662    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-e171e4-8dd49c65d-srsv2"
Sep 05 08:10:52.618155 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:52.618130    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-e171e4-8dd49c65d-srsv2"]
Sep 05 08:10:52.620295 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:10:52.620268    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42e6fe07_85c0_4203_8cfc_2648e7af301e.slice/crio-d208d02117721ec98231e1c534316a9650b41f59436ff13807f4dc823bdc6a3c WatchSource:0}: Error finding container d208d02117721ec98231e1c534316a9650b41f59436ff13807f4dc823bdc6a3c: Status 404 returned error can't find the container with id d208d02117721ec98231e1c534316a9650b41f59436ff13807f4dc823bdc6a3c
Sep 05 08:10:53.372132 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:53.372097    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-e171e4-8dd49c65d-srsv2" event={"ID":"42e6fe07-85c0-4203-8cfc-2648e7af301e","Type":"ContainerStarted","Data":"5881c2eb2d212bfb16b560ea35ef4658d072aa7c94ae46237c6364f34d95b441"}
Sep 05 08:10:53.372337 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:53.372156    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-e171e4-8dd49c65d-srsv2" event={"ID":"42e6fe07-85c0-4203-8cfc-2648e7af301e","Type":"ContainerStarted","Data":"d208d02117721ec98231e1c534316a9650b41f59436ff13807f4dc823bdc6a3c"}
Sep 05 08:10:53.372337 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:53.372180    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-e171e4-8dd49c65d-srsv2"
Sep 05 08:10:53.387649 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:53.387585    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-e171e4-8dd49c65d-srsv2" podStartSLOduration=1.38755439 podStartE2EDuration="1.38755439s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:10:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:10:53.386874852 +0000 UTC m=+1356.157017196" watchObservedRunningTime="2026-09-05 08:10:53.38755439 +0000 UTC m=+1356.157696728"
Sep 05 08:10:58.296182 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:58.296140    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-5ef325/sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2"]
Sep 05 08:10:58.299957 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:58.299933    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-5ef325/sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2"
Sep 05 08:10:58.303405 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:58.303378    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-5ef325\"/\"openshift-service-ca.crt\""
Sep 05 08:10:58.303541 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:58.303477    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-5ef325\"/\"kube-root-ca.crt\""
Sep 05 08:10:58.303541 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:58.303477    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-5ef325\"/\"sub-test-model-5ef325-kserve-self-signed-certs\""
Sep 05 08:10:58.303660 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:58.303481    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-5ef325\"/\"default-dockercfg-fqjc2\""
Sep 05 08:10:58.313996 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:58.313975    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-5ef325/sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2"]
Sep 05 08:10:58.339007 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:58.338979    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:10:58.369499 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:58.369465    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abd861b5-990a-4757-94db-516c25121b68-model-cache\") pod \"sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2\" (UID: \"abd861b5-990a-4757-94db-516c25121b68\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2"
Sep 05 08:10:58.369611 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:58.369512    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lfmf\" (UniqueName: \"kubernetes.io/projected/abd861b5-990a-4757-94db-516c25121b68-kube-api-access-9lfmf\") pod \"sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2\" (UID: \"abd861b5-990a-4757-94db-516c25121b68\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2"
Sep 05 08:10:58.369611 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:58.369557    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abd861b5-990a-4757-94db-516c25121b68-tls-certs\") pod \"sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2\" (UID: \"abd861b5-990a-4757-94db-516c25121b68\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2"
Sep 05 08:10:58.369611 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:58.369602    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abd861b5-990a-4757-94db-516c25121b68-dshm\") pod \"sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2\" (UID: \"abd861b5-990a-4757-94db-516c25121b68\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2"
Sep 05 08:10:58.369789 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:58.369650    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abd861b5-990a-4757-94db-516c25121b68-tmp-dir\") pod \"sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2\" (UID: \"abd861b5-990a-4757-94db-516c25121b68\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2"
Sep 05 08:10:58.369789 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:58.369696    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abd861b5-990a-4757-94db-516c25121b68-home\") pod \"sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2\" (UID: \"abd861b5-990a-4757-94db-516c25121b68\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2"
Sep 05 08:10:58.470219 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:58.470179    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abd861b5-990a-4757-94db-516c25121b68-home\") pod \"sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2\" (UID: \"abd861b5-990a-4757-94db-516c25121b68\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2"
Sep 05 08:10:58.470417 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:58.470241    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abd861b5-990a-4757-94db-516c25121b68-model-cache\") pod \"sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2\" (UID: \"abd861b5-990a-4757-94db-516c25121b68\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2"
Sep 05 08:10:58.470417 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:58.470293    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lfmf\" (UniqueName: \"kubernetes.io/projected/abd861b5-990a-4757-94db-516c25121b68-kube-api-access-9lfmf\") pod \"sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2\" (UID: \"abd861b5-990a-4757-94db-516c25121b68\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2"
Sep 05 08:10:58.470417 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:58.470317    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abd861b5-990a-4757-94db-516c25121b68-tls-certs\") pod \"sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2\" (UID: \"abd861b5-990a-4757-94db-516c25121b68\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2"
Sep 05 08:10:58.470417 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:58.470355    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abd861b5-990a-4757-94db-516c25121b68-dshm\") pod \"sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2\" (UID: \"abd861b5-990a-4757-94db-516c25121b68\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2"
Sep 05 08:10:58.470417 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:58.470373    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abd861b5-990a-4757-94db-516c25121b68-tmp-dir\") pod \"sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2\" (UID: \"abd861b5-990a-4757-94db-516c25121b68\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2"
Sep 05 08:10:58.470741 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:58.470655    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abd861b5-990a-4757-94db-516c25121b68-home\") pod \"sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2\" (UID: \"abd861b5-990a-4757-94db-516c25121b68\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2"
Sep 05 08:10:58.470741 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:58.470681    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abd861b5-990a-4757-94db-516c25121b68-model-cache\") pod \"sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2\" (UID: \"abd861b5-990a-4757-94db-516c25121b68\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2"
Sep 05 08:10:58.470741 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:58.470717    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abd861b5-990a-4757-94db-516c25121b68-tmp-dir\") pod \"sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2\" (UID: \"abd861b5-990a-4757-94db-516c25121b68\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2"
Sep 05 08:10:58.472736 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:58.472707    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abd861b5-990a-4757-94db-516c25121b68-dshm\") pod \"sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2\" (UID: \"abd861b5-990a-4757-94db-516c25121b68\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2"
Sep 05 08:10:58.473013 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:58.472995    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abd861b5-990a-4757-94db-516c25121b68-tls-certs\") pod \"sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2\" (UID: \"abd861b5-990a-4757-94db-516c25121b68\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2"
Sep 05 08:10:58.478529 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:58.478508    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lfmf\" (UniqueName: \"kubernetes.io/projected/abd861b5-990a-4757-94db-516c25121b68-kube-api-access-9lfmf\") pod \"sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2\" (UID: \"abd861b5-990a-4757-94db-516c25121b68\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2"
Sep 05 08:10:58.609790 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:58.609750    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-5ef325/sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2"
Sep 05 08:10:58.888743 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:58.888712    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-5ef325/sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2"]
Sep 05 08:10:58.891496 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:10:58.891461    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podabd861b5_990a_4757_94db_516c25121b68.slice/crio-6253145462a019de2c6b1209963454ac2516f0cf8b63e0630583fc9af86a20fd WatchSource:0}: Error finding container 6253145462a019de2c6b1209963454ac2516f0cf8b63e0630583fc9af86a20fd: Status 404 returned error can't find the container with id 6253145462a019de2c6b1209963454ac2516f0cf8b63e0630583fc9af86a20fd
Sep 05 08:10:59.409679 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:59.409645    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-5ef325/sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2" event={"ID":"abd861b5-990a-4757-94db-516c25121b68","Type":"ContainerStarted","Data":"0049086dca666a931b9c03e0dad7d38f6e247b4f0f76f45c49669cb395c76275"}
Sep 05 08:10:59.409679 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:59.409684    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-5ef325/sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2" event={"ID":"abd861b5-990a-4757-94db-516c25121b68","Type":"ContainerStarted","Data":"6253145462a019de2c6b1209963454ac2516f0cf8b63e0630583fc9af86a20fd"}
Sep 05 08:10:59.427814 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:10:59.427766    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-5ef325/sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2" podStartSLOduration=1.427753133 podStartE2EDuration="1.427753133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:10:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:10:59.425579797 +0000 UTC m=+1362.195722158" watchObservedRunningTime="2026-09-05 08:10:59.427753133 +0000 UTC m=+1362.197895515"
Sep 05 08:11:04.379202 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:04.379169    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-e171e4-8dd49c65d-srsv2"
Sep 05 08:11:08.610231 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:08.610186    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-5ef325/sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2"
Sep 05 08:11:08.610729 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:08.610333    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-5ef325/sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2"
Sep 05 08:11:08.623079 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:08.623054    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-5ef325/sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2"
Sep 05 08:11:09.463812 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:09.463716    2563 generic.go:356] "Generic (PLEG): container finished" podID="d5f08494-b053-4b39-ba19-317ec4148ffa" containerID="3410f9900af9992f126de30f6c9280c2243604a7005f7a3edc79a07001e70ee1" exitCode=0
Sep 05 08:11:09.463812 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:09.463792    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8" event={"ID":"d5f08494-b053-4b39-ba19-317ec4148ffa","Type":"ContainerDied","Data":"3410f9900af9992f126de30f6c9280c2243604a7005f7a3edc79a07001e70ee1"}
Sep 05 08:11:09.464354 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:09.464335    2563 scope.go:117] "RemoveContainer" containerID="3410f9900af9992f126de30f6c9280c2243604a7005f7a3edc79a07001e70ee1"
Sep 05 08:11:09.465600 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:09.465570    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 05 08:11:09.465700 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:09.465654    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 05 08:11:09.465763 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:09.465704    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 05 08:11:09.479095 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:09.479076    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-5ef325/sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2"
Sep 05 08:11:10.471371 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:10.471334    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8" event={"ID":"d5f08494-b053-4b39-ba19-317ec4148ffa","Type":"ContainerStarted","Data":"4b6ac0edb0455eb816a4385cf8376514acf9c0882d80e31654bfc381bc1051d3"}
Sep 05 08:11:10.619807 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:10.619771    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"
Sep 05 08:11:10.620007 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:10.619970    2563 patch_prober.go:28] interesting pod/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 05 08:11:10.620052 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:10.620026    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8" podUID="d5f08494-b053-4b39-ba19-317ec4148ffa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 05 08:11:11.619677 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:11.619634    2563 patch_prober.go:28] interesting pod/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 05 08:11:11.620209 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:11.619691    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8" podUID="d5f08494-b053-4b39-ba19-317ec4148ffa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 05 08:11:12.619426 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:12.619393    2563 patch_prober.go:28] interesting pod/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 05 08:11:12.619608 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:12.619448    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8" podUID="d5f08494-b053-4b39-ba19-317ec4148ffa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 05 08:11:13.620134 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:13.620092    2563 patch_prober.go:28] interesting pod/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 05 08:11:13.620622 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:13.620162    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8" podUID="d5f08494-b053-4b39-ba19-317ec4148ffa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 05 08:11:14.619538 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:14.619494    2563 patch_prober.go:28] interesting pod/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 05 08:11:14.619760 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:14.619560    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8" podUID="d5f08494-b053-4b39-ba19-317ec4148ffa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 05 08:11:15.276033 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:15.275993    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:11:15.619741 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:15.619705    2563 patch_prober.go:28] interesting pod/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 05 08:11:15.619956 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:15.619766    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8" podUID="d5f08494-b053-4b39-ba19-317ec4148ffa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 05 08:11:15.726049 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:15.726017    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:11:16.000726 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.000636    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"]
Sep 05 08:11:16.016163 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.016133    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"
Sep 05 08:11:16.019050 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.019024    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-d850e3-openshift-default-dockercfg-tx6bd\""
Sep 05 08:11:16.019403 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.019381    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"]
Sep 05 08:11:16.026955 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.026922    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:11:16.151939 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.151909    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92dd3f54-113e-421f-b276-d51c16796c43-workload-certs\") pod \"e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7\" (UID: \"92dd3f54-113e-421f-b276-d51c16796c43\") " pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"
Sep 05 08:11:16.152151 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.151967    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llqrm\" (UniqueName: \"kubernetes.io/projected/92dd3f54-113e-421f-b276-d51c16796c43-kube-api-access-llqrm\") pod \"e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7\" (UID: \"92dd3f54-113e-421f-b276-d51c16796c43\") " pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"
Sep 05 08:11:16.152151 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.152039    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92dd3f54-113e-421f-b276-d51c16796c43-workload-socket\") pod \"e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7\" (UID: \"92dd3f54-113e-421f-b276-d51c16796c43\") " pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"
Sep 05 08:11:16.152151 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.152080    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92dd3f54-113e-421f-b276-d51c16796c43-istiod-ca-cert\") pod \"e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7\" (UID: \"92dd3f54-113e-421f-b276-d51c16796c43\") " pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"
Sep 05 08:11:16.152151 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.152107    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92dd3f54-113e-421f-b276-d51c16796c43-istio-envoy\") pod \"e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7\" (UID: \"92dd3f54-113e-421f-b276-d51c16796c43\") " pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"
Sep 05 08:11:16.152328 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.152165    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92dd3f54-113e-421f-b276-d51c16796c43-credential-socket\") pod \"e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7\" (UID: \"92dd3f54-113e-421f-b276-d51c16796c43\") " pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"
Sep 05 08:11:16.152328 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.152188    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92dd3f54-113e-421f-b276-d51c16796c43-istio-token\") pod \"e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7\" (UID: \"92dd3f54-113e-421f-b276-d51c16796c43\") " pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"
Sep 05 08:11:16.152328 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.152222    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92dd3f54-113e-421f-b276-d51c16796c43-istio-data\") pod \"e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7\" (UID: \"92dd3f54-113e-421f-b276-d51c16796c43\") " pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"
Sep 05 08:11:16.152328 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.152244    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92dd3f54-113e-421f-b276-d51c16796c43-istio-podinfo\") pod \"e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7\" (UID: \"92dd3f54-113e-421f-b276-d51c16796c43\") " pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"
Sep 05 08:11:16.253195 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.253103    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92dd3f54-113e-421f-b276-d51c16796c43-workload-certs\") pod \"e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7\" (UID: \"92dd3f54-113e-421f-b276-d51c16796c43\") " pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"
Sep 05 08:11:16.253195 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.253183    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llqrm\" (UniqueName: \"kubernetes.io/projected/92dd3f54-113e-421f-b276-d51c16796c43-kube-api-access-llqrm\") pod \"e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7\" (UID: \"92dd3f54-113e-421f-b276-d51c16796c43\") " pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"
Sep 05 08:11:16.253439 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.253222    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92dd3f54-113e-421f-b276-d51c16796c43-workload-socket\") pod \"e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7\" (UID: \"92dd3f54-113e-421f-b276-d51c16796c43\") " pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"
Sep 05 08:11:16.253439 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.253247    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92dd3f54-113e-421f-b276-d51c16796c43-istiod-ca-cert\") pod \"e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7\" (UID: \"92dd3f54-113e-421f-b276-d51c16796c43\") " pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"
Sep 05 08:11:16.253439 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.253274    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92dd3f54-113e-421f-b276-d51c16796c43-istio-envoy\") pod \"e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7\" (UID: \"92dd3f54-113e-421f-b276-d51c16796c43\") " pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"
Sep 05 08:11:16.253439 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.253324    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92dd3f54-113e-421f-b276-d51c16796c43-credential-socket\") pod \"e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7\" (UID: \"92dd3f54-113e-421f-b276-d51c16796c43\") " pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"
Sep 05 08:11:16.253439 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.253352    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92dd3f54-113e-421f-b276-d51c16796c43-istio-token\") pod \"e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7\" (UID: \"92dd3f54-113e-421f-b276-d51c16796c43\") " pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"
Sep 05 08:11:16.253439 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.253402    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92dd3f54-113e-421f-b276-d51c16796c43-istio-data\") pod \"e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7\" (UID: \"92dd3f54-113e-421f-b276-d51c16796c43\") " pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"
Sep 05 08:11:16.253439 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.253435    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92dd3f54-113e-421f-b276-d51c16796c43-istio-podinfo\") pod \"e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7\" (UID: \"92dd3f54-113e-421f-b276-d51c16796c43\") " pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"
Sep 05 08:11:16.253762 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.253576    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92dd3f54-113e-421f-b276-d51c16796c43-workload-certs\") pod \"e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7\" (UID: \"92dd3f54-113e-421f-b276-d51c16796c43\") " pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"
Sep 05 08:11:16.253762 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.253650    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92dd3f54-113e-421f-b276-d51c16796c43-workload-socket\") pod \"e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7\" (UID: \"92dd3f54-113e-421f-b276-d51c16796c43\") " pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"
Sep 05 08:11:16.253974 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.253950    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92dd3f54-113e-421f-b276-d51c16796c43-credential-socket\") pod \"e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7\" (UID: \"92dd3f54-113e-421f-b276-d51c16796c43\") " pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"
Sep 05 08:11:16.254048 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.253995    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92dd3f54-113e-421f-b276-d51c16796c43-istio-data\") pod \"e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7\" (UID: \"92dd3f54-113e-421f-b276-d51c16796c43\") " pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"
Sep 05 08:11:16.254092 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.254064    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92dd3f54-113e-421f-b276-d51c16796c43-istiod-ca-cert\") pod \"e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7\" (UID: \"92dd3f54-113e-421f-b276-d51c16796c43\") " pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"
Sep 05 08:11:16.256027 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.256000    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92dd3f54-113e-421f-b276-d51c16796c43-istio-envoy\") pod \"e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7\" (UID: \"92dd3f54-113e-421f-b276-d51c16796c43\") " pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"
Sep 05 08:11:16.256165 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.256147    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92dd3f54-113e-421f-b276-d51c16796c43-istio-podinfo\") pod \"e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7\" (UID: \"92dd3f54-113e-421f-b276-d51c16796c43\") " pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"
Sep 05 08:11:16.262792 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.262763    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92dd3f54-113e-421f-b276-d51c16796c43-istio-token\") pod \"e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7\" (UID: \"92dd3f54-113e-421f-b276-d51c16796c43\") " pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"
Sep 05 08:11:16.262903 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.262867    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llqrm\" (UniqueName: \"kubernetes.io/projected/92dd3f54-113e-421f-b276-d51c16796c43-kube-api-access-llqrm\") pod \"e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7\" (UID: \"92dd3f54-113e-421f-b276-d51c16796c43\") " pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"
Sep 05 08:11:16.309795 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.309722    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:11:16.329790 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.329758    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"
Sep 05 08:11:16.604985 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.604950    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"]
Sep 05 08:11:16.608164 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:11:16.608134    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92dd3f54_113e_421f_b276_d51c16796c43.slice/crio-fb449e253cfc3ef7371aa48ef543b029fd5214cf70633277af6066151e51f24f WatchSource:0}: Error finding container fb449e253cfc3ef7371aa48ef543b029fd5214cf70633277af6066151e51f24f: Status 404 returned error can't find the container with id fb449e253cfc3ef7371aa48ef543b029fd5214cf70633277af6066151e51f24f
Sep 05 08:11:16.611050 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.611013    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 05 08:11:16.611168 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.611082    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 05 08:11:16.611168 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.611110    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 05 08:11:16.791908 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.791870    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"
Sep 05 08:11:16.792316 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.792287    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"
Sep 05 08:11:16.793639 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:16.793611    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"
Sep 05 08:11:17.516810 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:17.516771    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7" event={"ID":"92dd3f54-113e-421f-b276-d51c16796c43","Type":"ContainerStarted","Data":"f116c75e4306d79d732636d6ec9fb646282e81202f5f242483b831ea7f2607b6"}
Sep 05 08:11:17.517210 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:17.516832    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7" event={"ID":"92dd3f54-113e-421f-b276-d51c16796c43","Type":"ContainerStarted","Data":"fb449e253cfc3ef7371aa48ef543b029fd5214cf70633277af6066151e51f24f"}
Sep 05 08:11:17.536749 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:17.536689    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7" podStartSLOduration=2.536669466 podStartE2EDuration="2.536669466s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:11:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:11:17.535686366 +0000 UTC m=+1380.305828724" watchObservedRunningTime="2026-09-05 08:11:17.536669466 +0000 UTC m=+1380.306811803"
Sep 05 08:11:18.330146 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:18.330117    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"
Sep 05 08:11:18.335320 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:18.335297    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"
Sep 05 08:11:18.522418 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:18.522390    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"
Sep 05 08:11:18.523509 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:18.523487    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"
Sep 05 08:11:19.488833 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:19.488786    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:11:19.574120 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:19.574090    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:11:20.639212 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:20.639175    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:11:20.741742 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:20.741710    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:11:21.036124 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:21.036024    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:11:21.141017 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:21.140982    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:11:21.206716 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:21.206682    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d850e3-685859947d-v2jln"]
Sep 05 08:11:21.210025 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:21.209998    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:11:21.210545 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:21.210530    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d850e3-685859947d-v2jln"
Sep 05 08:11:21.213122 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:21.213096    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-d850e3\""
Sep 05 08:11:21.213241 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:21.213165    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-d850e3-dockercfg-wjcbc\""
Sep 05 08:11:21.220646 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:21.220621    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d850e3-685859947d-v2jln"]
Sep 05 08:11:21.306507 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:21.306477    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c9a9942-d766-4ad2-9e3b-c9490319d305-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-d850e3-685859947d-v2jln\" (UID: \"8c9a9942-d766-4ad2-9e3b-c9490319d305\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d850e3-685859947d-v2jln"
Sep 05 08:11:21.306692 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:21.306649    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lgr7\" (UniqueName: \"kubernetes.io/projected/8c9a9942-d766-4ad2-9e3b-c9490319d305-kube-api-access-7lgr7\") pod \"payload-pre-processing-e2e-api-b-d850e3-685859947d-v2jln\" (UID: \"8c9a9942-d766-4ad2-9e3b-c9490319d305\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d850e3-685859947d-v2jln"
Sep 05 08:11:21.408048 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:21.408008    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lgr7\" (UniqueName: \"kubernetes.io/projected/8c9a9942-d766-4ad2-9e3b-c9490319d305-kube-api-access-7lgr7\") pod \"payload-pre-processing-e2e-api-b-d850e3-685859947d-v2jln\" (UID: \"8c9a9942-d766-4ad2-9e3b-c9490319d305\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d850e3-685859947d-v2jln"
Sep 05 08:11:21.408198 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:21.408057    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c9a9942-d766-4ad2-9e3b-c9490319d305-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-d850e3-685859947d-v2jln\" (UID: \"8c9a9942-d766-4ad2-9e3b-c9490319d305\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d850e3-685859947d-v2jln"
Sep 05 08:11:21.408730 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:21.408711    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c9a9942-d766-4ad2-9e3b-c9490319d305-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-d850e3-685859947d-v2jln\" (UID: \"8c9a9942-d766-4ad2-9e3b-c9490319d305\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d850e3-685859947d-v2jln"
Sep 05 08:11:21.415736 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:21.415703    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lgr7\" (UniqueName: \"kubernetes.io/projected/8c9a9942-d766-4ad2-9e3b-c9490319d305-kube-api-access-7lgr7\") pod \"payload-pre-processing-e2e-api-b-d850e3-685859947d-v2jln\" (UID: \"8c9a9942-d766-4ad2-9e3b-c9490319d305\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d850e3-685859947d-v2jln"
Sep 05 08:11:21.521494 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:21.521447    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d850e3-685859947d-v2jln"
Sep 05 08:11:21.798052 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:21.798024    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d850e3-685859947d-v2jln"]
Sep 05 08:11:21.799395 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:11:21.799373    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c9a9942_d766_4ad2_9e3b_c9490319d305.slice/crio-c9fed41b8e82e7c6ef337a85c8371dd02719d06e0c6c5bdbddfa3dbcf9d55f89 WatchSource:0}: Error finding container c9fed41b8e82e7c6ef337a85c8371dd02719d06e0c6c5bdbddfa3dbcf9d55f89: Status 404 returned error can't find the container with id c9fed41b8e82e7c6ef337a85c8371dd02719d06e0c6c5bdbddfa3dbcf9d55f89
Sep 05 08:11:22.546646 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:22.546611    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d850e3-685859947d-v2jln" event={"ID":"8c9a9942-d766-4ad2-9e3b-c9490319d305","Type":"ContainerStarted","Data":"3dc6742a9b88356d5108d1d51e2c509a0574c8fa9343d7d50e88768b52684398"}
Sep 05 08:11:22.546912 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:22.546659    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d850e3-685859947d-v2jln" event={"ID":"8c9a9942-d766-4ad2-9e3b-c9490319d305","Type":"ContainerStarted","Data":"c9fed41b8e82e7c6ef337a85c8371dd02719d06e0c6c5bdbddfa3dbcf9d55f89"}
Sep 05 08:11:22.546912 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:22.546728    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d850e3-685859947d-v2jln"
Sep 05 08:11:22.563678 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:22.563631    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d850e3-685859947d-v2jln" podStartSLOduration=1.563612928 podStartE2EDuration="1.563612928s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:11:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:11:22.561277172 +0000 UTC m=+1385.331419521" watchObservedRunningTime="2026-09-05 08:11:22.563612928 +0000 UTC m=+1385.333755268"
Sep 05 08:11:33.246121 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:33.246062    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:11:33.553253 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:33.553172    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d850e3-685859947d-v2jln"
Sep 05 08:11:33.674560 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:33.674525    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-7850-7ed5b2"]
Sep 05 08:11:33.677563 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:33.677540    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7850-7ed5b2"
Sep 05 08:11:33.679639 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:33.679618    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-x4x7f\""
Sep 05 08:11:33.682269 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:33.682249    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7850-7ed5b2"]
Sep 05 08:11:33.829419 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:33.829374    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqjsc\" (UniqueName: \"kubernetes.io/projected/e313b300-4acb-452b-a280-981f1ae10366-kube-api-access-kqjsc\") pod \"mt-curl-7850-7ed5b2\" (UID: \"e313b300-4acb-452b-a280-981f1ae10366\") " pod="opendatahub/mt-curl-7850-7ed5b2"
Sep 05 08:11:33.930014 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:33.929974    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqjsc\" (UniqueName: \"kubernetes.io/projected/e313b300-4acb-452b-a280-981f1ae10366-kube-api-access-kqjsc\") pod \"mt-curl-7850-7ed5b2\" (UID: \"e313b300-4acb-452b-a280-981f1ae10366\") " pod="opendatahub/mt-curl-7850-7ed5b2"
Sep 05 08:11:33.938906 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:33.938864    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqjsc\" (UniqueName: \"kubernetes.io/projected/e313b300-4acb-452b-a280-981f1ae10366-kube-api-access-kqjsc\") pod \"mt-curl-7850-7ed5b2\" (UID: \"e313b300-4acb-452b-a280-981f1ae10366\") " pod="opendatahub/mt-curl-7850-7ed5b2"
Sep 05 08:11:34.005860 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:34.005779    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7850-7ed5b2"
Sep 05 08:11:34.277202 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:34.277111    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7850-7ed5b2"]
Sep 05 08:11:34.280182 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:11:34.280154    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode313b300_4acb_452b_a280_981f1ae10366.slice/crio-e40ea2b67143a32f30b4a99f8c43fc8765f3ebd48f056c74f512aa77b55c0ee0 WatchSource:0}: Error finding container e40ea2b67143a32f30b4a99f8c43fc8765f3ebd48f056c74f512aa77b55c0ee0: Status 404 returned error can't find the container with id e40ea2b67143a32f30b4a99f8c43fc8765f3ebd48f056c74f512aa77b55c0ee0
Sep 05 08:11:34.619870 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:34.619822    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7850-7ed5b2" event={"ID":"e313b300-4acb-452b-a280-981f1ae10366","Type":"ContainerStarted","Data":"e40ea2b67143a32f30b4a99f8c43fc8765f3ebd48f056c74f512aa77b55c0ee0"}
Sep 05 08:11:35.626353 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:35.626319    2563 generic.go:356] "Generic (PLEG): container finished" podID="e313b300-4acb-452b-a280-981f1ae10366" containerID="1d3e3863f2327986517f00fa8b18c1308e111232c4e148ccd190a3d19426121d" exitCode=0
Sep 05 08:11:35.626748 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:35.626400    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7850-7ed5b2" event={"ID":"e313b300-4acb-452b-a280-981f1ae10366","Type":"ContainerDied","Data":"1d3e3863f2327986517f00fa8b18c1308e111232c4e148ccd190a3d19426121d"}
Sep 05 08:11:36.780331 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:36.780297    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7850-7ed5b2"
Sep 05 08:11:36.810892 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:36.810581    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_mt-curl-7850-7ed5b2_e313b300-4acb-452b-a280-981f1ae10366/mt-curl-7850-7ed5b2/0.log"
Sep 05 08:11:36.831626 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:36.831596    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-7850-7ed5b2"]
Sep 05 08:11:36.833812 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:36.833790    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-7850-7ed5b2"]
Sep 05 08:11:36.859136 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:36.859118    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kqjsc\" (UniqueName: \"kubernetes.io/projected/e313b300-4acb-452b-a280-981f1ae10366-kube-api-access-kqjsc\") pod \"e313b300-4acb-452b-a280-981f1ae10366\" (UID: \"e313b300-4acb-452b-a280-981f1ae10366\") "
Sep 05 08:11:36.861323 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:36.861300    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e313b300-4acb-452b-a280-981f1ae10366-kube-api-access-kqjsc" (OuterVolumeSpecName: "kube-api-access-kqjsc") pod "e313b300-4acb-452b-a280-981f1ae10366" (UID: "e313b300-4acb-452b-a280-981f1ae10366"). InnerVolumeSpecName "kube-api-access-kqjsc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:11:36.960112 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:36.960021    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kqjsc\" (UniqueName: \"kubernetes.io/projected/e313b300-4acb-452b-a280-981f1ae10366-kube-api-access-kqjsc\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:11:36.971938 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:36.971902    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-7850-658b77"]
Sep 05 08:11:36.972285 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:36.972272    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e313b300-4acb-452b-a280-981f1ae10366" containerName="mt-curl-7850-7ed5b2"
Sep 05 08:11:36.972326 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:36.972286    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e313b300-4acb-452b-a280-981f1ae10366" containerName="mt-curl-7850-7ed5b2"
Sep 05 08:11:36.972372 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:36.972363    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e313b300-4acb-452b-a280-981f1ae10366" containerName="mt-curl-7850-7ed5b2"
Sep 05 08:11:36.974383 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:36.974367    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7850-658b77"
Sep 05 08:11:36.980783 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:36.980758    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7850-658b77"]
Sep 05 08:11:37.025052 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:37.025023    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:11:37.061304 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:37.061276    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8n7c\" (UniqueName: \"kubernetes.io/projected/cde7e13c-9780-4dc9-851c-decf990ddac2-kube-api-access-m8n7c\") pod \"mt-curl-7850-658b77\" (UID: \"cde7e13c-9780-4dc9-851c-decf990ddac2\") " pod="opendatahub/mt-curl-7850-658b77"
Sep 05 08:11:37.162418 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:37.162382    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8n7c\" (UniqueName: \"kubernetes.io/projected/cde7e13c-9780-4dc9-851c-decf990ddac2-kube-api-access-m8n7c\") pod \"mt-curl-7850-658b77\" (UID: \"cde7e13c-9780-4dc9-851c-decf990ddac2\") " pod="opendatahub/mt-curl-7850-658b77"
Sep 05 08:11:37.169731 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:37.169709    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8n7c\" (UniqueName: \"kubernetes.io/projected/cde7e13c-9780-4dc9-851c-decf990ddac2-kube-api-access-m8n7c\") pod \"mt-curl-7850-658b77\" (UID: \"cde7e13c-9780-4dc9-851c-decf990ddac2\") " pod="opendatahub/mt-curl-7850-658b77"
Sep 05 08:11:37.285135 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:37.285052    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7850-658b77"
Sep 05 08:11:37.555489 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:37.555457    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7850-658b77"]
Sep 05 08:11:37.556969 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:11:37.556944    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcde7e13c_9780_4dc9_851c_decf990ddac2.slice/crio-aa7aea2ece4b6e6b158485979b68cf97f449380b0998a92ece7cb4e2a97ee583 WatchSource:0}: Error finding container aa7aea2ece4b6e6b158485979b68cf97f449380b0998a92ece7cb4e2a97ee583: Status 404 returned error can't find the container with id aa7aea2ece4b6e6b158485979b68cf97f449380b0998a92ece7cb4e2a97ee583
Sep 05 08:11:37.639599 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:37.639572    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7850-7ed5b2"
Sep 05 08:11:37.639736 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:37.639610    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e40ea2b67143a32f30b4a99f8c43fc8765f3ebd48f056c74f512aa77b55c0ee0"
Sep 05 08:11:37.640901 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:37.640871    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7850-658b77" event={"ID":"cde7e13c-9780-4dc9-851c-decf990ddac2","Type":"ContainerStarted","Data":"aa7aea2ece4b6e6b158485979b68cf97f449380b0998a92ece7cb4e2a97ee583"}
Sep 05 08:11:37.650645 ip-10-0-133-14 kubenswrapper[2563]: E0905 08:11:37.650621    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"mt-curl-7850-7ed5b2\" is forbidden: User \"system:node:ip-10-0-133-14.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-14.ec2.internal' and this object" podUID="e313b300-4acb-452b-a280-981f1ae10366" pod="opendatahub/mt-curl-7850-7ed5b2"
Sep 05 08:11:37.728924 ip-10-0-133-14 kubenswrapper[2563]: E0905 08:11:37.728887    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"mt-curl-7850-7ed5b2\" is forbidden: User \"system:node:ip-10-0-133-14.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-14.ec2.internal' and this object" podUID="e313b300-4acb-452b-a280-981f1ae10366" pod="opendatahub/mt-curl-7850-7ed5b2"
Sep 05 08:11:37.731366 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:37.731345    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e313b300-4acb-452b-a280-981f1ae10366" path="/var/lib/kubelet/pods/e313b300-4acb-452b-a280-981f1ae10366/volumes"
Sep 05 08:11:38.648440 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:38.648401    2563 generic.go:356] "Generic (PLEG): container finished" podID="cde7e13c-9780-4dc9-851c-decf990ddac2" containerID="9f87c9e23371e68ef8213fc46ce35a99f29bd9a2f92be935a24b658a9977df36" exitCode=0
Sep 05 08:11:38.648970 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:38.648483    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7850-658b77" event={"ID":"cde7e13c-9780-4dc9-851c-decf990ddac2","Type":"ContainerDied","Data":"9f87c9e23371e68ef8213fc46ce35a99f29bd9a2f92be935a24b658a9977df36"}
Sep 05 08:11:39.784917 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:39.784888    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7850-658b77"
Sep 05 08:11:39.805894 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:39.805837    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_mt-curl-7850-658b77_cde7e13c-9780-4dc9-851c-decf990ddac2/mt-curl-7850-658b77/0.log"
Sep 05 08:11:39.828687 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:39.828656    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-7850-658b77"]
Sep 05 08:11:39.831435 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:39.831414    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-7850-658b77"]
Sep 05 08:11:39.890572 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:39.890540    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m8n7c\" (UniqueName: \"kubernetes.io/projected/cde7e13c-9780-4dc9-851c-decf990ddac2-kube-api-access-m8n7c\") pod \"cde7e13c-9780-4dc9-851c-decf990ddac2\" (UID: \"cde7e13c-9780-4dc9-851c-decf990ddac2\") "
Sep 05 08:11:39.892710 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:39.892680    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cde7e13c-9780-4dc9-851c-decf990ddac2-kube-api-access-m8n7c" (OuterVolumeSpecName: "kube-api-access-m8n7c") pod "cde7e13c-9780-4dc9-851c-decf990ddac2" (UID: "cde7e13c-9780-4dc9-851c-decf990ddac2"). InnerVolumeSpecName "kube-api-access-m8n7c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:11:39.992048 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:39.991962    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m8n7c\" (UniqueName: \"kubernetes.io/projected/cde7e13c-9780-4dc9-851c-decf990ddac2-kube-api-access-m8n7c\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:11:40.661522 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:40.661436    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7850-658b77"
Sep 05 08:11:40.661522 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:40.661479    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aa7aea2ece4b6e6b158485979b68cf97f449380b0998a92ece7cb4e2a97ee583"
Sep 05 08:11:40.663203 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:40.663176    2563 generic.go:356] "Generic (PLEG): container finished" podID="92dd3f54-113e-421f-b276-d51c16796c43" containerID="f116c75e4306d79d732636d6ec9fb646282e81202f5f242483b831ea7f2607b6" exitCode=0
Sep 05 08:11:40.663315 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:40.663239    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7" event={"ID":"92dd3f54-113e-421f-b276-d51c16796c43","Type":"ContainerDied","Data":"f116c75e4306d79d732636d6ec9fb646282e81202f5f242483b831ea7f2607b6"}
Sep 05 08:11:40.663658 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:40.663643    2563 scope.go:117] "RemoveContainer" containerID="f116c75e4306d79d732636d6ec9fb646282e81202f5f242483b831ea7f2607b6"
Sep 05 08:11:40.664896 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:40.664869    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 05 08:11:40.664974 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:40.664938    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 05 08:11:40.665012 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:40.664974    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 05 08:11:40.683774 ip-10-0-133-14 kubenswrapper[2563]: E0905 08:11:40.683740    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"mt-curl-7850-658b77\" is forbidden: User \"system:node:ip-10-0-133-14.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-14.ec2.internal' and this object" podUID="cde7e13c-9780-4dc9-851c-decf990ddac2" pod="opendatahub/mt-curl-7850-658b77"
Sep 05 08:11:41.501463 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:41.501418    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:11:41.670616 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:41.670582    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7" event={"ID":"92dd3f54-113e-421f-b276-d51c16796c43","Type":"ContainerStarted","Data":"29e0c6a753b615fb3ead833abf97bbd30b1a5193d45d92c5c7caa0635f19fccd"}
Sep 05 08:11:41.697436 ip-10-0-133-14 kubenswrapper[2563]: E0905 08:11:41.697393    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"mt-curl-7850-658b77\" is forbidden: User \"system:node:ip-10-0-133-14.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-14.ec2.internal' and this object" podUID="cde7e13c-9780-4dc9-851c-decf990ddac2" pod="opendatahub/mt-curl-7850-658b77"
Sep 05 08:11:41.733607 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:41.733565    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cde7e13c-9780-4dc9-851c-decf990ddac2" path="/var/lib/kubelet/pods/cde7e13c-9780-4dc9-851c-decf990ddac2/volumes"
Sep 05 08:11:42.190869 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:42.190658    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-8c2e20/sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg"]
Sep 05 08:11:42.209866 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:42.207348    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:11:42.330791 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:42.330752    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"
Sep 05 08:11:42.331107 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:42.330970    2563 patch_prober.go:28] interesting pod/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Sep 05 08:11:42.331107 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:42.331026    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7" podUID="92dd3f54-113e-421f-b276-d51c16796c43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Sep 05 08:11:43.218521 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:43.217067    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-5ef325/sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2"]
Sep 05 08:11:43.228281 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:43.228248    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:11:43.330168 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:43.330132    2563 patch_prober.go:28] interesting pod/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Sep 05 08:11:43.330342 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:43.330188    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7" podUID="92dd3f54-113e-421f-b276-d51c16796c43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Sep 05 08:11:43.914468 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:43.914428    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:11:44.331140 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:44.331103    2563 patch_prober.go:28] interesting pod/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Sep 05 08:11:44.331628 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:44.331179    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7" podUID="92dd3f54-113e-421f-b276-d51c16796c43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Sep 05 08:11:44.532905 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:44.532866    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:11:45.330120 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:45.330087    2563 patch_prober.go:28] interesting pod/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Sep 05 08:11:45.330329 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:45.330146    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7" podUID="92dd3f54-113e-421f-b276-d51c16796c43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Sep 05 08:11:46.330743 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:46.330706    2563 patch_prober.go:28] interesting pod/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Sep 05 08:11:46.331175 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:46.330769    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7" podUID="92dd3f54-113e-421f-b276-d51c16796c43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Sep 05 08:11:47.335762 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:47.335732    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"
Sep 05 08:11:47.336223 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:47.336073    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"
Sep 05 08:11:47.336779 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:47.336743    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"
Sep 05 08:11:47.691202 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:47.691117    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e171e4-7hcr8"]
Sep 05 08:11:47.691504 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:47.691491    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cde7e13c-9780-4dc9-851c-decf990ddac2" containerName="mt-curl-7850-658b77"
Sep 05 08:11:47.691557 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:47.691505    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cde7e13c-9780-4dc9-851c-decf990ddac2" containerName="mt-curl-7850-658b77"
Sep 05 08:11:47.691602 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:47.691564    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="cde7e13c-9780-4dc9-851c-decf990ddac2" containerName="mt-curl-7850-658b77"
Sep 05 08:11:47.694135 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:47.694119    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e171e4-7hcr8"
Sep 05 08:11:47.696218 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:47.696190    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 05 08:11:47.696218 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:47.696199    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mpmmp\""
Sep 05 08:11:47.700142 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:47.700118    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e171e4-7hcr8"]
Sep 05 08:11:47.773571 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:47.773535    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6dac6feb-d7ca-4148-b8b3-0147cda2ba53-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-e171e4-7hcr8\" (UID: \"6dac6feb-d7ca-4148-b8b3-0147cda2ba53\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e171e4-7hcr8"
Sep 05 08:11:47.825358 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:47.825314    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:11:47.874951 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:47.874914    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6dac6feb-d7ca-4148-b8b3-0147cda2ba53-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-e171e4-7hcr8\" (UID: \"6dac6feb-d7ca-4148-b8b3-0147cda2ba53\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e171e4-7hcr8"
Sep 05 08:11:47.875520 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:47.875496    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6dac6feb-d7ca-4148-b8b3-0147cda2ba53-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-e171e4-7hcr8\" (UID: \"6dac6feb-d7ca-4148-b8b3-0147cda2ba53\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e171e4-7hcr8"
Sep 05 08:11:48.006133 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:48.006056    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e171e4-7hcr8"
Sep 05 08:11:48.281263 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:48.281239    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e171e4-7hcr8"]
Sep 05 08:11:48.283259 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:11:48.283228    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dac6feb_d7ca_4148_b8b3_0147cda2ba53.slice/crio-6c6f4c54533329e2b512919bd21ab1481c3e8eb806f050d7771aa22770450541 WatchSource:0}: Error finding container 6c6f4c54533329e2b512919bd21ab1481c3e8eb806f050d7771aa22770450541: Status 404 returned error can't find the container with id 6c6f4c54533329e2b512919bd21ab1481c3e8eb806f050d7771aa22770450541
Sep 05 08:11:48.720816 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:48.720779    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e171e4-7hcr8" event={"ID":"6dac6feb-d7ca-4148-b8b3-0147cda2ba53","Type":"ContainerStarted","Data":"cae59ab90b0e9b9bc1e887a93a72be358576dda76360c50624e35f167ea972f3"}
Sep 05 08:11:48.721227 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:48.720836    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e171e4-7hcr8" event={"ID":"6dac6feb-d7ca-4148-b8b3-0147cda2ba53","Type":"ContainerStarted","Data":"6c6f4c54533329e2b512919bd21ab1481c3e8eb806f050d7771aa22770450541"}
Sep 05 08:11:48.735045 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:48.735001    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e171e4-7hcr8" podStartSLOduration=1.734985541 podStartE2EDuration="1.734985541s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:11:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:11:48.733264522 +0000 UTC m=+1411.503406869" watchObservedRunningTime="2026-09-05 08:11:48.734985541 +0000 UTC m=+1411.505127880"
Sep 05 08:11:49.727709 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:49.727675    2563 generic.go:356] "Generic (PLEG): container finished" podID="6dac6feb-d7ca-4148-b8b3-0147cda2ba53" containerID="cae59ab90b0e9b9bc1e887a93a72be358576dda76360c50624e35f167ea972f3" exitCode=0
Sep 05 08:11:49.730286 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:49.730263    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e171e4-7hcr8" event={"ID":"6dac6feb-d7ca-4148-b8b3-0147cda2ba53","Type":"ContainerDied","Data":"cae59ab90b0e9b9bc1e887a93a72be358576dda76360c50624e35f167ea972f3"}
Sep 05 08:11:50.860529 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:50.860504    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e171e4-7hcr8"
Sep 05 08:11:50.905748 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:50.905721    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6dac6feb-d7ca-4148-b8b3-0147cda2ba53-maas-api-service-ca\") pod \"6dac6feb-d7ca-4148-b8b3-0147cda2ba53\" (UID: \"6dac6feb-d7ca-4148-b8b3-0147cda2ba53\") "
Sep 05 08:11:50.906089 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:50.906065    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6dac6feb-d7ca-4148-b8b3-0147cda2ba53-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6dac6feb-d7ca-4148-b8b3-0147cda2ba53" (UID: "6dac6feb-d7ca-4148-b8b3-0147cda2ba53"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:11:51.007493 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:51.007416    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6dac6feb-d7ca-4148-b8b3-0147cda2ba53-maas-api-service-ca\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:11:51.738703 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:51.738674    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e171e4-7hcr8" event={"ID":"6dac6feb-d7ca-4148-b8b3-0147cda2ba53","Type":"ContainerDied","Data":"6c6f4c54533329e2b512919bd21ab1481c3e8eb806f050d7771aa22770450541"}
Sep 05 08:11:51.738824 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:51.738714    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e171e4-7hcr8"
Sep 05 08:11:51.738824 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:51.738719    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6c6f4c54533329e2b512919bd21ab1481c3e8eb806f050d7771aa22770450541"
Sep 05 08:11:54.391434 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:54.391401    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-8c2e20/rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm"]
Sep 05 08:11:54.391962 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:54.391770    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6dac6feb-d7ca-4148-b8b3-0147cda2ba53" containerName="revoke-keys"
Sep 05 08:11:54.391962 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:54.391782    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6dac6feb-d7ca-4148-b8b3-0147cda2ba53" containerName="revoke-keys"
Sep 05 08:11:54.391962 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:54.391864    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6dac6feb-d7ca-4148-b8b3-0147cda2ba53" containerName="revoke-keys"
Sep 05 08:11:54.394242 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:54.394221    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-8c2e20/rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm"
Sep 05 08:11:54.396295 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:54.396275    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-8c2e20\"/\"rate-test-model-8c2e20-kserve-self-signed-certs\""
Sep 05 08:11:54.403342 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:54.403318    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-8c2e20/rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm"]
Sep 05 08:11:54.421554 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:54.421527    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:11:54.438433 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:54.438404    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/463d2b7c-c704-4816-8043-08c4b8859236-tmp-dir\") pod \"rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm\" (UID: \"463d2b7c-c704-4816-8043-08c4b8859236\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm"
Sep 05 08:11:54.438563 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:54.438459    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/463d2b7c-c704-4816-8043-08c4b8859236-dshm\") pod \"rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm\" (UID: \"463d2b7c-c704-4816-8043-08c4b8859236\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm"
Sep 05 08:11:54.438563 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:54.438487    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/463d2b7c-c704-4816-8043-08c4b8859236-home\") pod \"rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm\" (UID: \"463d2b7c-c704-4816-8043-08c4b8859236\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm"
Sep 05 08:11:54.438563 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:54.438516    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xw2f\" (UniqueName: \"kubernetes.io/projected/463d2b7c-c704-4816-8043-08c4b8859236-kube-api-access-8xw2f\") pod \"rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm\" (UID: \"463d2b7c-c704-4816-8043-08c4b8859236\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm"
Sep 05 08:11:54.438563 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:54.438547    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/463d2b7c-c704-4816-8043-08c4b8859236-model-cache\") pod \"rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm\" (UID: \"463d2b7c-c704-4816-8043-08c4b8859236\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm"
Sep 05 08:11:54.438714 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:54.438613    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/463d2b7c-c704-4816-8043-08c4b8859236-tls-certs\") pod \"rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm\" (UID: \"463d2b7c-c704-4816-8043-08c4b8859236\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm"
Sep 05 08:11:54.540075 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:54.540037    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/463d2b7c-c704-4816-8043-08c4b8859236-model-cache\") pod \"rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm\" (UID: \"463d2b7c-c704-4816-8043-08c4b8859236\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm"
Sep 05 08:11:54.540258 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:54.540096    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/463d2b7c-c704-4816-8043-08c4b8859236-tls-certs\") pod \"rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm\" (UID: \"463d2b7c-c704-4816-8043-08c4b8859236\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm"
Sep 05 08:11:54.540258 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:54.540136    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/463d2b7c-c704-4816-8043-08c4b8859236-tmp-dir\") pod \"rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm\" (UID: \"463d2b7c-c704-4816-8043-08c4b8859236\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm"
Sep 05 08:11:54.540355 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:54.540283    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/463d2b7c-c704-4816-8043-08c4b8859236-dshm\") pod \"rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm\" (UID: \"463d2b7c-c704-4816-8043-08c4b8859236\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm"
Sep 05 08:11:54.540355 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:54.540338    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/463d2b7c-c704-4816-8043-08c4b8859236-home\") pod \"rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm\" (UID: \"463d2b7c-c704-4816-8043-08c4b8859236\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm"
Sep 05 08:11:54.540467 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:54.540402    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xw2f\" (UniqueName: \"kubernetes.io/projected/463d2b7c-c704-4816-8043-08c4b8859236-kube-api-access-8xw2f\") pod \"rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm\" (UID: \"463d2b7c-c704-4816-8043-08c4b8859236\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm"
Sep 05 08:11:54.540467 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:54.540427    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/463d2b7c-c704-4816-8043-08c4b8859236-model-cache\") pod \"rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm\" (UID: \"463d2b7c-c704-4816-8043-08c4b8859236\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm"
Sep 05 08:11:54.540576 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:54.540481    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/463d2b7c-c704-4816-8043-08c4b8859236-tmp-dir\") pod \"rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm\" (UID: \"463d2b7c-c704-4816-8043-08c4b8859236\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm"
Sep 05 08:11:54.540664 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:54.540641    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/463d2b7c-c704-4816-8043-08c4b8859236-home\") pod \"rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm\" (UID: \"463d2b7c-c704-4816-8043-08c4b8859236\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm"
Sep 05 08:11:54.542562 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:54.542543    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/463d2b7c-c704-4816-8043-08c4b8859236-dshm\") pod \"rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm\" (UID: \"463d2b7c-c704-4816-8043-08c4b8859236\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm"
Sep 05 08:11:54.542660 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:54.542644    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/463d2b7c-c704-4816-8043-08c4b8859236-tls-certs\") pod \"rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm\" (UID: \"463d2b7c-c704-4816-8043-08c4b8859236\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm"
Sep 05 08:11:54.548568 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:54.548541    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xw2f\" (UniqueName: \"kubernetes.io/projected/463d2b7c-c704-4816-8043-08c4b8859236-kube-api-access-8xw2f\") pod \"rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm\" (UID: \"463d2b7c-c704-4816-8043-08c4b8859236\") " pod="ai-tenant-e2e-shared-a-w3-8c2e20/rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm"
Sep 05 08:11:54.707321 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:54.707231    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-8c2e20/rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm"
Sep 05 08:11:54.977762 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:54.977737    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-8c2e20/rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm"]
Sep 05 08:11:54.979456 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:11:54.979428    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod463d2b7c_c704_4816_8043_08c4b8859236.slice/crio-8dccf122395a6e4af26011c2b4634bd5b63badcaf7a3841701212373437a6228 WatchSource:0}: Error finding container 8dccf122395a6e4af26011c2b4634bd5b63badcaf7a3841701212373437a6228: Status 404 returned error can't find the container with id 8dccf122395a6e4af26011c2b4634bd5b63badcaf7a3841701212373437a6228
Sep 05 08:11:55.762148 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:55.762107    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-8c2e20/rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm" event={"ID":"463d2b7c-c704-4816-8043-08c4b8859236","Type":"ContainerStarted","Data":"695a9acf9dd5b2cba08b452bbadd19ff58ac854b55d8e66a87789f404b31ae88"}
Sep 05 08:11:55.762571 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:55.762157    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-8c2e20/rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm" event={"ID":"463d2b7c-c704-4816-8043-08c4b8859236","Type":"ContainerStarted","Data":"8dccf122395a6e4af26011c2b4634bd5b63badcaf7a3841701212373437a6228"}
Sep 05 08:11:55.780663 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:55.780620    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-8c2e20/rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm" podStartSLOduration=1.780605625 podStartE2EDuration="1.780605625s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:11:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:11:55.777980465 +0000 UTC m=+1418.548122803" watchObservedRunningTime="2026-09-05 08:11:55.780605625 +0000 UTC m=+1418.550747962"
Sep 05 08:11:57.234324 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:57.234284    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-8c2e20/sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg" podUID="23ff45e4-cfba-4b8f-a52f-3714dc19eed2" containerName="main" containerID="cri-o://264c2635e07b5145561c0b9c9a020d2622516f45e540697ec482aa067a61c0b8" gracePeriod=45
Sep 05 08:11:57.479979 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:57.479954    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-8c2e20/sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg"
Sep 05 08:11:57.568494 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:57.568458    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d72jv\" (UniqueName: \"kubernetes.io/projected/23ff45e4-cfba-4b8f-a52f-3714dc19eed2-kube-api-access-d72jv\") pod \"23ff45e4-cfba-4b8f-a52f-3714dc19eed2\" (UID: \"23ff45e4-cfba-4b8f-a52f-3714dc19eed2\") "
Sep 05 08:11:57.568687 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:57.568505    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23ff45e4-cfba-4b8f-a52f-3714dc19eed2-home\") pod \"23ff45e4-cfba-4b8f-a52f-3714dc19eed2\" (UID: \"23ff45e4-cfba-4b8f-a52f-3714dc19eed2\") "
Sep 05 08:11:57.568687 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:57.568532    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23ff45e4-cfba-4b8f-a52f-3714dc19eed2-tmp-dir\") pod \"23ff45e4-cfba-4b8f-a52f-3714dc19eed2\" (UID: \"23ff45e4-cfba-4b8f-a52f-3714dc19eed2\") "
Sep 05 08:11:57.568687 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:57.568581    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23ff45e4-cfba-4b8f-a52f-3714dc19eed2-dshm\") pod \"23ff45e4-cfba-4b8f-a52f-3714dc19eed2\" (UID: \"23ff45e4-cfba-4b8f-a52f-3714dc19eed2\") "
Sep 05 08:11:57.568687 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:57.568658    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23ff45e4-cfba-4b8f-a52f-3714dc19eed2-model-cache\") pod \"23ff45e4-cfba-4b8f-a52f-3714dc19eed2\" (UID: \"23ff45e4-cfba-4b8f-a52f-3714dc19eed2\") "
Sep 05 08:11:57.568687 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:57.568677    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23ff45e4-cfba-4b8f-a52f-3714dc19eed2-tls-certs\") pod \"23ff45e4-cfba-4b8f-a52f-3714dc19eed2\" (UID: \"23ff45e4-cfba-4b8f-a52f-3714dc19eed2\") "
Sep 05 08:11:57.568985 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:57.568820    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23ff45e4-cfba-4b8f-a52f-3714dc19eed2-home" (OuterVolumeSpecName: "home") pod "23ff45e4-cfba-4b8f-a52f-3714dc19eed2" (UID: "23ff45e4-cfba-4b8f-a52f-3714dc19eed2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:11:57.568985 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:57.568871    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23ff45e4-cfba-4b8f-a52f-3714dc19eed2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "23ff45e4-cfba-4b8f-a52f-3714dc19eed2" (UID: "23ff45e4-cfba-4b8f-a52f-3714dc19eed2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:11:57.569056 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:57.568987    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23ff45e4-cfba-4b8f-a52f-3714dc19eed2-home\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:11:57.569056 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:57.569000    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23ff45e4-cfba-4b8f-a52f-3714dc19eed2-tmp-dir\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:11:57.569056 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:57.569004    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23ff45e4-cfba-4b8f-a52f-3714dc19eed2-model-cache" (OuterVolumeSpecName: "model-cache") pod "23ff45e4-cfba-4b8f-a52f-3714dc19eed2" (UID: "23ff45e4-cfba-4b8f-a52f-3714dc19eed2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:11:57.570877 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:57.570826    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23ff45e4-cfba-4b8f-a52f-3714dc19eed2-kube-api-access-d72jv" (OuterVolumeSpecName: "kube-api-access-d72jv") pod "23ff45e4-cfba-4b8f-a52f-3714dc19eed2" (UID: "23ff45e4-cfba-4b8f-a52f-3714dc19eed2"). InnerVolumeSpecName "kube-api-access-d72jv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:11:57.570977 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:57.570958    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23ff45e4-cfba-4b8f-a52f-3714dc19eed2-dshm" (OuterVolumeSpecName: "dshm") pod "23ff45e4-cfba-4b8f-a52f-3714dc19eed2" (UID: "23ff45e4-cfba-4b8f-a52f-3714dc19eed2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:11:57.571026 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:57.570970    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23ff45e4-cfba-4b8f-a52f-3714dc19eed2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "23ff45e4-cfba-4b8f-a52f-3714dc19eed2" (UID: "23ff45e4-cfba-4b8f-a52f-3714dc19eed2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:11:57.669865 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:57.669808    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23ff45e4-cfba-4b8f-a52f-3714dc19eed2-model-cache\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:11:57.669865 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:57.669839    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23ff45e4-cfba-4b8f-a52f-3714dc19eed2-tls-certs\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:11:57.670040 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:57.669878    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d72jv\" (UniqueName: \"kubernetes.io/projected/23ff45e4-cfba-4b8f-a52f-3714dc19eed2-kube-api-access-d72jv\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:11:57.670040 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:57.669891    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23ff45e4-cfba-4b8f-a52f-3714dc19eed2-dshm\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:11:57.775589 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:57.775553    2563 generic.go:356] "Generic (PLEG): container finished" podID="23ff45e4-cfba-4b8f-a52f-3714dc19eed2" containerID="264c2635e07b5145561c0b9c9a020d2622516f45e540697ec482aa067a61c0b8" exitCode=0
Sep 05 08:11:57.775796 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:57.775620    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-8c2e20/sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg"
Sep 05 08:11:57.775796 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:57.775621    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-8c2e20/sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg" event={"ID":"23ff45e4-cfba-4b8f-a52f-3714dc19eed2","Type":"ContainerDied","Data":"264c2635e07b5145561c0b9c9a020d2622516f45e540697ec482aa067a61c0b8"}
Sep 05 08:11:57.775796 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:57.775736    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-8c2e20/sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg" event={"ID":"23ff45e4-cfba-4b8f-a52f-3714dc19eed2","Type":"ContainerDied","Data":"1aac26f490399d55f34e4b954997fda8e1b9ebef4e7b25ac70ddf21c5a6a0359"}
Sep 05 08:11:57.775796 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:57.775759    2563 scope.go:117] "RemoveContainer" containerID="264c2635e07b5145561c0b9c9a020d2622516f45e540697ec482aa067a61c0b8"
Sep 05 08:11:57.786122 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:57.786101    2563 scope.go:117] "RemoveContainer" containerID="264c2635e07b5145561c0b9c9a020d2622516f45e540697ec482aa067a61c0b8"
Sep 05 08:11:57.786435 ip-10-0-133-14 kubenswrapper[2563]: E0905 08:11:57.786415    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"264c2635e07b5145561c0b9c9a020d2622516f45e540697ec482aa067a61c0b8\": container with ID starting with 264c2635e07b5145561c0b9c9a020d2622516f45e540697ec482aa067a61c0b8 not found: ID does not exist" containerID="264c2635e07b5145561c0b9c9a020d2622516f45e540697ec482aa067a61c0b8"
Sep 05 08:11:57.786483 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:57.786453    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"264c2635e07b5145561c0b9c9a020d2622516f45e540697ec482aa067a61c0b8"} err="failed to get container status \"264c2635e07b5145561c0b9c9a020d2622516f45e540697ec482aa067a61c0b8\": rpc error: code = NotFound desc = could not find container \"264c2635e07b5145561c0b9c9a020d2622516f45e540697ec482aa067a61c0b8\": container with ID starting with 264c2635e07b5145561c0b9c9a020d2622516f45e540697ec482aa067a61c0b8 not found: ID does not exist"
Sep 05 08:11:57.790463 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:57.790440    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-8c2e20/sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg"]
Sep 05 08:11:57.797105 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:57.797081    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-8c2e20/sub-test-model-8c2e20-kserve-676f7c4c47-9kmgg"]
Sep 05 08:11:58.244226 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:58.244164    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-5ef325/sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2" podUID="abd861b5-990a-4757-94db-516c25121b68" containerName="main" containerID="cri-o://0049086dca666a931b9c03e0dad7d38f6e247b4f0f76f45c49669cb395c76275" gracePeriod=45
Sep 05 08:11:58.512437 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:58.512373    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-5ef325/sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2"
Sep 05 08:11:58.577806 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:58.577779    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abd861b5-990a-4757-94db-516c25121b68-tls-certs\") pod \"abd861b5-990a-4757-94db-516c25121b68\" (UID: \"abd861b5-990a-4757-94db-516c25121b68\") "
Sep 05 08:11:58.578000 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:58.577894    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9lfmf\" (UniqueName: \"kubernetes.io/projected/abd861b5-990a-4757-94db-516c25121b68-kube-api-access-9lfmf\") pod \"abd861b5-990a-4757-94db-516c25121b68\" (UID: \"abd861b5-990a-4757-94db-516c25121b68\") "
Sep 05 08:11:58.578000 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:58.577920    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abd861b5-990a-4757-94db-516c25121b68-tmp-dir\") pod \"abd861b5-990a-4757-94db-516c25121b68\" (UID: \"abd861b5-990a-4757-94db-516c25121b68\") "
Sep 05 08:11:58.578000 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:58.577950    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abd861b5-990a-4757-94db-516c25121b68-home\") pod \"abd861b5-990a-4757-94db-516c25121b68\" (UID: \"abd861b5-990a-4757-94db-516c25121b68\") "
Sep 05 08:11:58.578000 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:58.577994    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abd861b5-990a-4757-94db-516c25121b68-model-cache\") pod \"abd861b5-990a-4757-94db-516c25121b68\" (UID: \"abd861b5-990a-4757-94db-516c25121b68\") "
Sep 05 08:11:58.578298 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:58.578023    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abd861b5-990a-4757-94db-516c25121b68-dshm\") pod \"abd861b5-990a-4757-94db-516c25121b68\" (UID: \"abd861b5-990a-4757-94db-516c25121b68\") "
Sep 05 08:11:58.578298 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:58.578230    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abd861b5-990a-4757-94db-516c25121b68-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "abd861b5-990a-4757-94db-516c25121b68" (UID: "abd861b5-990a-4757-94db-516c25121b68"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:11:58.578298 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:58.578222    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abd861b5-990a-4757-94db-516c25121b68-home" (OuterVolumeSpecName: "home") pod "abd861b5-990a-4757-94db-516c25121b68" (UID: "abd861b5-990a-4757-94db-516c25121b68"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:11:58.578448 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:58.578290    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abd861b5-990a-4757-94db-516c25121b68-model-cache" (OuterVolumeSpecName: "model-cache") pod "abd861b5-990a-4757-94db-516c25121b68" (UID: "abd861b5-990a-4757-94db-516c25121b68"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:11:58.578501 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:58.578451    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abd861b5-990a-4757-94db-516c25121b68-tmp-dir\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:11:58.578501 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:58.578467    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abd861b5-990a-4757-94db-516c25121b68-home\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:11:58.578501 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:58.578476    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abd861b5-990a-4757-94db-516c25121b68-model-cache\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:11:58.580171 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:58.580147    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abd861b5-990a-4757-94db-516c25121b68-dshm" (OuterVolumeSpecName: "dshm") pod "abd861b5-990a-4757-94db-516c25121b68" (UID: "abd861b5-990a-4757-94db-516c25121b68"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:11:58.580171 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:58.580163    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abd861b5-990a-4757-94db-516c25121b68-kube-api-access-9lfmf" (OuterVolumeSpecName: "kube-api-access-9lfmf") pod "abd861b5-990a-4757-94db-516c25121b68" (UID: "abd861b5-990a-4757-94db-516c25121b68"). InnerVolumeSpecName "kube-api-access-9lfmf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:11:58.580323 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:58.580238    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abd861b5-990a-4757-94db-516c25121b68-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "abd861b5-990a-4757-94db-516c25121b68" (UID: "abd861b5-990a-4757-94db-516c25121b68"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:11:58.679013 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:58.678977    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abd861b5-990a-4757-94db-516c25121b68-tls-certs\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:11:58.679013 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:58.679007    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lfmf\" (UniqueName: \"kubernetes.io/projected/abd861b5-990a-4757-94db-516c25121b68-kube-api-access-9lfmf\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:11:58.679013 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:58.679015    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abd861b5-990a-4757-94db-516c25121b68-dshm\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:11:58.783371 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:58.783286    2563 generic.go:356] "Generic (PLEG): container finished" podID="abd861b5-990a-4757-94db-516c25121b68" containerID="0049086dca666a931b9c03e0dad7d38f6e247b4f0f76f45c49669cb395c76275" exitCode=0
Sep 05 08:11:58.783371 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:58.783336    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-5ef325/sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2"
Sep 05 08:11:58.783566 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:58.783377    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-5ef325/sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2" event={"ID":"abd861b5-990a-4757-94db-516c25121b68","Type":"ContainerDied","Data":"0049086dca666a931b9c03e0dad7d38f6e247b4f0f76f45c49669cb395c76275"}
Sep 05 08:11:58.783566 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:58.783420    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-5ef325/sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2" event={"ID":"abd861b5-990a-4757-94db-516c25121b68","Type":"ContainerDied","Data":"6253145462a019de2c6b1209963454ac2516f0cf8b63e0630583fc9af86a20fd"}
Sep 05 08:11:58.783566 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:58.783438    2563 scope.go:117] "RemoveContainer" containerID="0049086dca666a931b9c03e0dad7d38f6e247b4f0f76f45c49669cb395c76275"
Sep 05 08:11:58.792858 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:58.792825    2563 scope.go:117] "RemoveContainer" containerID="0049086dca666a931b9c03e0dad7d38f6e247b4f0f76f45c49669cb395c76275"
Sep 05 08:11:58.793124 ip-10-0-133-14 kubenswrapper[2563]: E0905 08:11:58.793105    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0049086dca666a931b9c03e0dad7d38f6e247b4f0f76f45c49669cb395c76275\": container with ID starting with 0049086dca666a931b9c03e0dad7d38f6e247b4f0f76f45c49669cb395c76275 not found: ID does not exist" containerID="0049086dca666a931b9c03e0dad7d38f6e247b4f0f76f45c49669cb395c76275"
Sep 05 08:11:58.793171 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:58.793132    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0049086dca666a931b9c03e0dad7d38f6e247b4f0f76f45c49669cb395c76275"} err="failed to get container status \"0049086dca666a931b9c03e0dad7d38f6e247b4f0f76f45c49669cb395c76275\": rpc error: code = NotFound desc = could not find container \"0049086dca666a931b9c03e0dad7d38f6e247b4f0f76f45c49669cb395c76275\": container with ID starting with 0049086dca666a931b9c03e0dad7d38f6e247b4f0f76f45c49669cb395c76275 not found: ID does not exist"
Sep 05 08:11:58.803586 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:58.803566    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-5ef325/sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2"]
Sep 05 08:11:58.807724 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:58.807702    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-5ef325/sub-test-model-5ef325-kserve-7f98bd6b8c-9f9v2"]
Sep 05 08:11:59.730619 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:59.730575    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23ff45e4-cfba-4b8f-a52f-3714dc19eed2" path="/var/lib/kubelet/pods/23ff45e4-cfba-4b8f-a52f-3714dc19eed2/volumes"
Sep 05 08:11:59.731205 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:11:59.731186    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abd861b5-990a-4757-94db-516c25121b68" path="/var/lib/kubelet/pods/abd861b5-990a-4757-94db-516c25121b68/volumes"
Sep 05 08:12:03.167819 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:03.167787    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:12:03.204823 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:03.204788    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-e171e4-8dd49c65d-srsv2"]
Sep 05 08:12:03.205093 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:03.205063    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-e171e4-8dd49c65d-srsv2" podUID="42e6fe07-85c0-4203-8cfc-2648e7af301e" containerName="payload-pre-processing" containerID="cri-o://5881c2eb2d212bfb16b560ea35ef4658d072aa7c94ae46237c6364f34d95b441" gracePeriod=30
Sep 05 08:12:03.267415 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:03.267388    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:12:03.368272 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:03.368239    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:12:03.379795 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:03.379744    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e171e4-7hcr8"]
Sep 05 08:12:03.381758 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:03.381737    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e171e4-7hcr8"]
Sep 05 08:12:03.420504 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:03.420430    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:12:03.469804 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:03.469778    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-e171e4-8dd49c65d-srsv2"
Sep 05 08:12:03.625949 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:03.625922    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42e6fe07-85c0-4203-8cfc-2648e7af301e-plugins-config-volume\") pod \"42e6fe07-85c0-4203-8cfc-2648e7af301e\" (UID: \"42e6fe07-85c0-4203-8cfc-2648e7af301e\") "
Sep 05 08:12:03.626118 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:03.626053    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rsmk5\" (UniqueName: \"kubernetes.io/projected/42e6fe07-85c0-4203-8cfc-2648e7af301e-kube-api-access-rsmk5\") pod \"42e6fe07-85c0-4203-8cfc-2648e7af301e\" (UID: \"42e6fe07-85c0-4203-8cfc-2648e7af301e\") "
Sep 05 08:12:03.626278 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:03.626256    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42e6fe07-85c0-4203-8cfc-2648e7af301e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "42e6fe07-85c0-4203-8cfc-2648e7af301e" (UID: "42e6fe07-85c0-4203-8cfc-2648e7af301e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:12:03.628295 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:03.628263    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42e6fe07-85c0-4203-8cfc-2648e7af301e-kube-api-access-rsmk5" (OuterVolumeSpecName: "kube-api-access-rsmk5") pod "42e6fe07-85c0-4203-8cfc-2648e7af301e" (UID: "42e6fe07-85c0-4203-8cfc-2648e7af301e"). InnerVolumeSpecName "kube-api-access-rsmk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:12:03.727360 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:03.727279    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rsmk5\" (UniqueName: \"kubernetes.io/projected/42e6fe07-85c0-4203-8cfc-2648e7af301e-kube-api-access-rsmk5\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:12:03.727360 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:03.727302    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42e6fe07-85c0-4203-8cfc-2648e7af301e-plugins-config-volume\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:12:03.729875 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:03.729809    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6dac6feb-d7ca-4148-b8b3-0147cda2ba53" path="/var/lib/kubelet/pods/6dac6feb-d7ca-4148-b8b3-0147cda2ba53/volumes"
Sep 05 08:12:03.791413 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:03.791386    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:12:03.812120 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:03.812090    2563 generic.go:356] "Generic (PLEG): container finished" podID="42e6fe07-85c0-4203-8cfc-2648e7af301e" containerID="5881c2eb2d212bfb16b560ea35ef4658d072aa7c94ae46237c6364f34d95b441" exitCode=0
Sep 05 08:12:03.812233 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:03.812130    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-e171e4-8dd49c65d-srsv2"
Sep 05 08:12:03.812233 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:03.812150    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-e171e4-8dd49c65d-srsv2" event={"ID":"42e6fe07-85c0-4203-8cfc-2648e7af301e","Type":"ContainerDied","Data":"5881c2eb2d212bfb16b560ea35ef4658d072aa7c94ae46237c6364f34d95b441"}
Sep 05 08:12:03.812233 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:03.812199    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-e171e4-8dd49c65d-srsv2" event={"ID":"42e6fe07-85c0-4203-8cfc-2648e7af301e","Type":"ContainerDied","Data":"d208d02117721ec98231e1c534316a9650b41f59436ff13807f4dc823bdc6a3c"}
Sep 05 08:12:03.812233 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:03.812224    2563 scope.go:117] "RemoveContainer" containerID="5881c2eb2d212bfb16b560ea35ef4658d072aa7c94ae46237c6364f34d95b441"
Sep 05 08:12:03.821031 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:03.821013    2563 scope.go:117] "RemoveContainer" containerID="5881c2eb2d212bfb16b560ea35ef4658d072aa7c94ae46237c6364f34d95b441"
Sep 05 08:12:03.821313 ip-10-0-133-14 kubenswrapper[2563]: E0905 08:12:03.821293    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5881c2eb2d212bfb16b560ea35ef4658d072aa7c94ae46237c6364f34d95b441\": container with ID starting with 5881c2eb2d212bfb16b560ea35ef4658d072aa7c94ae46237c6364f34d95b441 not found: ID does not exist" containerID="5881c2eb2d212bfb16b560ea35ef4658d072aa7c94ae46237c6364f34d95b441"
Sep 05 08:12:03.821362 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:03.821321    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5881c2eb2d212bfb16b560ea35ef4658d072aa7c94ae46237c6364f34d95b441"} err="failed to get container status \"5881c2eb2d212bfb16b560ea35ef4658d072aa7c94ae46237c6364f34d95b441\": rpc error: code = NotFound desc = could not find container \"5881c2eb2d212bfb16b560ea35ef4658d072aa7c94ae46237c6364f34d95b441\": container with ID starting with 5881c2eb2d212bfb16b560ea35ef4658d072aa7c94ae46237c6364f34d95b441 not found: ID does not exist"
Sep 05 08:12:03.830365 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:03.830344    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-e171e4-8dd49c65d-srsv2"]
Sep 05 08:12:03.834486 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:03.834458    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-e171e4-8dd49c65d-srsv2"]
Sep 05 08:12:03.904544 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:03.904507    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:12:04.707477 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:04.707432    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-8c2e20/rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm"
Sep 05 08:12:04.707905 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:04.707693    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-8c2e20/rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm"
Sep 05 08:12:04.720257 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:04.720227    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-8c2e20/rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm"
Sep 05 08:12:04.830577 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:04.830546    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-8c2e20/rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm"
Sep 05 08:12:05.730254 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:05.730209    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42e6fe07-85c0-4203-8cfc-2648e7af301e" path="/var/lib/kubelet/pods/42e6fe07-85c0-4203-8cfc-2648e7af301e/volumes"
Sep 05 08:12:09.500050 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:09.499965    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:12:09.601199 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:09.601169    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:12:09.766255 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:09.766170    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"]
Sep 05 08:12:09.766406 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:09.766358    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8" podUID="d5f08494-b053-4b39-ba19-317ec4148ffa" containerName="istio-proxy" containerID="cri-o://4b6ac0edb0455eb816a4385cf8376514acf9c0882d80e31654bfc381bc1051d3" gracePeriod=30
Sep 05 08:12:09.791123 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:09.791095    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:12:10.160226 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:10.160188    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d850e3-b7rxb"]
Sep 05 08:12:10.160586 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:10.160573    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abd861b5-990a-4757-94db-516c25121b68" containerName="main"
Sep 05 08:12:10.160634 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:10.160587    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="abd861b5-990a-4757-94db-516c25121b68" containerName="main"
Sep 05 08:12:10.160634 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:10.160608    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23ff45e4-cfba-4b8f-a52f-3714dc19eed2" containerName="main"
Sep 05 08:12:10.160634 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:10.160613    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="23ff45e4-cfba-4b8f-a52f-3714dc19eed2" containerName="main"
Sep 05 08:12:10.160634 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:10.160625    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42e6fe07-85c0-4203-8cfc-2648e7af301e" containerName="payload-pre-processing"
Sep 05 08:12:10.160634 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:10.160630    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="42e6fe07-85c0-4203-8cfc-2648e7af301e" containerName="payload-pre-processing"
Sep 05 08:12:10.160790 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:10.160689    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="23ff45e4-cfba-4b8f-a52f-3714dc19eed2" containerName="main"
Sep 05 08:12:10.160790 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:10.160700    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="42e6fe07-85c0-4203-8cfc-2648e7af301e" containerName="payload-pre-processing"
Sep 05 08:12:10.160790 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:10.160707    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="abd861b5-990a-4757-94db-516c25121b68" containerName="main"
Sep 05 08:12:10.163557 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:10.163541    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d850e3-b7rxb"
Sep 05 08:12:10.165514 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:10.165492    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mpmmp\""
Sep 05 08:12:10.165601 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:10.165543    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 05 08:12:10.170237 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:10.170212    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d850e3-b7rxb"]
Sep 05 08:12:10.273726 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:10.273693    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:12:10.297022 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:10.296994    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/320ce8dd-8e38-4d28-82cc-b4e0e721c3aa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-d850e3-b7rxb\" (UID: \"320ce8dd-8e38-4d28-82cc-b4e0e721c3aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d850e3-b7rxb"
Sep 05 08:12:10.398518 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:10.398488    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/320ce8dd-8e38-4d28-82cc-b4e0e721c3aa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-d850e3-b7rxb\" (UID: \"320ce8dd-8e38-4d28-82cc-b4e0e721c3aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d850e3-b7rxb"
Sep 05 08:12:10.399145 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:10.399125    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/320ce8dd-8e38-4d28-82cc-b4e0e721c3aa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-d850e3-b7rxb\" (UID: \"320ce8dd-8e38-4d28-82cc-b4e0e721c3aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d850e3-b7rxb"
Sep 05 08:12:10.475745 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:10.475667    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d850e3-b7rxb"
Sep 05 08:12:10.750210 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:10.750182    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d850e3-b7rxb"]
Sep 05 08:12:10.753185 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:12:10.753150    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod320ce8dd_8e38_4d28_82cc_b4e0e721c3aa.slice/crio-80d9e95f1763ba9089327f222f8c21b389c0136a290d2da8a65ef33b70832194 WatchSource:0}: Error finding container 80d9e95f1763ba9089327f222f8c21b389c0136a290d2da8a65ef33b70832194: Status 404 returned error can't find the container with id 80d9e95f1763ba9089327f222f8c21b389c0136a290d2da8a65ef33b70832194
Sep 05 08:12:10.852467 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:10.852428    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d850e3-b7rxb" event={"ID":"320ce8dd-8e38-4d28-82cc-b4e0e721c3aa","Type":"ContainerStarted","Data":"80d9e95f1763ba9089327f222f8c21b389c0136a290d2da8a65ef33b70832194"}
Sep 05 08:12:11.858923 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:11.858881    2563 generic.go:356] "Generic (PLEG): container finished" podID="320ce8dd-8e38-4d28-82cc-b4e0e721c3aa" containerID="a0c40aa01ad2dede63b5dd1a13d90d53edf30c0bf460e8ef22bb635df07f2d67" exitCode=0
Sep 05 08:12:11.859312 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:11.858928    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d850e3-b7rxb" event={"ID":"320ce8dd-8e38-4d28-82cc-b4e0e721c3aa","Type":"ContainerDied","Data":"a0c40aa01ad2dede63b5dd1a13d90d53edf30c0bf460e8ef22bb635df07f2d67"}
Sep 05 08:12:11.916270 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:11.916239    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:12:12.992573 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:12.992545    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d850e3-b7rxb"
Sep 05 08:12:13.123099 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:13.123023    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/320ce8dd-8e38-4d28-82cc-b4e0e721c3aa-maas-api-service-ca\") pod \"320ce8dd-8e38-4d28-82cc-b4e0e721c3aa\" (UID: \"320ce8dd-8e38-4d28-82cc-b4e0e721c3aa\") "
Sep 05 08:12:13.123408 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:13.123389    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/320ce8dd-8e38-4d28-82cc-b4e0e721c3aa-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "320ce8dd-8e38-4d28-82cc-b4e0e721c3aa" (UID: "320ce8dd-8e38-4d28-82cc-b4e0e721c3aa"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:12:13.224128 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:13.224078    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/320ce8dd-8e38-4d28-82cc-b4e0e721c3aa-maas-api-service-ca\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:12:13.871824 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:13.871792    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d850e3-b7rxb"
Sep 05 08:12:13.872015 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:13.871820    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d850e3-b7rxb" event={"ID":"320ce8dd-8e38-4d28-82cc-b4e0e721c3aa","Type":"ContainerDied","Data":"80d9e95f1763ba9089327f222f8c21b389c0136a290d2da8a65ef33b70832194"}
Sep 05 08:12:13.872015 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:13.871884    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="80d9e95f1763ba9089327f222f8c21b389c0136a290d2da8a65ef33b70832194"
Sep 05 08:12:15.019272 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:15.019247    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"
Sep 05 08:12:15.147911 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:15.147874    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5f08494-b053-4b39-ba19-317ec4148ffa-istio-data\") pod \"d5f08494-b053-4b39-ba19-317ec4148ffa\" (UID: \"d5f08494-b053-4b39-ba19-317ec4148ffa\") "
Sep 05 08:12:15.148086 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:15.147936    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5f08494-b053-4b39-ba19-317ec4148ffa-istio-podinfo\") pod \"d5f08494-b053-4b39-ba19-317ec4148ffa\" (UID: \"d5f08494-b053-4b39-ba19-317ec4148ffa\") "
Sep 05 08:12:15.148086 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:15.147961    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5f08494-b053-4b39-ba19-317ec4148ffa-istio-token\") pod \"d5f08494-b053-4b39-ba19-317ec4148ffa\" (UID: \"d5f08494-b053-4b39-ba19-317ec4148ffa\") "
Sep 05 08:12:15.148086 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:15.148013    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5f08494-b053-4b39-ba19-317ec4148ffa-istio-envoy\") pod \"d5f08494-b053-4b39-ba19-317ec4148ffa\" (UID: \"d5f08494-b053-4b39-ba19-317ec4148ffa\") "
Sep 05 08:12:15.148219 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:15.148110    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5f08494-b053-4b39-ba19-317ec4148ffa-istio-data" (OuterVolumeSpecName: "istio-data") pod "d5f08494-b053-4b39-ba19-317ec4148ffa" (UID: "d5f08494-b053-4b39-ba19-317ec4148ffa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:12:15.148219 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:15.148148    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5f08494-b053-4b39-ba19-317ec4148ffa-istiod-ca-cert\") pod \"d5f08494-b053-4b39-ba19-317ec4148ffa\" (UID: \"d5f08494-b053-4b39-ba19-317ec4148ffa\") "
Sep 05 08:12:15.148219 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:15.148193    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5f08494-b053-4b39-ba19-317ec4148ffa-workload-socket\") pod \"d5f08494-b053-4b39-ba19-317ec4148ffa\" (UID: \"d5f08494-b053-4b39-ba19-317ec4148ffa\") "
Sep 05 08:12:15.148339 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:15.148224    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-szj9m\" (UniqueName: \"kubernetes.io/projected/d5f08494-b053-4b39-ba19-317ec4148ffa-kube-api-access-szj9m\") pod \"d5f08494-b053-4b39-ba19-317ec4148ffa\" (UID: \"d5f08494-b053-4b39-ba19-317ec4148ffa\") "
Sep 05 08:12:15.148339 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:15.148332    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5f08494-b053-4b39-ba19-317ec4148ffa-workload-certs\") pod \"d5f08494-b053-4b39-ba19-317ec4148ffa\" (UID: \"d5f08494-b053-4b39-ba19-317ec4148ffa\") "
Sep 05 08:12:15.148454 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:15.148369    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5f08494-b053-4b39-ba19-317ec4148ffa-credential-socket\") pod \"d5f08494-b053-4b39-ba19-317ec4148ffa\" (UID: \"d5f08494-b053-4b39-ba19-317ec4148ffa\") "
Sep 05 08:12:15.148454 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:15.148442    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5f08494-b053-4b39-ba19-317ec4148ffa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d5f08494-b053-4b39-ba19-317ec4148ffa" (UID: "d5f08494-b053-4b39-ba19-317ec4148ffa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:12:15.148564 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:15.148516    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5f08494-b053-4b39-ba19-317ec4148ffa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d5f08494-b053-4b39-ba19-317ec4148ffa" (UID: "d5f08494-b053-4b39-ba19-317ec4148ffa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:12:15.148688 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:15.148632    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5f08494-b053-4b39-ba19-317ec4148ffa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d5f08494-b053-4b39-ba19-317ec4148ffa" (UID: "d5f08494-b053-4b39-ba19-317ec4148ffa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:12:15.148818 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:15.148731    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5f08494-b053-4b39-ba19-317ec4148ffa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d5f08494-b053-4b39-ba19-317ec4148ffa" (UID: "d5f08494-b053-4b39-ba19-317ec4148ffa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:12:15.149050 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:15.149025    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5f08494-b053-4b39-ba19-317ec4148ffa-istiod-ca-cert\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:12:15.149162 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:15.149053    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5f08494-b053-4b39-ba19-317ec4148ffa-workload-socket\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:12:15.149162 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:15.149068    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5f08494-b053-4b39-ba19-317ec4148ffa-workload-certs\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:12:15.149162 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:15.149083    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5f08494-b053-4b39-ba19-317ec4148ffa-credential-socket\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:12:15.149162 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:15.149099    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5f08494-b053-4b39-ba19-317ec4148ffa-istio-data\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:12:15.150473 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:15.150447    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d5f08494-b053-4b39-ba19-317ec4148ffa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d5f08494-b053-4b39-ba19-317ec4148ffa" (UID: "d5f08494-b053-4b39-ba19-317ec4148ffa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 05 08:12:15.150554 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:15.150514    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5f08494-b053-4b39-ba19-317ec4148ffa-istio-token" (OuterVolumeSpecName: "istio-token") pod "d5f08494-b053-4b39-ba19-317ec4148ffa" (UID: "d5f08494-b053-4b39-ba19-317ec4148ffa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:12:15.150554 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:15.150516    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5f08494-b053-4b39-ba19-317ec4148ffa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d5f08494-b053-4b39-ba19-317ec4148ffa" (UID: "d5f08494-b053-4b39-ba19-317ec4148ffa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:12:15.150635 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:15.150614    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5f08494-b053-4b39-ba19-317ec4148ffa-kube-api-access-szj9m" (OuterVolumeSpecName: "kube-api-access-szj9m") pod "d5f08494-b053-4b39-ba19-317ec4148ffa" (UID: "d5f08494-b053-4b39-ba19-317ec4148ffa"). InnerVolumeSpecName "kube-api-access-szj9m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:12:15.250602 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:15.250571    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5f08494-b053-4b39-ba19-317ec4148ffa-istio-envoy\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:12:15.250602 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:15.250596    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szj9m\" (UniqueName: \"kubernetes.io/projected/d5f08494-b053-4b39-ba19-317ec4148ffa-kube-api-access-szj9m\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:12:15.250602 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:15.250606    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5f08494-b053-4b39-ba19-317ec4148ffa-istio-podinfo\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:12:15.250820 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:15.250614    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5f08494-b053-4b39-ba19-317ec4148ffa-istio-token\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:12:15.882932 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:15.882902    2563 generic.go:356] "Generic (PLEG): container finished" podID="d5f08494-b053-4b39-ba19-317ec4148ffa" containerID="4b6ac0edb0455eb816a4385cf8376514acf9c0882d80e31654bfc381bc1051d3" exitCode=0
Sep 05 08:12:15.883115 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:15.882954    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"
Sep 05 08:12:15.883115 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:15.882995    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8" event={"ID":"d5f08494-b053-4b39-ba19-317ec4148ffa","Type":"ContainerDied","Data":"4b6ac0edb0455eb816a4385cf8376514acf9c0882d80e31654bfc381bc1051d3"}
Sep 05 08:12:15.883115 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:15.883054    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8" event={"ID":"d5f08494-b053-4b39-ba19-317ec4148ffa","Type":"ContainerDied","Data":"1ca096a79758fdf83e4d4653e6f100ae5a4945041d0602c7055c2ea1c8fd1b10"}
Sep 05 08:12:15.883115 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:15.883078    2563 scope.go:117] "RemoveContainer" containerID="4b6ac0edb0455eb816a4385cf8376514acf9c0882d80e31654bfc381bc1051d3"
Sep 05 08:12:15.892383 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:15.892357    2563 scope.go:117] "RemoveContainer" containerID="3410f9900af9992f126de30f6c9280c2243604a7005f7a3edc79a07001e70ee1"
Sep 05 08:12:15.900818 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:15.900802    2563 scope.go:117] "RemoveContainer" containerID="4b6ac0edb0455eb816a4385cf8376514acf9c0882d80e31654bfc381bc1051d3"
Sep 05 08:12:15.901108 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:15.901086    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"]
Sep 05 08:12:15.901228 ip-10-0-133-14 kubenswrapper[2563]: E0905 08:12:15.901210    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b6ac0edb0455eb816a4385cf8376514acf9c0882d80e31654bfc381bc1051d3\": container with ID starting with 4b6ac0edb0455eb816a4385cf8376514acf9c0882d80e31654bfc381bc1051d3 not found: ID does not exist" containerID="4b6ac0edb0455eb816a4385cf8376514acf9c0882d80e31654bfc381bc1051d3"
Sep 05 08:12:15.901276 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:15.901236    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b6ac0edb0455eb816a4385cf8376514acf9c0882d80e31654bfc381bc1051d3"} err="failed to get container status \"4b6ac0edb0455eb816a4385cf8376514acf9c0882d80e31654bfc381bc1051d3\": rpc error: code = NotFound desc = could not find container \"4b6ac0edb0455eb816a4385cf8376514acf9c0882d80e31654bfc381bc1051d3\": container with ID starting with 4b6ac0edb0455eb816a4385cf8376514acf9c0882d80e31654bfc381bc1051d3 not found: ID does not exist"
Sep 05 08:12:15.901276 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:15.901255    2563 scope.go:117] "RemoveContainer" containerID="3410f9900af9992f126de30f6c9280c2243604a7005f7a3edc79a07001e70ee1"
Sep 05 08:12:15.901495 ip-10-0-133-14 kubenswrapper[2563]: E0905 08:12:15.901479    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3410f9900af9992f126de30f6c9280c2243604a7005f7a3edc79a07001e70ee1\": container with ID starting with 3410f9900af9992f126de30f6c9280c2243604a7005f7a3edc79a07001e70ee1 not found: ID does not exist" containerID="3410f9900af9992f126de30f6c9280c2243604a7005f7a3edc79a07001e70ee1"
Sep 05 08:12:15.901539 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:15.901500    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3410f9900af9992f126de30f6c9280c2243604a7005f7a3edc79a07001e70ee1"} err="failed to get container status \"3410f9900af9992f126de30f6c9280c2243604a7005f7a3edc79a07001e70ee1\": rpc error: code = NotFound desc = could not find container \"3410f9900af9992f126de30f6c9280c2243604a7005f7a3edc79a07001e70ee1\": container with ID starting with 3410f9900af9992f126de30f6c9280c2243604a7005f7a3edc79a07001e70ee1 not found: ID does not exist"
Sep 05 08:12:15.904485 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:15.904464    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-e171e4-openshift-default-5df4c946b6-qp9f8"]
Sep 05 08:12:17.730587 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:17.730557    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5f08494-b053-4b39-ba19-317ec4148ffa" path="/var/lib/kubelet/pods/d5f08494-b053-4b39-ba19-317ec4148ffa/volumes"
Sep 05 08:12:17.925619 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:17.925590    2563 scope.go:117] "RemoveContainer" containerID="35ba458bdd625187cce661301efea1a58605304d4e738511da6ed739cb05b4c5"
Sep 05 08:12:17.934048 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:17.934027    2563 scope.go:117] "RemoveContainer" containerID="dd6efa31d71c508869b8b906395b4f963f7a9ee29c8b26448363ebd7a87b8dcb"
Sep 05 08:12:17.941692 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:17.941676    2563 scope.go:117] "RemoveContainer" containerID="da649e8c566dd6fc2bc47b41c540751a6cb3a91781e96bdd85c76a875338cc74"
Sep 05 08:12:17.951051 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:17.951032    2563 scope.go:117] "RemoveContainer" containerID="c9354057df982967ed478495bbeb87084bd6a419daf1744deff4b2a24458a628"
Sep 05 08:12:20.295734 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:20.295700    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-5ef325/rate-test-model-5ef325-kserve-f9465c7d7-45h7q"]
Sep 05 08:12:20.296250 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:20.296099    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="320ce8dd-8e38-4d28-82cc-b4e0e721c3aa" containerName="revoke-keys"
Sep 05 08:12:20.296250 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:20.296112    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="320ce8dd-8e38-4d28-82cc-b4e0e721c3aa" containerName="revoke-keys"
Sep 05 08:12:20.296250 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:20.296135    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5f08494-b053-4b39-ba19-317ec4148ffa" containerName="istio-proxy"
Sep 05 08:12:20.296250 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:20.296142    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5f08494-b053-4b39-ba19-317ec4148ffa" containerName="istio-proxy"
Sep 05 08:12:20.296250 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:20.296157    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5f08494-b053-4b39-ba19-317ec4148ffa" containerName="istio-proxy"
Sep 05 08:12:20.296250 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:20.296163    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5f08494-b053-4b39-ba19-317ec4148ffa" containerName="istio-proxy"
Sep 05 08:12:20.296250 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:20.296237    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5f08494-b053-4b39-ba19-317ec4148ffa" containerName="istio-proxy"
Sep 05 08:12:20.296250 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:20.296244    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5f08494-b053-4b39-ba19-317ec4148ffa" containerName="istio-proxy"
Sep 05 08:12:20.296250 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:20.296251    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="320ce8dd-8e38-4d28-82cc-b4e0e721c3aa" containerName="revoke-keys"
Sep 05 08:12:20.299658 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:20.299637    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-5ef325/rate-test-model-5ef325-kserve-f9465c7d7-45h7q"
Sep 05 08:12:20.302634 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:20.302606    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-5ef325\"/\"default-dockercfg-fqjc2\""
Sep 05 08:12:20.302634 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:20.302627    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-5ef325\"/\"kube-root-ca.crt\""
Sep 05 08:12:20.302817 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:20.302642    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-5ef325\"/\"openshift-service-ca.crt\""
Sep 05 08:12:20.302962 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:20.302934    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-5ef325\"/\"rate-test-model-5ef325-kserve-self-signed-certs\""
Sep 05 08:12:20.309932 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:20.309909    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-5ef325/rate-test-model-5ef325-kserve-f9465c7d7-45h7q"]
Sep 05 08:12:20.331458 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:20.331432    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:12:20.390772 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:20.390741    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmk8m\" (UniqueName: \"kubernetes.io/projected/df5324d5-c28f-4f43-8875-343bd2d822c2-kube-api-access-fmk8m\") pod \"rate-test-model-5ef325-kserve-f9465c7d7-45h7q\" (UID: \"df5324d5-c28f-4f43-8875-343bd2d822c2\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/rate-test-model-5ef325-kserve-f9465c7d7-45h7q"
Sep 05 08:12:20.391014 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:20.390891    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df5324d5-c28f-4f43-8875-343bd2d822c2-dshm\") pod \"rate-test-model-5ef325-kserve-f9465c7d7-45h7q\" (UID: \"df5324d5-c28f-4f43-8875-343bd2d822c2\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/rate-test-model-5ef325-kserve-f9465c7d7-45h7q"
Sep 05 08:12:20.391014 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:20.390983    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df5324d5-c28f-4f43-8875-343bd2d822c2-model-cache\") pod \"rate-test-model-5ef325-kserve-f9465c7d7-45h7q\" (UID: \"df5324d5-c28f-4f43-8875-343bd2d822c2\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/rate-test-model-5ef325-kserve-f9465c7d7-45h7q"
Sep 05 08:12:20.391153 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:20.391031    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df5324d5-c28f-4f43-8875-343bd2d822c2-tmp-dir\") pod \"rate-test-model-5ef325-kserve-f9465c7d7-45h7q\" (UID: \"df5324d5-c28f-4f43-8875-343bd2d822c2\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/rate-test-model-5ef325-kserve-f9465c7d7-45h7q"
Sep 05 08:12:20.391153 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:20.391071    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df5324d5-c28f-4f43-8875-343bd2d822c2-tls-certs\") pod \"rate-test-model-5ef325-kserve-f9465c7d7-45h7q\" (UID: \"df5324d5-c28f-4f43-8875-343bd2d822c2\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/rate-test-model-5ef325-kserve-f9465c7d7-45h7q"
Sep 05 08:12:20.391153 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:20.391119    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df5324d5-c28f-4f43-8875-343bd2d822c2-home\") pod \"rate-test-model-5ef325-kserve-f9465c7d7-45h7q\" (UID: \"df5324d5-c28f-4f43-8875-343bd2d822c2\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/rate-test-model-5ef325-kserve-f9465c7d7-45h7q"
Sep 05 08:12:20.425784 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:20.425758    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:12:20.492490 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:20.492450    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df5324d5-c28f-4f43-8875-343bd2d822c2-dshm\") pod \"rate-test-model-5ef325-kserve-f9465c7d7-45h7q\" (UID: \"df5324d5-c28f-4f43-8875-343bd2d822c2\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/rate-test-model-5ef325-kserve-f9465c7d7-45h7q"
Sep 05 08:12:20.492628 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:20.492530    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df5324d5-c28f-4f43-8875-343bd2d822c2-model-cache\") pod \"rate-test-model-5ef325-kserve-f9465c7d7-45h7q\" (UID: \"df5324d5-c28f-4f43-8875-343bd2d822c2\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/rate-test-model-5ef325-kserve-f9465c7d7-45h7q"
Sep 05 08:12:20.492628 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:20.492569    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df5324d5-c28f-4f43-8875-343bd2d822c2-tmp-dir\") pod \"rate-test-model-5ef325-kserve-f9465c7d7-45h7q\" (UID: \"df5324d5-c28f-4f43-8875-343bd2d822c2\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/rate-test-model-5ef325-kserve-f9465c7d7-45h7q"
Sep 05 08:12:20.492628 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:20.492603    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df5324d5-c28f-4f43-8875-343bd2d822c2-tls-certs\") pod \"rate-test-model-5ef325-kserve-f9465c7d7-45h7q\" (UID: \"df5324d5-c28f-4f43-8875-343bd2d822c2\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/rate-test-model-5ef325-kserve-f9465c7d7-45h7q"
Sep 05 08:12:20.492816 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:20.492647    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df5324d5-c28f-4f43-8875-343bd2d822c2-home\") pod \"rate-test-model-5ef325-kserve-f9465c7d7-45h7q\" (UID: \"df5324d5-c28f-4f43-8875-343bd2d822c2\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/rate-test-model-5ef325-kserve-f9465c7d7-45h7q"
Sep 05 08:12:20.492816 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:20.492699    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmk8m\" (UniqueName: \"kubernetes.io/projected/df5324d5-c28f-4f43-8875-343bd2d822c2-kube-api-access-fmk8m\") pod \"rate-test-model-5ef325-kserve-f9465c7d7-45h7q\" (UID: \"df5324d5-c28f-4f43-8875-343bd2d822c2\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/rate-test-model-5ef325-kserve-f9465c7d7-45h7q"
Sep 05 08:12:20.493022 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:20.492989    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df5324d5-c28f-4f43-8875-343bd2d822c2-model-cache\") pod \"rate-test-model-5ef325-kserve-f9465c7d7-45h7q\" (UID: \"df5324d5-c28f-4f43-8875-343bd2d822c2\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/rate-test-model-5ef325-kserve-f9465c7d7-45h7q"
Sep 05 08:12:20.493126 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:20.493042    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df5324d5-c28f-4f43-8875-343bd2d822c2-tmp-dir\") pod \"rate-test-model-5ef325-kserve-f9465c7d7-45h7q\" (UID: \"df5324d5-c28f-4f43-8875-343bd2d822c2\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/rate-test-model-5ef325-kserve-f9465c7d7-45h7q"
Sep 05 08:12:20.493126 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:20.493112    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df5324d5-c28f-4f43-8875-343bd2d822c2-home\") pod \"rate-test-model-5ef325-kserve-f9465c7d7-45h7q\" (UID: \"df5324d5-c28f-4f43-8875-343bd2d822c2\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/rate-test-model-5ef325-kserve-f9465c7d7-45h7q"
Sep 05 08:12:20.494781 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:20.494762    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df5324d5-c28f-4f43-8875-343bd2d822c2-dshm\") pod \"rate-test-model-5ef325-kserve-f9465c7d7-45h7q\" (UID: \"df5324d5-c28f-4f43-8875-343bd2d822c2\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/rate-test-model-5ef325-kserve-f9465c7d7-45h7q"
Sep 05 08:12:20.495132 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:20.495112    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df5324d5-c28f-4f43-8875-343bd2d822c2-tls-certs\") pod \"rate-test-model-5ef325-kserve-f9465c7d7-45h7q\" (UID: \"df5324d5-c28f-4f43-8875-343bd2d822c2\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/rate-test-model-5ef325-kserve-f9465c7d7-45h7q"
Sep 05 08:12:20.502796 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:20.502773    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmk8m\" (UniqueName: \"kubernetes.io/projected/df5324d5-c28f-4f43-8875-343bd2d822c2-kube-api-access-fmk8m\") pod \"rate-test-model-5ef325-kserve-f9465c7d7-45h7q\" (UID: \"df5324d5-c28f-4f43-8875-343bd2d822c2\") " pod="ai-tenant-e2e-shared-b-w3-5ef325/rate-test-model-5ef325-kserve-f9465c7d7-45h7q"
Sep 05 08:12:20.611401 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:20.611371    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-5ef325/rate-test-model-5ef325-kserve-f9465c7d7-45h7q"
Sep 05 08:12:20.889292 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:20.889269    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-5ef325/rate-test-model-5ef325-kserve-f9465c7d7-45h7q"]
Sep 05 08:12:20.891180 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:12:20.891150    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddf5324d5_c28f_4f43_8875_343bd2d822c2.slice/crio-f57d7efcfd811430f4c0dc8f211628e8cf6e92784b786c2a7b141874e2378ebe WatchSource:0}: Error finding container f57d7efcfd811430f4c0dc8f211628e8cf6e92784b786c2a7b141874e2378ebe: Status 404 returned error can't find the container with id f57d7efcfd811430f4c0dc8f211628e8cf6e92784b786c2a7b141874e2378ebe
Sep 05 08:12:20.912375 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:20.912347    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-5ef325/rate-test-model-5ef325-kserve-f9465c7d7-45h7q" event={"ID":"df5324d5-c28f-4f43-8875-343bd2d822c2","Type":"ContainerStarted","Data":"f57d7efcfd811430f4c0dc8f211628e8cf6e92784b786c2a7b141874e2378ebe"}
Sep 05 08:12:21.918908 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:21.918875    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-5ef325/rate-test-model-5ef325-kserve-f9465c7d7-45h7q" event={"ID":"df5324d5-c28f-4f43-8875-343bd2d822c2","Type":"ContainerStarted","Data":"f058fbb6c4454f10b95327ffdac1e4b9c58a9c9867fd8989b4642b66296bb61d"}
Sep 05 08:12:21.936162 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:21.936099    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-5ef325/rate-test-model-5ef325-kserve-f9465c7d7-45h7q" podStartSLOduration=1.936085141 podStartE2EDuration="1.936085141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:12:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:12:21.934445962 +0000 UTC m=+1444.704588309" watchObservedRunningTime="2026-09-05 08:12:21.936085141 +0000 UTC m=+1444.706227478"
Sep 05 08:12:25.995364 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:25.995329    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:12:26.031806 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:26.031773    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d850e3-685859947d-v2jln"]
Sep 05 08:12:26.032038 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:26.032003    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d850e3-685859947d-v2jln" podUID="8c9a9942-d766-4ad2-9e3b-c9490319d305" containerName="payload-pre-processing" containerID="cri-o://3dc6742a9b88356d5108d1d51e2c509a0574c8fa9343d7d50e88768b52684398" gracePeriod=30
Sep 05 08:12:26.066264 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:26.066231    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:12:26.166817 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:26.166788    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:12:26.171535 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:26.171507    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d850e3-b7rxb"]
Sep 05 08:12:26.174887 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:26.174864    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d850e3-b7rxb"]
Sep 05 08:12:26.258037 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:26.256971    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:12:26.302997 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:26.302966    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d850e3-685859947d-v2jln"
Sep 05 08:12:26.354591 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:26.354558    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c9a9942-d766-4ad2-9e3b-c9490319d305-plugins-config-volume\") pod \"8c9a9942-d766-4ad2-9e3b-c9490319d305\" (UID: \"8c9a9942-d766-4ad2-9e3b-c9490319d305\") "
Sep 05 08:12:26.354760 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:26.354617    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7lgr7\" (UniqueName: \"kubernetes.io/projected/8c9a9942-d766-4ad2-9e3b-c9490319d305-kube-api-access-7lgr7\") pod \"8c9a9942-d766-4ad2-9e3b-c9490319d305\" (UID: \"8c9a9942-d766-4ad2-9e3b-c9490319d305\") "
Sep 05 08:12:26.355043 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:26.355010    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c9a9942-d766-4ad2-9e3b-c9490319d305-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8c9a9942-d766-4ad2-9e3b-c9490319d305" (UID: "8c9a9942-d766-4ad2-9e3b-c9490319d305"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:12:26.356876 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:26.356825    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c9a9942-d766-4ad2-9e3b-c9490319d305-kube-api-access-7lgr7" (OuterVolumeSpecName: "kube-api-access-7lgr7") pod "8c9a9942-d766-4ad2-9e3b-c9490319d305" (UID: "8c9a9942-d766-4ad2-9e3b-c9490319d305"). InnerVolumeSpecName "kube-api-access-7lgr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:12:26.456284 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:26.456247    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c9a9942-d766-4ad2-9e3b-c9490319d305-plugins-config-volume\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:12:26.456284 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:26.456277    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7lgr7\" (UniqueName: \"kubernetes.io/projected/8c9a9942-d766-4ad2-9e3b-c9490319d305-kube-api-access-7lgr7\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:12:26.595680 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:26.595640    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:12:26.946590 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:26.946499    2563 generic.go:356] "Generic (PLEG): container finished" podID="8c9a9942-d766-4ad2-9e3b-c9490319d305" containerID="3dc6742a9b88356d5108d1d51e2c509a0574c8fa9343d7d50e88768b52684398" exitCode=0
Sep 05 08:12:26.946590 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:26.946547    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d850e3-685859947d-v2jln" event={"ID":"8c9a9942-d766-4ad2-9e3b-c9490319d305","Type":"ContainerDied","Data":"3dc6742a9b88356d5108d1d51e2c509a0574c8fa9343d7d50e88768b52684398"}
Sep 05 08:12:26.946590 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:26.946584    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d850e3-685859947d-v2jln" event={"ID":"8c9a9942-d766-4ad2-9e3b-c9490319d305","Type":"ContainerDied","Data":"c9fed41b8e82e7c6ef337a85c8371dd02719d06e0c6c5bdbddfa3dbcf9d55f89"}
Sep 05 08:12:26.946825 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:26.946607    2563 scope.go:117] "RemoveContainer" containerID="3dc6742a9b88356d5108d1d51e2c509a0574c8fa9343d7d50e88768b52684398"
Sep 05 08:12:26.946825 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:26.946546    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d850e3-685859947d-v2jln"
Sep 05 08:12:26.955809 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:26.955789    2563 scope.go:117] "RemoveContainer" containerID="3dc6742a9b88356d5108d1d51e2c509a0574c8fa9343d7d50e88768b52684398"
Sep 05 08:12:26.956119 ip-10-0-133-14 kubenswrapper[2563]: E0905 08:12:26.956099    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3dc6742a9b88356d5108d1d51e2c509a0574c8fa9343d7d50e88768b52684398\": container with ID starting with 3dc6742a9b88356d5108d1d51e2c509a0574c8fa9343d7d50e88768b52684398 not found: ID does not exist" containerID="3dc6742a9b88356d5108d1d51e2c509a0574c8fa9343d7d50e88768b52684398"
Sep 05 08:12:26.956178 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:26.956127    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3dc6742a9b88356d5108d1d51e2c509a0574c8fa9343d7d50e88768b52684398"} err="failed to get container status \"3dc6742a9b88356d5108d1d51e2c509a0574c8fa9343d7d50e88768b52684398\": rpc error: code = NotFound desc = could not find container \"3dc6742a9b88356d5108d1d51e2c509a0574c8fa9343d7d50e88768b52684398\": container with ID starting with 3dc6742a9b88356d5108d1d51e2c509a0574c8fa9343d7d50e88768b52684398 not found: ID does not exist"
Sep 05 08:12:26.966791 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:26.966768    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d850e3-685859947d-v2jln"]
Sep 05 08:12:26.970087 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:26.970067    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d850e3-685859947d-v2jln"]
Sep 05 08:12:27.731528 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:27.731491    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="320ce8dd-8e38-4d28-82cc-b4e0e721c3aa" path="/var/lib/kubelet/pods/320ce8dd-8e38-4d28-82cc-b4e0e721c3aa/volumes"
Sep 05 08:12:27.732081 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:27.732060    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c9a9942-d766-4ad2-9e3b-c9490319d305" path="/var/lib/kubelet/pods/8c9a9942-d766-4ad2-9e3b-c9490319d305/volumes"
Sep 05 08:12:30.612436 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:30.612407    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-5ef325/rate-test-model-5ef325-kserve-f9465c7d7-45h7q"
Sep 05 08:12:30.612804 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:30.612449    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-5ef325/rate-test-model-5ef325-kserve-f9465c7d7-45h7q"
Sep 05 08:12:30.624945 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:30.624917    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-5ef325/rate-test-model-5ef325-kserve-f9465c7d7-45h7q"
Sep 05 08:12:30.978620 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:30.978521    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-5ef325/rate-test-model-5ef325-kserve-f9465c7d7-45h7q"
Sep 05 08:12:32.794762 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:32.794735    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:12:32.895488 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:32.895449    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:12:33.041749 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:33.041710    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"]
Sep 05 08:12:33.041996 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:33.041973    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7" podUID="92dd3f54-113e-421f-b276-d51c16796c43" containerName="istio-proxy" containerID="cri-o://29e0c6a753b615fb3ead833abf97bbd30b1a5193d45d92c5c7caa0635f19fccd" gracePeriod=30
Sep 05 08:12:33.065375 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:33.065347    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:12:33.404796 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:33.404715    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-8b6928-9vjn2"]
Sep 05 08:12:33.405104 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:33.405091    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c9a9942-d766-4ad2-9e3b-c9490319d305" containerName="payload-pre-processing"
Sep 05 08:12:33.405160 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:33.405106    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c9a9942-d766-4ad2-9e3b-c9490319d305" containerName="payload-pre-processing"
Sep 05 08:12:33.405196 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:33.405158    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c9a9942-d766-4ad2-9e3b-c9490319d305" containerName="payload-pre-processing"
Sep 05 08:12:33.408389 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:33.408371    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-8b6928-9vjn2"
Sep 05 08:12:33.410578 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:33.410557    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 05 08:12:33.410658 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:33.410614    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mpmmp\""
Sep 05 08:12:33.414150 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:33.414124    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-8b6928-9vjn2"]
Sep 05 08:12:33.511783 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:33.511752    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:12:33.527072 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:33.527046    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/59cf63d6-814d-4fe2-8cb7-ba13f6f3f339-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-8b6928-9vjn2\" (UID: \"59cf63d6-814d-4fe2-8cb7-ba13f6f3f339\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-8b6928-9vjn2"
Sep 05 08:12:33.628545 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:33.628516    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/59cf63d6-814d-4fe2-8cb7-ba13f6f3f339-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-8b6928-9vjn2\" (UID: \"59cf63d6-814d-4fe2-8cb7-ba13f6f3f339\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-8b6928-9vjn2"
Sep 05 08:12:33.629110 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:33.629091    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/59cf63d6-814d-4fe2-8cb7-ba13f6f3f339-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-8b6928-9vjn2\" (UID: \"59cf63d6-814d-4fe2-8cb7-ba13f6f3f339\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-8b6928-9vjn2"
Sep 05 08:12:33.719492 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:33.719408    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-8b6928-9vjn2"
Sep 05 08:12:33.995445 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:33.995421    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-8b6928-9vjn2"]
Sep 05 08:12:33.997422 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:12:33.997390    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59cf63d6_814d_4fe2_8cb7_ba13f6f3f339.slice/crio-d325797e9ed50cc6053c637073628de7bf28ef9781eed6c329c5239fd966e405 WatchSource:0}: Error finding container d325797e9ed50cc6053c637073628de7bf28ef9781eed6c329c5239fd966e405: Status 404 returned error can't find the container with id d325797e9ed50cc6053c637073628de7bf28ef9781eed6c329c5239fd966e405
Sep 05 08:12:34.989821 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:34.989734    2563 generic.go:356] "Generic (PLEG): container finished" podID="59cf63d6-814d-4fe2-8cb7-ba13f6f3f339" containerID="cf808fab0fdcfe306c1efe36e0e3acfdc5f2108cd1b50cbe9383fa882df6260f" exitCode=0
Sep 05 08:12:34.989821 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:34.989770    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-8b6928-9vjn2" event={"ID":"59cf63d6-814d-4fe2-8cb7-ba13f6f3f339","Type":"ContainerDied","Data":"cf808fab0fdcfe306c1efe36e0e3acfdc5f2108cd1b50cbe9383fa882df6260f"}
Sep 05 08:12:34.989821 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:34.989819    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-8b6928-9vjn2" event={"ID":"59cf63d6-814d-4fe2-8cb7-ba13f6f3f339","Type":"ContainerStarted","Data":"d325797e9ed50cc6053c637073628de7bf28ef9781eed6c329c5239fd966e405"}
Sep 05 08:12:36.127235 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:36.127213    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-8b6928-9vjn2"
Sep 05 08:12:36.255312 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:36.255229    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/59cf63d6-814d-4fe2-8cb7-ba13f6f3f339-maas-api-service-ca\") pod \"59cf63d6-814d-4fe2-8cb7-ba13f6f3f339\" (UID: \"59cf63d6-814d-4fe2-8cb7-ba13f6f3f339\") "
Sep 05 08:12:36.255575 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:36.255553    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59cf63d6-814d-4fe2-8cb7-ba13f6f3f339-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "59cf63d6-814d-4fe2-8cb7-ba13f6f3f339" (UID: "59cf63d6-814d-4fe2-8cb7-ba13f6f3f339"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:12:36.356787 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:36.356754    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/59cf63d6-814d-4fe2-8cb7-ba13f6f3f339-maas-api-service-ca\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:12:37.002148 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:37.002120    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-8b6928-9vjn2"
Sep 05 08:12:37.002324 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:37.002157    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-8b6928-9vjn2" event={"ID":"59cf63d6-814d-4fe2-8cb7-ba13f6f3f339","Type":"ContainerDied","Data":"d325797e9ed50cc6053c637073628de7bf28ef9781eed6c329c5239fd966e405"}
Sep 05 08:12:37.002324 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:37.002205    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d325797e9ed50cc6053c637073628de7bf28ef9781eed6c329c5239fd966e405"
Sep 05 08:12:38.295146 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:38.295124    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"
Sep 05 08:12:38.375491 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:38.375466    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92dd3f54-113e-421f-b276-d51c16796c43-istio-data\") pod \"92dd3f54-113e-421f-b276-d51c16796c43\" (UID: \"92dd3f54-113e-421f-b276-d51c16796c43\") "
Sep 05 08:12:38.375688 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:38.375528    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92dd3f54-113e-421f-b276-d51c16796c43-istiod-ca-cert\") pod \"92dd3f54-113e-421f-b276-d51c16796c43\" (UID: \"92dd3f54-113e-421f-b276-d51c16796c43\") "
Sep 05 08:12:38.375688 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:38.375545    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92dd3f54-113e-421f-b276-d51c16796c43-istio-token\") pod \"92dd3f54-113e-421f-b276-d51c16796c43\" (UID: \"92dd3f54-113e-421f-b276-d51c16796c43\") "
Sep 05 08:12:38.375688 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:38.375582    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-llqrm\" (UniqueName: \"kubernetes.io/projected/92dd3f54-113e-421f-b276-d51c16796c43-kube-api-access-llqrm\") pod \"92dd3f54-113e-421f-b276-d51c16796c43\" (UID: \"92dd3f54-113e-421f-b276-d51c16796c43\") "
Sep 05 08:12:38.375688 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:38.375608    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92dd3f54-113e-421f-b276-d51c16796c43-istio-podinfo\") pod \"92dd3f54-113e-421f-b276-d51c16796c43\" (UID: \"92dd3f54-113e-421f-b276-d51c16796c43\") "
Sep 05 08:12:38.375688 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:38.375637    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92dd3f54-113e-421f-b276-d51c16796c43-workload-certs\") pod \"92dd3f54-113e-421f-b276-d51c16796c43\" (UID: \"92dd3f54-113e-421f-b276-d51c16796c43\") "
Sep 05 08:12:38.375688 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:38.375669    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92dd3f54-113e-421f-b276-d51c16796c43-credential-socket\") pod \"92dd3f54-113e-421f-b276-d51c16796c43\" (UID: \"92dd3f54-113e-421f-b276-d51c16796c43\") "
Sep 05 08:12:38.376035 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:38.375698    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92dd3f54-113e-421f-b276-d51c16796c43-workload-socket\") pod \"92dd3f54-113e-421f-b276-d51c16796c43\" (UID: \"92dd3f54-113e-421f-b276-d51c16796c43\") "
Sep 05 08:12:38.376035 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:38.375737    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92dd3f54-113e-421f-b276-d51c16796c43-istio-envoy\") pod \"92dd3f54-113e-421f-b276-d51c16796c43\" (UID: \"92dd3f54-113e-421f-b276-d51c16796c43\") "
Sep 05 08:12:38.376035 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:38.375762    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92dd3f54-113e-421f-b276-d51c16796c43-istio-data" (OuterVolumeSpecName: "istio-data") pod "92dd3f54-113e-421f-b276-d51c16796c43" (UID: "92dd3f54-113e-421f-b276-d51c16796c43"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:12:38.376035 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:38.375958    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92dd3f54-113e-421f-b276-d51c16796c43-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "92dd3f54-113e-421f-b276-d51c16796c43" (UID: "92dd3f54-113e-421f-b276-d51c16796c43"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:12:38.376035 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:38.375982    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92dd3f54-113e-421f-b276-d51c16796c43-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "92dd3f54-113e-421f-b276-d51c16796c43" (UID: "92dd3f54-113e-421f-b276-d51c16796c43"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:12:38.376035 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:38.376003    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92dd3f54-113e-421f-b276-d51c16796c43-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "92dd3f54-113e-421f-b276-d51c16796c43" (UID: "92dd3f54-113e-421f-b276-d51c16796c43"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:12:38.376312 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:38.376108    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92dd3f54-113e-421f-b276-d51c16796c43-workload-certs\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:12:38.376312 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:38.376126    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92dd3f54-113e-421f-b276-d51c16796c43-credential-socket\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:12:38.376312 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:38.376193    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92dd3f54-113e-421f-b276-d51c16796c43-istio-data\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:12:38.376312 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:38.376208    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92dd3f54-113e-421f-b276-d51c16796c43-istiod-ca-cert\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:12:38.376312 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:38.376206    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92dd3f54-113e-421f-b276-d51c16796c43-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "92dd3f54-113e-421f-b276-d51c16796c43" (UID: "92dd3f54-113e-421f-b276-d51c16796c43"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:12:38.378076 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:38.378051    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92dd3f54-113e-421f-b276-d51c16796c43-kube-api-access-llqrm" (OuterVolumeSpecName: "kube-api-access-llqrm") pod "92dd3f54-113e-421f-b276-d51c16796c43" (UID: "92dd3f54-113e-421f-b276-d51c16796c43"). InnerVolumeSpecName "kube-api-access-llqrm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:12:38.378076 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:38.378055    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/92dd3f54-113e-421f-b276-d51c16796c43-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "92dd3f54-113e-421f-b276-d51c16796c43" (UID: "92dd3f54-113e-421f-b276-d51c16796c43"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 05 08:12:38.378450 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:38.378427    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92dd3f54-113e-421f-b276-d51c16796c43-istio-token" (OuterVolumeSpecName: "istio-token") pod "92dd3f54-113e-421f-b276-d51c16796c43" (UID: "92dd3f54-113e-421f-b276-d51c16796c43"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:12:38.378450 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:38.378436    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92dd3f54-113e-421f-b276-d51c16796c43-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "92dd3f54-113e-421f-b276-d51c16796c43" (UID: "92dd3f54-113e-421f-b276-d51c16796c43"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:12:38.477098 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:38.477071    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92dd3f54-113e-421f-b276-d51c16796c43-istio-podinfo\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:12:38.477098 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:38.477095    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92dd3f54-113e-421f-b276-d51c16796c43-workload-socket\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:12:38.477098 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:38.477104    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92dd3f54-113e-421f-b276-d51c16796c43-istio-envoy\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:12:38.477294 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:38.477112    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92dd3f54-113e-421f-b276-d51c16796c43-istio-token\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:12:38.477294 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:38.477120    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-llqrm\" (UniqueName: \"kubernetes.io/projected/92dd3f54-113e-421f-b276-d51c16796c43-kube-api-access-llqrm\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:12:39.014412 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:39.014371    2563 generic.go:356] "Generic (PLEG): container finished" podID="92dd3f54-113e-421f-b276-d51c16796c43" containerID="29e0c6a753b615fb3ead833abf97bbd30b1a5193d45d92c5c7caa0635f19fccd" exitCode=0
Sep 05 08:12:39.014619 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:39.014432    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"
Sep 05 08:12:39.014619 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:39.014463    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7" event={"ID":"92dd3f54-113e-421f-b276-d51c16796c43","Type":"ContainerDied","Data":"29e0c6a753b615fb3ead833abf97bbd30b1a5193d45d92c5c7caa0635f19fccd"}
Sep 05 08:12:39.014619 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:39.014531    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7" event={"ID":"92dd3f54-113e-421f-b276-d51c16796c43","Type":"ContainerDied","Data":"fb449e253cfc3ef7371aa48ef543b029fd5214cf70633277af6066151e51f24f"}
Sep 05 08:12:39.014619 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:39.014561    2563 scope.go:117] "RemoveContainer" containerID="29e0c6a753b615fb3ead833abf97bbd30b1a5193d45d92c5c7caa0635f19fccd"
Sep 05 08:12:39.026345 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:39.026325    2563 scope.go:117] "RemoveContainer" containerID="f116c75e4306d79d732636d6ec9fb646282e81202f5f242483b831ea7f2607b6"
Sep 05 08:12:39.034803 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:39.034784    2563 scope.go:117] "RemoveContainer" containerID="29e0c6a753b615fb3ead833abf97bbd30b1a5193d45d92c5c7caa0635f19fccd"
Sep 05 08:12:39.035081 ip-10-0-133-14 kubenswrapper[2563]: E0905 08:12:39.035058    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29e0c6a753b615fb3ead833abf97bbd30b1a5193d45d92c5c7caa0635f19fccd\": container with ID starting with 29e0c6a753b615fb3ead833abf97bbd30b1a5193d45d92c5c7caa0635f19fccd not found: ID does not exist" containerID="29e0c6a753b615fb3ead833abf97bbd30b1a5193d45d92c5c7caa0635f19fccd"
Sep 05 08:12:39.035157 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:39.035090    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29e0c6a753b615fb3ead833abf97bbd30b1a5193d45d92c5c7caa0635f19fccd"} err="failed to get container status \"29e0c6a753b615fb3ead833abf97bbd30b1a5193d45d92c5c7caa0635f19fccd\": rpc error: code = NotFound desc = could not find container \"29e0c6a753b615fb3ead833abf97bbd30b1a5193d45d92c5c7caa0635f19fccd\": container with ID starting with 29e0c6a753b615fb3ead833abf97bbd30b1a5193d45d92c5c7caa0635f19fccd not found: ID does not exist"
Sep 05 08:12:39.035157 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:39.035107    2563 scope.go:117] "RemoveContainer" containerID="f116c75e4306d79d732636d6ec9fb646282e81202f5f242483b831ea7f2607b6"
Sep 05 08:12:39.035376 ip-10-0-133-14 kubenswrapper[2563]: E0905 08:12:39.035358    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f116c75e4306d79d732636d6ec9fb646282e81202f5f242483b831ea7f2607b6\": container with ID starting with f116c75e4306d79d732636d6ec9fb646282e81202f5f242483b831ea7f2607b6 not found: ID does not exist" containerID="f116c75e4306d79d732636d6ec9fb646282e81202f5f242483b831ea7f2607b6"
Sep 05 08:12:39.035433 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:39.035382    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f116c75e4306d79d732636d6ec9fb646282e81202f5f242483b831ea7f2607b6"} err="failed to get container status \"f116c75e4306d79d732636d6ec9fb646282e81202f5f242483b831ea7f2607b6\": rpc error: code = NotFound desc = could not find container \"f116c75e4306d79d732636d6ec9fb646282e81202f5f242483b831ea7f2607b6\": container with ID starting with f116c75e4306d79d732636d6ec9fb646282e81202f5f242483b831ea7f2607b6 not found: ID does not exist"
Sep 05 08:12:39.036433 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:39.036409    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"]
Sep 05 08:12:39.039017 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:39.038998    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-d850e3-openshift-default-6489bbd5c6-dtxm7"]
Sep 05 08:12:39.730423 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:39.730389    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92dd3f54-113e-421f-b276-d51c16796c43" path="/var/lib/kubelet/pods/92dd3f54-113e-421f-b276-d51c16796c43/volumes"
Sep 05 08:12:43.848027 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:43.847990    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:12:43.919615 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:43.919582    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:12:44.018763 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:44.018730    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:12:44.022961 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:44.022930    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-8b6928-9vjn2"]
Sep 05 08:12:44.026014 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:44.025990    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-8b6928-9vjn2"]
Sep 05 08:12:44.081198 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:44.081167    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:12:44.451866 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:44.451812    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:12:45.730109 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:45.730068    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59cf63d6-814d-4fe2-8cb7-ba13f6f3f339" path="/var/lib/kubelet/pods/59cf63d6-814d-4fe2-8cb7-ba13f6f3f339/volumes"
Sep 05 08:12:50.247809 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:50.247778    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:12:50.347748 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:50.347719    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:12:50.509538 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:50.509445    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:12:50.842109 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:50.842074    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-cec446-2cmq5"]
Sep 05 08:12:50.842443 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:50.842430    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59cf63d6-814d-4fe2-8cb7-ba13f6f3f339" containerName="revoke-keys"
Sep 05 08:12:50.842496 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:50.842444    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="59cf63d6-814d-4fe2-8cb7-ba13f6f3f339" containerName="revoke-keys"
Sep 05 08:12:50.842496 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:50.842452    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92dd3f54-113e-421f-b276-d51c16796c43" containerName="istio-proxy"
Sep 05 08:12:50.842496 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:50.842457    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="92dd3f54-113e-421f-b276-d51c16796c43" containerName="istio-proxy"
Sep 05 08:12:50.842496 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:50.842474    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92dd3f54-113e-421f-b276-d51c16796c43" containerName="istio-proxy"
Sep 05 08:12:50.842496 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:50.842480    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="92dd3f54-113e-421f-b276-d51c16796c43" containerName="istio-proxy"
Sep 05 08:12:50.842654 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:50.842536    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="92dd3f54-113e-421f-b276-d51c16796c43" containerName="istio-proxy"
Sep 05 08:12:50.842654 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:50.842545    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="92dd3f54-113e-421f-b276-d51c16796c43" containerName="istio-proxy"
Sep 05 08:12:50.842654 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:50.842551    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="59cf63d6-814d-4fe2-8cb7-ba13f6f3f339" containerName="revoke-keys"
Sep 05 08:12:50.844577 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:50.844562    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-cec446-2cmq5"
Sep 05 08:12:50.847102 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:50.847057    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 05 08:12:50.847700 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:50.847648    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mpmmp\""
Sep 05 08:12:50.853269 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:50.853224    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-cec446-2cmq5"]
Sep 05 08:12:50.961696 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:50.961638    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:12:51.001155 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:51.001122    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e689b348-b5cd-4542-a773-6fe141e5e4de-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-cec446-2cmq5\" (UID: \"e689b348-b5cd-4542-a773-6fe141e5e4de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-cec446-2cmq5"
Sep 05 08:12:51.102064 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:51.101969    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e689b348-b5cd-4542-a773-6fe141e5e4de-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-cec446-2cmq5\" (UID: \"e689b348-b5cd-4542-a773-6fe141e5e4de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-cec446-2cmq5"
Sep 05 08:12:51.102625 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:51.102600    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e689b348-b5cd-4542-a773-6fe141e5e4de-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-cec446-2cmq5\" (UID: \"e689b348-b5cd-4542-a773-6fe141e5e4de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-cec446-2cmq5"
Sep 05 08:12:51.159822 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:51.159790    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-cec446-2cmq5"
Sep 05 08:12:51.429168 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:51.429088    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-cec446-2cmq5"]
Sep 05 08:12:51.432316 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:12:51.432290    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode689b348_b5cd_4542_a773_6fe141e5e4de.slice/crio-66be9a802f6378adbe0c5863bb1f5f5da1f21fecb227c9b96b8ffe2f251517ec WatchSource:0}: Error finding container 66be9a802f6378adbe0c5863bb1f5f5da1f21fecb227c9b96b8ffe2f251517ec: Status 404 returned error can't find the container with id 66be9a802f6378adbe0c5863bb1f5f5da1f21fecb227c9b96b8ffe2f251517ec
Sep 05 08:12:52.086580 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:52.086543    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-cec446-2cmq5" event={"ID":"e689b348-b5cd-4542-a773-6fe141e5e4de","Type":"ContainerStarted","Data":"d754a6269311700512b7987e20ba429a780bc7de98088b6c0959da8ffd8f962b"}
Sep 05 08:12:52.086734 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:52.086606    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-cec446-2cmq5" event={"ID":"e689b348-b5cd-4542-a773-6fe141e5e4de","Type":"ContainerStarted","Data":"66be9a802f6378adbe0c5863bb1f5f5da1f21fecb227c9b96b8ffe2f251517ec"}
Sep 05 08:12:52.101336 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:52.101294    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-cec446-2cmq5" podStartSLOduration=2.101277865 podStartE2EDuration="2.101277865s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:12:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:12:52.099371884 +0000 UTC m=+1474.869514223" watchObservedRunningTime="2026-09-05 08:12:52.101277865 +0000 UTC m=+1474.871420259"
Sep 05 08:12:53.093171 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:53.093135    2563 generic.go:356] "Generic (PLEG): container finished" podID="e689b348-b5cd-4542-a773-6fe141e5e4de" containerID="d754a6269311700512b7987e20ba429a780bc7de98088b6c0959da8ffd8f962b" exitCode=0
Sep 05 08:12:53.093571 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:53.093212    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-cec446-2cmq5" event={"ID":"e689b348-b5cd-4542-a773-6fe141e5e4de","Type":"ContainerDied","Data":"d754a6269311700512b7987e20ba429a780bc7de98088b6c0959da8ffd8f962b"}
Sep 05 08:12:54.229299 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:54.229274    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-cec446-2cmq5"
Sep 05 08:12:54.333096 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:54.333065    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e689b348-b5cd-4542-a773-6fe141e5e4de-maas-api-service-ca\") pod \"e689b348-b5cd-4542-a773-6fe141e5e4de\" (UID: \"e689b348-b5cd-4542-a773-6fe141e5e4de\") "
Sep 05 08:12:54.333420 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:54.333395    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e689b348-b5cd-4542-a773-6fe141e5e4de-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e689b348-b5cd-4542-a773-6fe141e5e4de" (UID: "e689b348-b5cd-4542-a773-6fe141e5e4de"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:12:54.434159 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:54.434073    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e689b348-b5cd-4542-a773-6fe141e5e4de-maas-api-service-ca\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:12:55.104298 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:55.104261    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-cec446-2cmq5" event={"ID":"e689b348-b5cd-4542-a773-6fe141e5e4de","Type":"ContainerDied","Data":"66be9a802f6378adbe0c5863bb1f5f5da1f21fecb227c9b96b8ffe2f251517ec"}
Sep 05 08:12:55.104298 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:55.104297    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-cec446-2cmq5"
Sep 05 08:12:55.104528 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:55.104322    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="66be9a802f6378adbe0c5863bb1f5f5da1f21fecb227c9b96b8ffe2f251517ec"
Sep 05 08:12:58.625738 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:12:58.625700    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:13:00.417002 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:00.416968    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:13:00.940613 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:00.940581    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:13:01.007364 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:01.007338    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:13:01.077435 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:01.077399    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:13:01.110285 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:01.110251    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-cec446-2cmq5"]
Sep 05 08:13:01.113362 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:01.113341    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-cec446-2cmq5"]
Sep 05 08:13:01.174551 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:01.174514    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:13:01.521660 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:01.521624    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:13:01.729624 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:01.729588    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e689b348-b5cd-4542-a773-6fe141e5e4de" path="/var/lib/kubelet/pods/e689b348-b5cd-4542-a773-6fe141e5e4de/volumes"
Sep 05 08:13:07.051550 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:07.051518    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:13:07.115497 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:07.115463    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:13:07.337956 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:07.337928    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:13:17.771937 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:17.771909    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2bzp8_569cffae-1c5c-4b02-af57-9ca53c455e4c/ovn-acl-logging/0.log"
Sep 05 08:13:17.774606 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:17.774586    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2bzp8_569cffae-1c5c-4b02-af57-9ca53c455e4c/ovn-acl-logging/0.log"
Sep 05 08:13:17.995944 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:17.995912    2563 scope.go:117] "RemoveContainer" containerID="ff7392003bc46a58db41c79e546fd2d3a8b9de7777947f10aa797847c5b8c04c"
Sep 05 08:13:18.004225 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:18.004204    2563 scope.go:117] "RemoveContainer" containerID="afb5359518b280975d7718c32d741c6d3b82fbfc281aa1be6e9f75f38ae80589"
Sep 05 08:13:18.275638 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:18.275557    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-8c2e20/rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm"]
Sep 05 08:13:19.009343 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:19.009311    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:13:19.283040 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:19.282957    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-5ef325/rate-test-model-5ef325-kserve-f9465c7d7-45h7q"]
Sep 05 08:13:19.302322 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:19.302292    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:13:19.765652 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:19.765617    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:13:20.089176 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:20.089145    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:13:20.205579 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:20.205549    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:13:22.835249 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:22.835191    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:13:22.935581 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:22.935554    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:13:24.005084 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:24.005051    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:13:24.181545 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:24.181515    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:13:24.407707 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:24.407677    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:13:24.509491 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:24.509449    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:13:24.604982 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:24.604929    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:13:33.295673 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:33.295632    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-8c2e20/rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm" podUID="463d2b7c-c704-4816-8043-08c4b8859236" containerName="main" containerID="cri-o://695a9acf9dd5b2cba08b452bbadd19ff58ac854b55d8e66a87789f404b31ae88" gracePeriod=45
Sep 05 08:13:33.549031 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:33.548975    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-8c2e20/rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm"
Sep 05 08:13:33.601058 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:33.601030    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8xw2f\" (UniqueName: \"kubernetes.io/projected/463d2b7c-c704-4816-8043-08c4b8859236-kube-api-access-8xw2f\") pod \"463d2b7c-c704-4816-8043-08c4b8859236\" (UID: \"463d2b7c-c704-4816-8043-08c4b8859236\") "
Sep 05 08:13:33.601209 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:33.601068    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/463d2b7c-c704-4816-8043-08c4b8859236-tls-certs\") pod \"463d2b7c-c704-4816-8043-08c4b8859236\" (UID: \"463d2b7c-c704-4816-8043-08c4b8859236\") "
Sep 05 08:13:33.601209 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:33.601088    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/463d2b7c-c704-4816-8043-08c4b8859236-dshm\") pod \"463d2b7c-c704-4816-8043-08c4b8859236\" (UID: \"463d2b7c-c704-4816-8043-08c4b8859236\") "
Sep 05 08:13:33.601209 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:33.601119    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/463d2b7c-c704-4816-8043-08c4b8859236-home\") pod \"463d2b7c-c704-4816-8043-08c4b8859236\" (UID: \"463d2b7c-c704-4816-8043-08c4b8859236\") "
Sep 05 08:13:33.601209 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:33.601163    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/463d2b7c-c704-4816-8043-08c4b8859236-model-cache\") pod \"463d2b7c-c704-4816-8043-08c4b8859236\" (UID: \"463d2b7c-c704-4816-8043-08c4b8859236\") "
Sep 05 08:13:33.601434 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:33.601284    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/463d2b7c-c704-4816-8043-08c4b8859236-tmp-dir\") pod \"463d2b7c-c704-4816-8043-08c4b8859236\" (UID: \"463d2b7c-c704-4816-8043-08c4b8859236\") "
Sep 05 08:13:33.601490 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:33.601428    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/463d2b7c-c704-4816-8043-08c4b8859236-home" (OuterVolumeSpecName: "home") pod "463d2b7c-c704-4816-8043-08c4b8859236" (UID: "463d2b7c-c704-4816-8043-08c4b8859236"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:13:33.601490 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:33.601467    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/463d2b7c-c704-4816-8043-08c4b8859236-model-cache" (OuterVolumeSpecName: "model-cache") pod "463d2b7c-c704-4816-8043-08c4b8859236" (UID: "463d2b7c-c704-4816-8043-08c4b8859236"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:13:33.601616 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:33.601589    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/463d2b7c-c704-4816-8043-08c4b8859236-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "463d2b7c-c704-4816-8043-08c4b8859236" (UID: "463d2b7c-c704-4816-8043-08c4b8859236"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:13:33.601682 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:33.601663    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/463d2b7c-c704-4816-8043-08c4b8859236-home\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:13:33.601731 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:33.601688    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/463d2b7c-c704-4816-8043-08c4b8859236-model-cache\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:13:33.603296 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:33.603274    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/463d2b7c-c704-4816-8043-08c4b8859236-dshm" (OuterVolumeSpecName: "dshm") pod "463d2b7c-c704-4816-8043-08c4b8859236" (UID: "463d2b7c-c704-4816-8043-08c4b8859236"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:13:33.603407 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:33.603276    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/463d2b7c-c704-4816-8043-08c4b8859236-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "463d2b7c-c704-4816-8043-08c4b8859236" (UID: "463d2b7c-c704-4816-8043-08c4b8859236"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:13:33.603407 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:33.603320    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/463d2b7c-c704-4816-8043-08c4b8859236-kube-api-access-8xw2f" (OuterVolumeSpecName: "kube-api-access-8xw2f") pod "463d2b7c-c704-4816-8043-08c4b8859236" (UID: "463d2b7c-c704-4816-8043-08c4b8859236"). InnerVolumeSpecName "kube-api-access-8xw2f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:13:33.702180 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:33.702151    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/463d2b7c-c704-4816-8043-08c4b8859236-tmp-dir\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:13:33.702180 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:33.702175    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xw2f\" (UniqueName: \"kubernetes.io/projected/463d2b7c-c704-4816-8043-08c4b8859236-kube-api-access-8xw2f\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:13:33.702180 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:33.702185    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/463d2b7c-c704-4816-8043-08c4b8859236-tls-certs\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:13:33.702385 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:33.702194    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/463d2b7c-c704-4816-8043-08c4b8859236-dshm\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:13:34.303116 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:34.303062    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-5ef325/rate-test-model-5ef325-kserve-f9465c7d7-45h7q" podUID="df5324d5-c28f-4f43-8875-343bd2d822c2" containerName="main" containerID="cri-o://f058fbb6c4454f10b95327ffdac1e4b9c58a9c9867fd8989b4642b66296bb61d" gracePeriod=45
Sep 05 08:13:34.304398 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:34.304374    2563 generic.go:356] "Generic (PLEG): container finished" podID="463d2b7c-c704-4816-8043-08c4b8859236" containerID="695a9acf9dd5b2cba08b452bbadd19ff58ac854b55d8e66a87789f404b31ae88" exitCode=0
Sep 05 08:13:34.304528 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:34.304452    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-8c2e20/rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm"
Sep 05 08:13:34.304528 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:34.304468    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-8c2e20/rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm" event={"ID":"463d2b7c-c704-4816-8043-08c4b8859236","Type":"ContainerDied","Data":"695a9acf9dd5b2cba08b452bbadd19ff58ac854b55d8e66a87789f404b31ae88"}
Sep 05 08:13:34.304636 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:34.304530    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-8c2e20/rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm" event={"ID":"463d2b7c-c704-4816-8043-08c4b8859236","Type":"ContainerDied","Data":"8dccf122395a6e4af26011c2b4634bd5b63badcaf7a3841701212373437a6228"}
Sep 05 08:13:34.304636 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:34.304557    2563 scope.go:117] "RemoveContainer" containerID="695a9acf9dd5b2cba08b452bbadd19ff58ac854b55d8e66a87789f404b31ae88"
Sep 05 08:13:34.313471 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:34.313449    2563 scope.go:117] "RemoveContainer" containerID="695a9acf9dd5b2cba08b452bbadd19ff58ac854b55d8e66a87789f404b31ae88"
Sep 05 08:13:34.313733 ip-10-0-133-14 kubenswrapper[2563]: E0905 08:13:34.313715    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"695a9acf9dd5b2cba08b452bbadd19ff58ac854b55d8e66a87789f404b31ae88\": container with ID starting with 695a9acf9dd5b2cba08b452bbadd19ff58ac854b55d8e66a87789f404b31ae88 not found: ID does not exist" containerID="695a9acf9dd5b2cba08b452bbadd19ff58ac854b55d8e66a87789f404b31ae88"
Sep 05 08:13:34.313793 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:34.313742    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"695a9acf9dd5b2cba08b452bbadd19ff58ac854b55d8e66a87789f404b31ae88"} err="failed to get container status \"695a9acf9dd5b2cba08b452bbadd19ff58ac854b55d8e66a87789f404b31ae88\": rpc error: code = NotFound desc = could not find container \"695a9acf9dd5b2cba08b452bbadd19ff58ac854b55d8e66a87789f404b31ae88\": container with ID starting with 695a9acf9dd5b2cba08b452bbadd19ff58ac854b55d8e66a87789f404b31ae88 not found: ID does not exist"
Sep 05 08:13:34.319905 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:34.319882    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-8c2e20/rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm"]
Sep 05 08:13:34.323097 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:34.323071    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-8c2e20/rate-test-model-8c2e20-kserve-77d4d8b5f5-rgxcm"]
Sep 05 08:13:34.555994 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:34.555975    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-5ef325/rate-test-model-5ef325-kserve-f9465c7d7-45h7q"
Sep 05 08:13:34.607579 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:34.607544    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df5324d5-c28f-4f43-8875-343bd2d822c2-dshm\") pod \"df5324d5-c28f-4f43-8875-343bd2d822c2\" (UID: \"df5324d5-c28f-4f43-8875-343bd2d822c2\") "
Sep 05 08:13:34.607745 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:34.607591    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df5324d5-c28f-4f43-8875-343bd2d822c2-model-cache\") pod \"df5324d5-c28f-4f43-8875-343bd2d822c2\" (UID: \"df5324d5-c28f-4f43-8875-343bd2d822c2\") "
Sep 05 08:13:34.607745 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:34.607637    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df5324d5-c28f-4f43-8875-343bd2d822c2-tls-certs\") pod \"df5324d5-c28f-4f43-8875-343bd2d822c2\" (UID: \"df5324d5-c28f-4f43-8875-343bd2d822c2\") "
Sep 05 08:13:34.607745 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:34.607697    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fmk8m\" (UniqueName: \"kubernetes.io/projected/df5324d5-c28f-4f43-8875-343bd2d822c2-kube-api-access-fmk8m\") pod \"df5324d5-c28f-4f43-8875-343bd2d822c2\" (UID: \"df5324d5-c28f-4f43-8875-343bd2d822c2\") "
Sep 05 08:13:34.607745 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:34.607719    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df5324d5-c28f-4f43-8875-343bd2d822c2-home\") pod \"df5324d5-c28f-4f43-8875-343bd2d822c2\" (UID: \"df5324d5-c28f-4f43-8875-343bd2d822c2\") "
Sep 05 08:13:34.607745 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:34.607736    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df5324d5-c28f-4f43-8875-343bd2d822c2-tmp-dir\") pod \"df5324d5-c28f-4f43-8875-343bd2d822c2\" (UID: \"df5324d5-c28f-4f43-8875-343bd2d822c2\") "
Sep 05 08:13:34.608056 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:34.607911    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df5324d5-c28f-4f43-8875-343bd2d822c2-model-cache" (OuterVolumeSpecName: "model-cache") pod "df5324d5-c28f-4f43-8875-343bd2d822c2" (UID: "df5324d5-c28f-4f43-8875-343bd2d822c2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:13:34.608056 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:34.607976    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df5324d5-c28f-4f43-8875-343bd2d822c2-home" (OuterVolumeSpecName: "home") pod "df5324d5-c28f-4f43-8875-343bd2d822c2" (UID: "df5324d5-c28f-4f43-8875-343bd2d822c2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:13:34.608056 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:34.608025    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df5324d5-c28f-4f43-8875-343bd2d822c2-model-cache\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:13:34.608213 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:34.608057    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df5324d5-c28f-4f43-8875-343bd2d822c2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "df5324d5-c28f-4f43-8875-343bd2d822c2" (UID: "df5324d5-c28f-4f43-8875-343bd2d822c2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:13:34.609834 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:34.609809    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df5324d5-c28f-4f43-8875-343bd2d822c2-dshm" (OuterVolumeSpecName: "dshm") pod "df5324d5-c28f-4f43-8875-343bd2d822c2" (UID: "df5324d5-c28f-4f43-8875-343bd2d822c2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:13:34.609944 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:34.609911    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df5324d5-c28f-4f43-8875-343bd2d822c2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "df5324d5-c28f-4f43-8875-343bd2d822c2" (UID: "df5324d5-c28f-4f43-8875-343bd2d822c2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:13:34.609980 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:34.609934    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df5324d5-c28f-4f43-8875-343bd2d822c2-kube-api-access-fmk8m" (OuterVolumeSpecName: "kube-api-access-fmk8m") pod "df5324d5-c28f-4f43-8875-343bd2d822c2" (UID: "df5324d5-c28f-4f43-8875-343bd2d822c2"). InnerVolumeSpecName "kube-api-access-fmk8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:13:34.709037 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:34.709002    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmk8m\" (UniqueName: \"kubernetes.io/projected/df5324d5-c28f-4f43-8875-343bd2d822c2-kube-api-access-fmk8m\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:13:34.709037 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:34.709030    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df5324d5-c28f-4f43-8875-343bd2d822c2-home\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:13:34.709037 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:34.709039    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df5324d5-c28f-4f43-8875-343bd2d822c2-tmp-dir\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:13:34.709037 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:34.709047    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df5324d5-c28f-4f43-8875-343bd2d822c2-dshm\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:13:34.709293 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:34.709054    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df5324d5-c28f-4f43-8875-343bd2d822c2-tls-certs\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:13:35.312903 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:35.312869    2563 generic.go:356] "Generic (PLEG): container finished" podID="df5324d5-c28f-4f43-8875-343bd2d822c2" containerID="f058fbb6c4454f10b95327ffdac1e4b9c58a9c9867fd8989b4642b66296bb61d" exitCode=0
Sep 05 08:13:35.312903 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:35.312910    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-5ef325/rate-test-model-5ef325-kserve-f9465c7d7-45h7q" event={"ID":"df5324d5-c28f-4f43-8875-343bd2d822c2","Type":"ContainerDied","Data":"f058fbb6c4454f10b95327ffdac1e4b9c58a9c9867fd8989b4642b66296bb61d"}
Sep 05 08:13:35.313427 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:35.312922    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-5ef325/rate-test-model-5ef325-kserve-f9465c7d7-45h7q"
Sep 05 08:13:35.313427 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:35.312937    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-5ef325/rate-test-model-5ef325-kserve-f9465c7d7-45h7q" event={"ID":"df5324d5-c28f-4f43-8875-343bd2d822c2","Type":"ContainerDied","Data":"f57d7efcfd811430f4c0dc8f211628e8cf6e92784b786c2a7b141874e2378ebe"}
Sep 05 08:13:35.313427 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:35.312967    2563 scope.go:117] "RemoveContainer" containerID="f058fbb6c4454f10b95327ffdac1e4b9c58a9c9867fd8989b4642b66296bb61d"
Sep 05 08:13:35.323415 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:35.323395    2563 scope.go:117] "RemoveContainer" containerID="f058fbb6c4454f10b95327ffdac1e4b9c58a9c9867fd8989b4642b66296bb61d"
Sep 05 08:13:35.323715 ip-10-0-133-14 kubenswrapper[2563]: E0905 08:13:35.323697    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f058fbb6c4454f10b95327ffdac1e4b9c58a9c9867fd8989b4642b66296bb61d\": container with ID starting with f058fbb6c4454f10b95327ffdac1e4b9c58a9c9867fd8989b4642b66296bb61d not found: ID does not exist" containerID="f058fbb6c4454f10b95327ffdac1e4b9c58a9c9867fd8989b4642b66296bb61d"
Sep 05 08:13:35.323790 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:35.323724    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f058fbb6c4454f10b95327ffdac1e4b9c58a9c9867fd8989b4642b66296bb61d"} err="failed to get container status \"f058fbb6c4454f10b95327ffdac1e4b9c58a9c9867fd8989b4642b66296bb61d\": rpc error: code = NotFound desc = could not find container \"f058fbb6c4454f10b95327ffdac1e4b9c58a9c9867fd8989b4642b66296bb61d\": container with ID starting with f058fbb6c4454f10b95327ffdac1e4b9c58a9c9867fd8989b4642b66296bb61d not found: ID does not exist"
Sep 05 08:13:35.332105 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:35.332080    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-5ef325/rate-test-model-5ef325-kserve-f9465c7d7-45h7q"]
Sep 05 08:13:35.335696 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:35.335677    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-5ef325/rate-test-model-5ef325-kserve-f9465c7d7-45h7q"]
Sep 05 08:13:35.730382 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:35.730305    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="463d2b7c-c704-4816-8043-08c4b8859236" path="/var/lib/kubelet/pods/463d2b7c-c704-4816-8043-08c4b8859236/volumes"
Sep 05 08:13:35.730673 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:35.730660    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df5324d5-c28f-4f43-8875-343bd2d822c2" path="/var/lib/kubelet/pods/df5324d5-c28f-4f43-8875-343bd2d822c2/volumes"
Sep 05 08:13:47.870219 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:47.870134    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:13:48.576161 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:48.576130    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:13:49.988673 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:49.988642    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:13:51.975758 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:51.975722    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:13:52.064917 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:52.064883    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:13:53.128517 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:53.128474    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:13:53.231377 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:53.231298    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:13:53.342377 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:53.342338    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:13:53.594096 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:53.594060    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:13:53.706356 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:53.706321    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:13:53.736125 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:53.736098    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d14aea-79fb454499-4sb89"]
Sep 05 08:13:53.736432 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:53.736418    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df5324d5-c28f-4f43-8875-343bd2d822c2" containerName="main"
Sep 05 08:13:53.736489 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:53.736433    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="df5324d5-c28f-4f43-8875-343bd2d822c2" containerName="main"
Sep 05 08:13:53.736489 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:53.736448    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e689b348-b5cd-4542-a773-6fe141e5e4de" containerName="revoke-keys"
Sep 05 08:13:53.736489 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:53.736453    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e689b348-b5cd-4542-a773-6fe141e5e4de" containerName="revoke-keys"
Sep 05 08:13:53.736489 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:53.736459    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="463d2b7c-c704-4816-8043-08c4b8859236" containerName="main"
Sep 05 08:13:53.736489 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:53.736466    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="463d2b7c-c704-4816-8043-08c4b8859236" containerName="main"
Sep 05 08:13:53.736644 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:53.736549    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="df5324d5-c28f-4f43-8875-343bd2d822c2" containerName="main"
Sep 05 08:13:53.736644 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:53.736558    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="463d2b7c-c704-4816-8043-08c4b8859236" containerName="main"
Sep 05 08:13:53.736644 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:53.736563    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e689b348-b5cd-4542-a773-6fe141e5e4de" containerName="revoke-keys"
Sep 05 08:13:53.739699 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:53.739673    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d14aea-79fb454499-4sb89"
Sep 05 08:13:53.741909 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:53.741884    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-d14aea\""
Sep 05 08:13:53.750773 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:53.750752    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d14aea-79fb454499-4sb89"]
Sep 05 08:13:53.809772 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:53.809740    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d14aea-5c4975697d-h6q7p"]
Sep 05 08:13:53.813083 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:53.813068    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d14aea-5c4975697d-h6q7p"
Sep 05 08:13:53.814996 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:53.814974    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-d14aea-dockercfg-p6q8q\""
Sep 05 08:13:53.823352 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:53.823330    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d14aea-5c4975697d-h6q7p"]
Sep 05 08:13:53.887471 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:53.887389    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5cf0811d-1f0a-4452-a295-2d9c732ec0d5-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-d14aea-79fb454499-4sb89\" (UID: \"5cf0811d-1f0a-4452-a295-2d9c732ec0d5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d14aea-79fb454499-4sb89"
Sep 05 08:13:53.887471 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:53.887432    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/996b705a-1d70-4bf6-a84d-4e73d334c7f2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-d14aea-5c4975697d-h6q7p\" (UID: \"996b705a-1d70-4bf6-a84d-4e73d334c7f2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d14aea-5c4975697d-h6q7p"
Sep 05 08:13:53.887471 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:53.887459    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cfc2\" (UniqueName: \"kubernetes.io/projected/996b705a-1d70-4bf6-a84d-4e73d334c7f2-kube-api-access-9cfc2\") pod \"payload-pre-processing-e2e-ipp-b-d14aea-5c4975697d-h6q7p\" (UID: \"996b705a-1d70-4bf6-a84d-4e73d334c7f2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d14aea-5c4975697d-h6q7p"
Sep 05 08:13:53.887737 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:53.887557    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h45v9\" (UniqueName: \"kubernetes.io/projected/5cf0811d-1f0a-4452-a295-2d9c732ec0d5-kube-api-access-h45v9\") pod \"payload-processing-e2e-ipp-b-d14aea-79fb454499-4sb89\" (UID: \"5cf0811d-1f0a-4452-a295-2d9c732ec0d5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d14aea-79fb454499-4sb89"
Sep 05 08:13:53.988632 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:53.988592    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/996b705a-1d70-4bf6-a84d-4e73d334c7f2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-d14aea-5c4975697d-h6q7p\" (UID: \"996b705a-1d70-4bf6-a84d-4e73d334c7f2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d14aea-5c4975697d-h6q7p"
Sep 05 08:13:53.988828 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:53.988648    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cfc2\" (UniqueName: \"kubernetes.io/projected/996b705a-1d70-4bf6-a84d-4e73d334c7f2-kube-api-access-9cfc2\") pod \"payload-pre-processing-e2e-ipp-b-d14aea-5c4975697d-h6q7p\" (UID: \"996b705a-1d70-4bf6-a84d-4e73d334c7f2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d14aea-5c4975697d-h6q7p"
Sep 05 08:13:53.988828 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:53.988686    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h45v9\" (UniqueName: \"kubernetes.io/projected/5cf0811d-1f0a-4452-a295-2d9c732ec0d5-kube-api-access-h45v9\") pod \"payload-processing-e2e-ipp-b-d14aea-79fb454499-4sb89\" (UID: \"5cf0811d-1f0a-4452-a295-2d9c732ec0d5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d14aea-79fb454499-4sb89"
Sep 05 08:13:53.988828 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:53.988781    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5cf0811d-1f0a-4452-a295-2d9c732ec0d5-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-d14aea-79fb454499-4sb89\" (UID: \"5cf0811d-1f0a-4452-a295-2d9c732ec0d5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d14aea-79fb454499-4sb89"
Sep 05 08:13:53.989331 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:53.989309    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/996b705a-1d70-4bf6-a84d-4e73d334c7f2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-d14aea-5c4975697d-h6q7p\" (UID: \"996b705a-1d70-4bf6-a84d-4e73d334c7f2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d14aea-5c4975697d-h6q7p"
Sep 05 08:13:53.989520 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:53.989504    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5cf0811d-1f0a-4452-a295-2d9c732ec0d5-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-d14aea-79fb454499-4sb89\" (UID: \"5cf0811d-1f0a-4452-a295-2d9c732ec0d5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d14aea-79fb454499-4sb89"
Sep 05 08:13:54.002654 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:54.002624    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h45v9\" (UniqueName: \"kubernetes.io/projected/5cf0811d-1f0a-4452-a295-2d9c732ec0d5-kube-api-access-h45v9\") pod \"payload-processing-e2e-ipp-b-d14aea-79fb454499-4sb89\" (UID: \"5cf0811d-1f0a-4452-a295-2d9c732ec0d5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d14aea-79fb454499-4sb89"
Sep 05 08:13:54.002773 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:54.002707    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cfc2\" (UniqueName: \"kubernetes.io/projected/996b705a-1d70-4bf6-a84d-4e73d334c7f2-kube-api-access-9cfc2\") pod \"payload-pre-processing-e2e-ipp-b-d14aea-5c4975697d-h6q7p\" (UID: \"996b705a-1d70-4bf6-a84d-4e73d334c7f2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d14aea-5c4975697d-h6q7p"
Sep 05 08:13:54.050435 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:54.050395    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d14aea-79fb454499-4sb89"
Sep 05 08:13:54.123099 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:54.123071    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d14aea-5c4975697d-h6q7p"
Sep 05 08:13:54.321366 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:54.321338    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d14aea-79fb454499-4sb89"]
Sep 05 08:13:54.322899 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:13:54.322870    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cf0811d_1f0a_4452_a295_2d9c732ec0d5.slice/crio-4228eeb4b47dbeba9e7d07d7c8540716efd8a2060bc35e0e82a3c9cfb65fb803 WatchSource:0}: Error finding container 4228eeb4b47dbeba9e7d07d7c8540716efd8a2060bc35e0e82a3c9cfb65fb803: Status 404 returned error can't find the container with id 4228eeb4b47dbeba9e7d07d7c8540716efd8a2060bc35e0e82a3c9cfb65fb803
Sep 05 08:13:54.395907 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:54.395884    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d14aea-5c4975697d-h6q7p"]
Sep 05 08:13:54.398147 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:13:54.398120    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod996b705a_1d70_4bf6_a84d_4e73d334c7f2.slice/crio-5bb0b49347797a2339228ee62c9d05ef499e28dcf2401349d28d8601332cd162 WatchSource:0}: Error finding container 5bb0b49347797a2339228ee62c9d05ef499e28dcf2401349d28d8601332cd162: Status 404 returned error can't find the container with id 5bb0b49347797a2339228ee62c9d05ef499e28dcf2401349d28d8601332cd162
Sep 05 08:13:54.408468 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:54.408427    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d14aea-79fb454499-4sb89" event={"ID":"5cf0811d-1f0a-4452-a295-2d9c732ec0d5","Type":"ContainerStarted","Data":"17fd1bb292067544e9a0e1a94355c1c8930bd2bab4ea1d601c4e1e33a17552de"}
Sep 05 08:13:54.408585 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:54.408506    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-d14aea-79fb454499-4sb89"
Sep 05 08:13:54.408585 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:54.408531    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d14aea-79fb454499-4sb89" event={"ID":"5cf0811d-1f0a-4452-a295-2d9c732ec0d5","Type":"ContainerStarted","Data":"4228eeb4b47dbeba9e7d07d7c8540716efd8a2060bc35e0e82a3c9cfb65fb803"}
Sep 05 08:13:54.409717 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:54.409694    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d14aea-5c4975697d-h6q7p" event={"ID":"996b705a-1d70-4bf6-a84d-4e73d334c7f2","Type":"ContainerStarted","Data":"5bb0b49347797a2339228ee62c9d05ef499e28dcf2401349d28d8601332cd162"}
Sep 05 08:13:54.428431 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:54.425116    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-d14aea-79fb454499-4sb89" podStartSLOduration=1.42509349 podStartE2EDuration="1.42509349s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:13:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:13:54.421743093 +0000 UTC m=+1537.191885454" watchObservedRunningTime="2026-09-05 08:13:54.42509349 +0000 UTC m=+1537.195235828"
Sep 05 08:13:55.417151 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:55.417113    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d14aea-5c4975697d-h6q7p" event={"ID":"996b705a-1d70-4bf6-a84d-4e73d334c7f2","Type":"ContainerStarted","Data":"4870ad661d6f2da57e2693bf929697db4c7d4922f2cac79f3de96e7bd96ac40c"}
Sep 05 08:13:55.417644 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:55.417271    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d14aea-5c4975697d-h6q7p"
Sep 05 08:13:55.432200 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:13:55.432154    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d14aea-5c4975697d-h6q7p" podStartSLOduration=2.432138771 podStartE2EDuration="2.432138771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:13:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:13:55.430881841 +0000 UTC m=+1538.201024182" watchObservedRunningTime="2026-09-05 08:13:55.432138771 +0000 UTC m=+1538.202281107"
Sep 05 08:14:02.685642 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:14:02.685603    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:14:05.420042 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:14:05.420009    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-d14aea-79fb454499-4sb89"
Sep 05 08:14:06.424439 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:14:06.424398    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d14aea-5c4975697d-h6q7p"
Sep 05 08:14:17.007167 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:14:17.007132    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:14:22.423023 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:14:22.422987    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-d14aea-79fb454499-4sb89_5cf0811d-1f0a-4452-a295-2d9c732ec0d5/payload-processing/0.log"
Sep 05 08:14:34.596548 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:14:34.596512    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-3d4dc1/ipp-route-3d4dc1-kserve-64b8465fff-6dwj6"]
Sep 05 08:14:34.601435 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:14:34.601416    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-3d4dc1/ipp-route-3d4dc1-kserve-64b8465fff-6dwj6"
Sep 05 08:14:34.604238 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:14:34.604215    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-3d4dc1\"/\"ipp-route-3d4dc1-kserve-self-signed-certs\""
Sep 05 08:14:34.604238 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:14:34.604224    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-3d4dc1\"/\"openshift-service-ca.crt\""
Sep 05 08:14:34.604431 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:14:34.604216    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-3d4dc1\"/\"default-dockercfg-vgstb\""
Sep 05 08:14:34.604431 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:14:34.604224    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-3d4dc1\"/\"kube-root-ca.crt\""
Sep 05 08:14:34.609955 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:14:34.609934    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-3d4dc1/ipp-route-3d4dc1-kserve-64b8465fff-6dwj6"]
Sep 05 08:14:34.623726 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:14:34.623697    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:14:34.663009 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:14:34.662982    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lvjr\" (UniqueName: \"kubernetes.io/projected/75ad22c4-6c8a-47ab-9d51-c457af6d60c1-kube-api-access-8lvjr\") pod \"ipp-route-3d4dc1-kserve-64b8465fff-6dwj6\" (UID: \"75ad22c4-6c8a-47ab-9d51-c457af6d60c1\") " pod="ai-tenant-e2e-ipp-a-3d4dc1/ipp-route-3d4dc1-kserve-64b8465fff-6dwj6"
Sep 05 08:14:34.663116 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:14:34.663041    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75ad22c4-6c8a-47ab-9d51-c457af6d60c1-tmp-dir\") pod \"ipp-route-3d4dc1-kserve-64b8465fff-6dwj6\" (UID: \"75ad22c4-6c8a-47ab-9d51-c457af6d60c1\") " pod="ai-tenant-e2e-ipp-a-3d4dc1/ipp-route-3d4dc1-kserve-64b8465fff-6dwj6"
Sep 05 08:14:34.663116 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:14:34.663072    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75ad22c4-6c8a-47ab-9d51-c457af6d60c1-model-cache\") pod \"ipp-route-3d4dc1-kserve-64b8465fff-6dwj6\" (UID: \"75ad22c4-6c8a-47ab-9d51-c457af6d60c1\") " pod="ai-tenant-e2e-ipp-a-3d4dc1/ipp-route-3d4dc1-kserve-64b8465fff-6dwj6"
Sep 05 08:14:34.663116 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:14:34.663088    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75ad22c4-6c8a-47ab-9d51-c457af6d60c1-tls-certs\") pod \"ipp-route-3d4dc1-kserve-64b8465fff-6dwj6\" (UID: \"75ad22c4-6c8a-47ab-9d51-c457af6d60c1\") " pod="ai-tenant-e2e-ipp-a-3d4dc1/ipp-route-3d4dc1-kserve-64b8465fff-6dwj6"
Sep 05 08:14:34.663116 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:14:34.663103    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75ad22c4-6c8a-47ab-9d51-c457af6d60c1-dshm\") pod \"ipp-route-3d4dc1-kserve-64b8465fff-6dwj6\" (UID: \"75ad22c4-6c8a-47ab-9d51-c457af6d60c1\") " pod="ai-tenant-e2e-ipp-a-3d4dc1/ipp-route-3d4dc1-kserve-64b8465fff-6dwj6"
Sep 05 08:14:34.663254 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:14:34.663152    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75ad22c4-6c8a-47ab-9d51-c457af6d60c1-home\") pod \"ipp-route-3d4dc1-kserve-64b8465fff-6dwj6\" (UID: \"75ad22c4-6c8a-47ab-9d51-c457af6d60c1\") " pod="ai-tenant-e2e-ipp-a-3d4dc1/ipp-route-3d4dc1-kserve-64b8465fff-6dwj6"
Sep 05 08:14:34.764346 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:14:34.764309    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lvjr\" (UniqueName: \"kubernetes.io/projected/75ad22c4-6c8a-47ab-9d51-c457af6d60c1-kube-api-access-8lvjr\") pod \"ipp-route-3d4dc1-kserve-64b8465fff-6dwj6\" (UID: \"75ad22c4-6c8a-47ab-9d51-c457af6d60c1\") " pod="ai-tenant-e2e-ipp-a-3d4dc1/ipp-route-3d4dc1-kserve-64b8465fff-6dwj6"
Sep 05 08:14:34.764516 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:14:34.764407    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75ad22c4-6c8a-47ab-9d51-c457af6d60c1-tmp-dir\") pod \"ipp-route-3d4dc1-kserve-64b8465fff-6dwj6\" (UID: \"75ad22c4-6c8a-47ab-9d51-c457af6d60c1\") " pod="ai-tenant-e2e-ipp-a-3d4dc1/ipp-route-3d4dc1-kserve-64b8465fff-6dwj6"
Sep 05 08:14:34.764516 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:14:34.764458    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75ad22c4-6c8a-47ab-9d51-c457af6d60c1-model-cache\") pod \"ipp-route-3d4dc1-kserve-64b8465fff-6dwj6\" (UID: \"75ad22c4-6c8a-47ab-9d51-c457af6d60c1\") " pod="ai-tenant-e2e-ipp-a-3d4dc1/ipp-route-3d4dc1-kserve-64b8465fff-6dwj6"
Sep 05 08:14:34.764516 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:14:34.764480    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75ad22c4-6c8a-47ab-9d51-c457af6d60c1-tls-certs\") pod \"ipp-route-3d4dc1-kserve-64b8465fff-6dwj6\" (UID: \"75ad22c4-6c8a-47ab-9d51-c457af6d60c1\") " pod="ai-tenant-e2e-ipp-a-3d4dc1/ipp-route-3d4dc1-kserve-64b8465fff-6dwj6"
Sep 05 08:14:34.764516 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:14:34.764504    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75ad22c4-6c8a-47ab-9d51-c457af6d60c1-dshm\") pod \"ipp-route-3d4dc1-kserve-64b8465fff-6dwj6\" (UID: \"75ad22c4-6c8a-47ab-9d51-c457af6d60c1\") " pod="ai-tenant-e2e-ipp-a-3d4dc1/ipp-route-3d4dc1-kserve-64b8465fff-6dwj6"
Sep 05 08:14:34.764773 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:14:34.764532    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75ad22c4-6c8a-47ab-9d51-c457af6d60c1-home\") pod \"ipp-route-3d4dc1-kserve-64b8465fff-6dwj6\" (UID: \"75ad22c4-6c8a-47ab-9d51-c457af6d60c1\") " pod="ai-tenant-e2e-ipp-a-3d4dc1/ipp-route-3d4dc1-kserve-64b8465fff-6dwj6"
Sep 05 08:14:34.764959 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:14:34.764936    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75ad22c4-6c8a-47ab-9d51-c457af6d60c1-home\") pod \"ipp-route-3d4dc1-kserve-64b8465fff-6dwj6\" (UID: \"75ad22c4-6c8a-47ab-9d51-c457af6d60c1\") " pod="ai-tenant-e2e-ipp-a-3d4dc1/ipp-route-3d4dc1-kserve-64b8465fff-6dwj6"
Sep 05 08:14:34.765153 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:14:34.765012    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75ad22c4-6c8a-47ab-9d51-c457af6d60c1-model-cache\") pod \"ipp-route-3d4dc1-kserve-64b8465fff-6dwj6\" (UID: \"75ad22c4-6c8a-47ab-9d51-c457af6d60c1\") " pod="ai-tenant-e2e-ipp-a-3d4dc1/ipp-route-3d4dc1-kserve-64b8465fff-6dwj6"
Sep 05 08:14:34.765153 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:14:34.765058    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75ad22c4-6c8a-47ab-9d51-c457af6d60c1-tmp-dir\") pod \"ipp-route-3d4dc1-kserve-64b8465fff-6dwj6\" (UID: \"75ad22c4-6c8a-47ab-9d51-c457af6d60c1\") " pod="ai-tenant-e2e-ipp-a-3d4dc1/ipp-route-3d4dc1-kserve-64b8465fff-6dwj6"
Sep 05 08:14:34.767055 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:14:34.767035    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75ad22c4-6c8a-47ab-9d51-c457af6d60c1-dshm\") pod \"ipp-route-3d4dc1-kserve-64b8465fff-6dwj6\" (UID: \"75ad22c4-6c8a-47ab-9d51-c457af6d60c1\") " pod="ai-tenant-e2e-ipp-a-3d4dc1/ipp-route-3d4dc1-kserve-64b8465fff-6dwj6"
Sep 05 08:14:34.767496 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:14:34.767480    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75ad22c4-6c8a-47ab-9d51-c457af6d60c1-tls-certs\") pod \"ipp-route-3d4dc1-kserve-64b8465fff-6dwj6\" (UID: \"75ad22c4-6c8a-47ab-9d51-c457af6d60c1\") " pod="ai-tenant-e2e-ipp-a-3d4dc1/ipp-route-3d4dc1-kserve-64b8465fff-6dwj6"
Sep 05 08:14:34.788577 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:14:34.788551    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lvjr\" (UniqueName: \"kubernetes.io/projected/75ad22c4-6c8a-47ab-9d51-c457af6d60c1-kube-api-access-8lvjr\") pod \"ipp-route-3d4dc1-kserve-64b8465fff-6dwj6\" (UID: \"75ad22c4-6c8a-47ab-9d51-c457af6d60c1\") " pod="ai-tenant-e2e-ipp-a-3d4dc1/ipp-route-3d4dc1-kserve-64b8465fff-6dwj6"
Sep 05 08:14:34.910903 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:14:34.910779    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-3d4dc1/ipp-route-3d4dc1-kserve-64b8465fff-6dwj6"
Sep 05 08:14:35.188104 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:14:35.188019    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-3d4dc1/ipp-route-3d4dc1-kserve-64b8465fff-6dwj6"]
Sep 05 08:14:35.190271 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:14:35.190239    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod75ad22c4_6c8a_47ab_9d51_c457af6d60c1.slice/crio-83004793ec10247b16ea09577e97c053cda8113840d365da2ea3f7c0f0f5e3ae WatchSource:0}: Error finding container 83004793ec10247b16ea09577e97c053cda8113840d365da2ea3f7c0f0f5e3ae: Status 404 returned error can't find the container with id 83004793ec10247b16ea09577e97c053cda8113840d365da2ea3f7c0f0f5e3ae
Sep 05 08:14:35.629484 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:14:35.629445    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-3d4dc1/ipp-route-3d4dc1-kserve-64b8465fff-6dwj6" event={"ID":"75ad22c4-6c8a-47ab-9d51-c457af6d60c1","Type":"ContainerStarted","Data":"bb6553c4ee8983f21c1f6e9ef88530a03c99464ac5f281d4a3f61c2733cd4c08"}
Sep 05 08:14:35.629920 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:14:35.629497    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-3d4dc1/ipp-route-3d4dc1-kserve-64b8465fff-6dwj6" event={"ID":"75ad22c4-6c8a-47ab-9d51-c457af6d60c1","Type":"ContainerStarted","Data":"83004793ec10247b16ea09577e97c053cda8113840d365da2ea3f7c0f0f5e3ae"}
Sep 05 08:14:35.645901 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:14:35.645835    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-3d4dc1/ipp-route-3d4dc1-kserve-64b8465fff-6dwj6" podStartSLOduration=1.6458184390000001 podStartE2EDuration="1.645818439s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:14:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:14:35.644240845 +0000 UTC m=+1578.414383202" watchObservedRunningTime="2026-09-05 08:14:35.645818439 +0000 UTC m=+1578.415960775"
Sep 05 08:14:44.911613 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:14:44.911575    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-3d4dc1/ipp-route-3d4dc1-kserve-64b8465fff-6dwj6"
Sep 05 08:14:44.912033 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:14:44.911628    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-3d4dc1/ipp-route-3d4dc1-kserve-64b8465fff-6dwj6"
Sep 05 08:14:44.924231 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:14:44.924209    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-3d4dc1/ipp-route-3d4dc1-kserve-64b8465fff-6dwj6"
Sep 05 08:14:45.689116 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:14:45.689087    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-3d4dc1/ipp-route-3d4dc1-kserve-64b8465fff-6dwj6"
Sep 05 08:15:00.132574 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:00.132527    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-d14aea-29809935-fdmx2"]
Sep 05 08:15:00.136068 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:00.136045    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-5ef325-29809935-xxrst"]
Sep 05 08:15:00.136208 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:00.136176    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-d14aea-29809935-fdmx2"
Sep 05 08:15:00.138327 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:00.138306    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mpmmp\""
Sep 05 08:15:00.139263 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:00.139246    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29809935-qksv5"]
Sep 05 08:15:00.139322 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:00.139310    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-5ef325-29809935-xxrst"
Sep 05 08:15:00.142442 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:00.142425    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809935-qksv5"
Sep 05 08:15:00.145791 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:00.145766    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-d14aea-29809935-fdmx2"]
Sep 05 08:15:00.149064 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:00.148942    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-d14aea-29809935-fdmx2"
Sep 05 08:15:00.149778 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:00.149714    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29809935-qksv5"]
Sep 05 08:15:00.153052 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:00.152354    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-5ef325-29809935-xxrst"]
Sep 05 08:15:00.158792 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:00.158758    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-5ef325-29809935-xxrst"
Sep 05 08:15:00.191064 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:00.191041    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809935-qksv5"
Sep 05 08:15:00.435342 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:00.435318    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-d14aea-29809935-fdmx2"]
Sep 05 08:15:00.437584 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:15:00.437550    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc954f0e_477e_44db_8f4c_c425905ceb19.slice/crio-7ae631c3eb8c003611b8125e55f48df932a5d2343a55c3db2856dab2a670b744 WatchSource:0}: Error finding container 7ae631c3eb8c003611b8125e55f48df932a5d2343a55c3db2856dab2a670b744: Status 404 returned error can't find the container with id 7ae631c3eb8c003611b8125e55f48df932a5d2343a55c3db2856dab2a670b744
Sep 05 08:15:00.441624 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:00.441604    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-5ef325-29809935-xxrst"]
Sep 05 08:15:00.443606 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:15:00.443578    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4d8d2a7_3ab8_49ec_b21b_60edb0ecc8b4.slice/crio-debd95f218a186ebcd21ee40ca8229abdf8e88c2173459e0e25cb3818eb778f7 WatchSource:0}: Error finding container debd95f218a186ebcd21ee40ca8229abdf8e88c2173459e0e25cb3818eb778f7: Status 404 returned error can't find the container with id debd95f218a186ebcd21ee40ca8229abdf8e88c2173459e0e25cb3818eb778f7
Sep 05 08:15:00.479280 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:00.479259    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29809935-qksv5"]
Sep 05 08:15:00.481205 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:15:00.481182    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdbfa942_081a_4296_898b_e3538a51051e.slice/crio-936d49bac3e1fdf3ce4a4d9813fba900527c9d4ba8d3c54c23d66e1d056ab8e6 WatchSource:0}: Error finding container 936d49bac3e1fdf3ce4a4d9813fba900527c9d4ba8d3c54c23d66e1d056ab8e6: Status 404 returned error can't find the container with id 936d49bac3e1fdf3ce4a4d9813fba900527c9d4ba8d3c54c23d66e1d056ab8e6
Sep 05 08:15:00.754043 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:00.754009    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-5ef325-29809935-xxrst" event={"ID":"f4d8d2a7-3ab8-49ec-b21b-60edb0ecc8b4","Type":"ContainerStarted","Data":"1737ecc7f8a6e5a9c15122ca93cea30bfeb8c853da73d784c0ceb1fbe7c6747f"}
Sep 05 08:15:00.754374 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:00.754061    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-5ef325-29809935-xxrst" event={"ID":"f4d8d2a7-3ab8-49ec-b21b-60edb0ecc8b4","Type":"ContainerStarted","Data":"debd95f218a186ebcd21ee40ca8229abdf8e88c2173459e0e25cb3818eb778f7"}
Sep 05 08:15:00.755396 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:00.755371    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-d14aea-29809935-fdmx2" event={"ID":"fc954f0e-477e-44db-8f4c-c425905ceb19","Type":"ContainerStarted","Data":"7ae631c3eb8c003611b8125e55f48df932a5d2343a55c3db2856dab2a670b744"}
Sep 05 08:15:00.756624 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:00.756602    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809935-qksv5" event={"ID":"cdbfa942-081a-4296-898b-e3538a51051e","Type":"ContainerStarted","Data":"936d49bac3e1fdf3ce4a4d9813fba900527c9d4ba8d3c54c23d66e1d056ab8e6"}
Sep 05 08:15:00.768030 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:00.767990    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-5ef325-29809935-xxrst" podStartSLOduration=0.767975899 podStartE2EDuration="767.975899ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:15:00.766149824 +0000 UTC m=+1603.536292173" watchObservedRunningTime="2026-09-05 08:15:00.767975899 +0000 UTC m=+1603.538118237"
Sep 05 08:15:01.769688 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:01.769587    2563 generic.go:356] "Generic (PLEG): container finished" podID="fc954f0e-477e-44db-8f4c-c425905ceb19" containerID="26308eeb67010b6bcc05b4511aadb496b7f6be7751e953b514b9acd75f22fd13" exitCode=0
Sep 05 08:15:01.769688 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:01.769674    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-d14aea-29809935-fdmx2" event={"ID":"fc954f0e-477e-44db-8f4c-c425905ceb19","Type":"ContainerDied","Data":"26308eeb67010b6bcc05b4511aadb496b7f6be7751e953b514b9acd75f22fd13"}
Sep 05 08:15:01.771342 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:01.771311    2563 generic.go:356] "Generic (PLEG): container finished" podID="cdbfa942-081a-4296-898b-e3538a51051e" containerID="785e5828beede625bee6b4cb595dee94ea618afd0aeaca8f07f3cd5569246c12" exitCode=0
Sep 05 08:15:01.771404 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:01.771390    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809935-qksv5" event={"ID":"cdbfa942-081a-4296-898b-e3538a51051e","Type":"ContainerDied","Data":"785e5828beede625bee6b4cb595dee94ea618afd0aeaca8f07f3cd5569246c12"}
Sep 05 08:15:01.773098 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:01.773079    2563 generic.go:356] "Generic (PLEG): container finished" podID="f4d8d2a7-3ab8-49ec-b21b-60edb0ecc8b4" containerID="1737ecc7f8a6e5a9c15122ca93cea30bfeb8c853da73d784c0ceb1fbe7c6747f" exitCode=0
Sep 05 08:15:01.773219 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:01.773109    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-5ef325-29809935-xxrst" event={"ID":"f4d8d2a7-3ab8-49ec-b21b-60edb0ecc8b4","Type":"ContainerDied","Data":"1737ecc7f8a6e5a9c15122ca93cea30bfeb8c853da73d784c0ceb1fbe7c6747f"}
Sep 05 08:15:02.942268 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:02.942244    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-d14aea-29809935-fdmx2"
Sep 05 08:15:02.964168 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:02.964135    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-5ef325-29809935-xxrst"
Sep 05 08:15:02.967651 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:02.967632    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809935-qksv5"
Sep 05 08:15:03.023331 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:03.023297    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-d14aea-79fb454499-4sb89_5cf0811d-1f0a-4452-a295-2d9c732ec0d5/payload-processing/0.log"
Sep 05 08:15:03.312621 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:03.312569    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:15:03.620834 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:03.620760    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:15:03.784064 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:03.784044    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-5ef325-29809935-xxrst"
Sep 05 08:15:03.784192 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:03.784041    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-5ef325-29809935-xxrst" event={"ID":"f4d8d2a7-3ab8-49ec-b21b-60edb0ecc8b4","Type":"ContainerDied","Data":"debd95f218a186ebcd21ee40ca8229abdf8e88c2173459e0e25cb3818eb778f7"}
Sep 05 08:15:03.784192 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:03.784176    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="debd95f218a186ebcd21ee40ca8229abdf8e88c2173459e0e25cb3818eb778f7"
Sep 05 08:15:03.785404 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:03.785385    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-d14aea-29809935-fdmx2"
Sep 05 08:15:03.785404 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:03.785396    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-d14aea-29809935-fdmx2" event={"ID":"fc954f0e-477e-44db-8f4c-c425905ceb19","Type":"ContainerDied","Data":"7ae631c3eb8c003611b8125e55f48df932a5d2343a55c3db2856dab2a670b744"}
Sep 05 08:15:03.785600 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:03.785430    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7ae631c3eb8c003611b8125e55f48df932a5d2343a55c3db2856dab2a670b744"
Sep 05 08:15:03.786884 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:03.786865    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809935-qksv5"
Sep 05 08:15:03.786884 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:03.786878    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809935-qksv5" event={"ID":"cdbfa942-081a-4296-898b-e3538a51051e","Type":"ContainerDied","Data":"936d49bac3e1fdf3ce4a4d9813fba900527c9d4ba8d3c54c23d66e1d056ab8e6"}
Sep 05 08:15:03.787054 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:03.786910    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="936d49bac3e1fdf3ce4a4d9813fba900527c9d4ba8d3c54c23d66e1d056ab8e6"
Sep 05 08:15:04.147863 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:04.147824    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:15:06.411020 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:06.410984    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:15:06.509543 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:06.509508    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:15:06.708495 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:06.708407    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:15:06.809134 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:06.809107    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:15:07.562355 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:07.562322    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:15:07.661519 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:07.661485    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:15:07.778219 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:07.778189    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:15:08.086502 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:08.086469    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:15:08.203292 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:08.203261    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:15:08.227370 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:08.227337    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9d46f-7df8998-x7jzc"]
Sep 05 08:15:08.227690 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:08.227677    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4d8d2a7-3ab8-49ec-b21b-60edb0ecc8b4" containerName="cleanup"
Sep 05 08:15:08.227731 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:08.227691    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4d8d2a7-3ab8-49ec-b21b-60edb0ecc8b4" containerName="cleanup"
Sep 05 08:15:08.227731 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:08.227703    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc954f0e-477e-44db-8f4c-c425905ceb19" containerName="cleanup"
Sep 05 08:15:08.227731 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:08.227708    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc954f0e-477e-44db-8f4c-c425905ceb19" containerName="cleanup"
Sep 05 08:15:08.227731 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:08.227716    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdbfa942-081a-4296-898b-e3538a51051e" containerName="cleanup"
Sep 05 08:15:08.227731 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:08.227722    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdbfa942-081a-4296-898b-e3538a51051e" containerName="cleanup"
Sep 05 08:15:08.227949 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:08.227786    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4d8d2a7-3ab8-49ec-b21b-60edb0ecc8b4" containerName="cleanup"
Sep 05 08:15:08.227949 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:08.227794    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc954f0e-477e-44db-8f4c-c425905ceb19" containerName="cleanup"
Sep 05 08:15:08.227949 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:08.227800    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdbfa942-081a-4296-898b-e3538a51051e" containerName="cleanup"
Sep 05 08:15:08.232202 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:08.232185    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9d46f-7df8998-x7jzc"
Sep 05 08:15:08.234488 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:08.234468    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-d9d46f\""
Sep 05 08:15:08.234619 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:08.234601    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-d9d46f-dockercfg-l6zbl\""
Sep 05 08:15:08.248712 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:08.248690    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9d46f-7df8998-x7jzc"]
Sep 05 08:15:08.268906 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:08.268878    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98db5\" (UniqueName: \"kubernetes.io/projected/13017a08-dd33-4206-a3df-5f0bef5ea494-kube-api-access-98db5\") pod \"payload-pre-processing-e2e-ipp-cleanup-d9d46f-7df8998-x7jzc\" (UID: \"13017a08-dd33-4206-a3df-5f0bef5ea494\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9d46f-7df8998-x7jzc"
Sep 05 08:15:08.269020 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:08.268980    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13017a08-dd33-4206-a3df-5f0bef5ea494-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-d9d46f-7df8998-x7jzc\" (UID: \"13017a08-dd33-4206-a3df-5f0bef5ea494\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9d46f-7df8998-x7jzc"
Sep 05 08:15:08.369724 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:08.369629    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13017a08-dd33-4206-a3df-5f0bef5ea494-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-d9d46f-7df8998-x7jzc\" (UID: \"13017a08-dd33-4206-a3df-5f0bef5ea494\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9d46f-7df8998-x7jzc"
Sep 05 08:15:08.369902 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:08.369734    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98db5\" (UniqueName: \"kubernetes.io/projected/13017a08-dd33-4206-a3df-5f0bef5ea494-kube-api-access-98db5\") pod \"payload-pre-processing-e2e-ipp-cleanup-d9d46f-7df8998-x7jzc\" (UID: \"13017a08-dd33-4206-a3df-5f0bef5ea494\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9d46f-7df8998-x7jzc"
Sep 05 08:15:08.370393 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:08.370372    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13017a08-dd33-4206-a3df-5f0bef5ea494-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-d9d46f-7df8998-x7jzc\" (UID: \"13017a08-dd33-4206-a3df-5f0bef5ea494\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9d46f-7df8998-x7jzc"
Sep 05 08:15:08.377018 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:08.376996    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98db5\" (UniqueName: \"kubernetes.io/projected/13017a08-dd33-4206-a3df-5f0bef5ea494-kube-api-access-98db5\") pod \"payload-pre-processing-e2e-ipp-cleanup-d9d46f-7df8998-x7jzc\" (UID: \"13017a08-dd33-4206-a3df-5f0bef5ea494\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9d46f-7df8998-x7jzc"
Sep 05 08:15:08.542594 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:08.542553    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9d46f-7df8998-x7jzc"
Sep 05 08:15:08.814158 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:08.814133    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9d46f-7df8998-x7jzc"]
Sep 05 08:15:08.816308 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:15:08.816284    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13017a08_dd33_4206_a3df_5f0bef5ea494.slice/crio-cd19ebe3bfc0926c7a8f51e5694993c64b7e65f8d2dae5ff5368ebcced3db8c1 WatchSource:0}: Error finding container cd19ebe3bfc0926c7a8f51e5694993c64b7e65f8d2dae5ff5368ebcced3db8c1: Status 404 returned error can't find the container with id cd19ebe3bfc0926c7a8f51e5694993c64b7e65f8d2dae5ff5368ebcced3db8c1
Sep 05 08:15:09.819626 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:09.819595    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9d46f-7df8998-x7jzc" event={"ID":"13017a08-dd33-4206-a3df-5f0bef5ea494","Type":"ContainerStarted","Data":"5299400d3bfdd19cde80b7a163cbda69e897c95083f4aebfb4173100b385fec0"}
Sep 05 08:15:09.820012 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:09.819646    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9d46f-7df8998-x7jzc" event={"ID":"13017a08-dd33-4206-a3df-5f0bef5ea494","Type":"ContainerStarted","Data":"cd19ebe3bfc0926c7a8f51e5694993c64b7e65f8d2dae5ff5368ebcced3db8c1"}
Sep 05 08:15:09.820012 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:09.819672    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9d46f-7df8998-x7jzc"
Sep 05 08:15:09.834867 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:09.834799    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9d46f-7df8998-x7jzc" podStartSLOduration=1.834782672 podStartE2EDuration="1.834782672s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:15:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:15:09.833485471 +0000 UTC m=+1612.603627831" watchObservedRunningTime="2026-09-05 08:15:09.834782672 +0000 UTC m=+1612.604925009"
Sep 05 08:15:18.049969 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:18.049936    2563 scope.go:117] "RemoveContainer" containerID="51ed61e87a980070ff3f54b4a10c818d3a927719d7c6e16f76f110aade645081"
Sep 05 08:15:20.827637 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:20.827598    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9d46f-7df8998-x7jzc"
Sep 05 08:15:31.429592 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:31.429551    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:15:32.151802 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:32.151760    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d9d46f-6t2qz"]
Sep 05 08:15:32.156682 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:32.156665    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d9d46f-6t2qz"
Sep 05 08:15:32.159468 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:32.159441    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mpmmp\""
Sep 05 08:15:32.159627 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:32.159529    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 05 08:15:32.161324 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:32.161301    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d9d46f-6t2qz"]
Sep 05 08:15:32.302874 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:32.302807    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f14477b-08a3-4dcc-990a-4ee6456b7420-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-d9d46f-6t2qz\" (UID: \"8f14477b-08a3-4dcc-990a-4ee6456b7420\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d9d46f-6t2qz"
Sep 05 08:15:32.404029 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:32.403941    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f14477b-08a3-4dcc-990a-4ee6456b7420-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-d9d46f-6t2qz\" (UID: \"8f14477b-08a3-4dcc-990a-4ee6456b7420\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d9d46f-6t2qz"
Sep 05 08:15:32.404563 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:32.404541    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f14477b-08a3-4dcc-990a-4ee6456b7420-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-d9d46f-6t2qz\" (UID: \"8f14477b-08a3-4dcc-990a-4ee6456b7420\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d9d46f-6t2qz"
Sep 05 08:15:32.470960 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:32.470925    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d9d46f-6t2qz"
Sep 05 08:15:32.746157 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:32.746134    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d9d46f-6t2qz"]
Sep 05 08:15:32.748470 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:15:32.748434    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f14477b_08a3_4dcc_990a_4ee6456b7420.slice/crio-314bd2d0d75f5c1531314b17c516515c7b8740c15c59958fa8fc9ee7587981c5 WatchSource:0}: Error finding container 314bd2d0d75f5c1531314b17c516515c7b8740c15c59958fa8fc9ee7587981c5: Status 404 returned error can't find the container with id 314bd2d0d75f5c1531314b17c516515c7b8740c15c59958fa8fc9ee7587981c5
Sep 05 08:15:32.939964 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:32.939931    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d9d46f-6t2qz" event={"ID":"8f14477b-08a3-4dcc-990a-4ee6456b7420","Type":"ContainerStarted","Data":"314bd2d0d75f5c1531314b17c516515c7b8740c15c59958fa8fc9ee7587981c5"}
Sep 05 08:15:33.947030 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:33.946995    2563 generic.go:356] "Generic (PLEG): container finished" podID="8f14477b-08a3-4dcc-990a-4ee6456b7420" containerID="d22a0ab86207629ab8a927270e71e2363ffa245299214f037949409af9e16e87" exitCode=0
Sep 05 08:15:33.947414 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:33.947082    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d9d46f-6t2qz" event={"ID":"8f14477b-08a3-4dcc-990a-4ee6456b7420","Type":"ContainerDied","Data":"d22a0ab86207629ab8a927270e71e2363ffa245299214f037949409af9e16e87"}
Sep 05 08:15:35.076714 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:35.076688    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d9d46f-6t2qz"
Sep 05 08:15:35.126788 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:35.126761    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f14477b-08a3-4dcc-990a-4ee6456b7420-maas-api-service-ca\") pod \"8f14477b-08a3-4dcc-990a-4ee6456b7420\" (UID: \"8f14477b-08a3-4dcc-990a-4ee6456b7420\") "
Sep 05 08:15:35.127134 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:35.127111    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f14477b-08a3-4dcc-990a-4ee6456b7420-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8f14477b-08a3-4dcc-990a-4ee6456b7420" (UID: "8f14477b-08a3-4dcc-990a-4ee6456b7420"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:15:35.228084 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:35.228002    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f14477b-08a3-4dcc-990a-4ee6456b7420-maas-api-service-ca\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:15:35.961497 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:35.961461    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d9d46f-6t2qz"
Sep 05 08:15:35.961650 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:35.961501    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d9d46f-6t2qz" event={"ID":"8f14477b-08a3-4dcc-990a-4ee6456b7420","Type":"ContainerDied","Data":"314bd2d0d75f5c1531314b17c516515c7b8740c15c59958fa8fc9ee7587981c5"}
Sep 05 08:15:35.961650 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:35.961548    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="314bd2d0d75f5c1531314b17c516515c7b8740c15c59958fa8fc9ee7587981c5"
Sep 05 08:15:38.135808 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:38.135775    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:15:38.162177 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:38.162137    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9d46f-7df8998-x7jzc"]
Sep 05 08:15:38.162372 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:38.162344    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9d46f-7df8998-x7jzc" podUID="13017a08-dd33-4206-a3df-5f0bef5ea494" containerName="payload-pre-processing" containerID="cri-o://5299400d3bfdd19cde80b7a163cbda69e897c95083f4aebfb4173100b385fec0" gracePeriod=30
Sep 05 08:15:38.200820 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:38.200790    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:15:38.302114 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:38.302077    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:15:38.318140 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:38.318098    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d9d46f-6t2qz"]
Sep 05 08:15:38.321136 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:38.321110    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d9d46f-6t2qz"]
Sep 05 08:15:38.402437 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:38.402104    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:15:38.422006 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:38.421981    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9d46f-7df8998-x7jzc"
Sep 05 08:15:38.453289 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:38.453265    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13017a08-dd33-4206-a3df-5f0bef5ea494-plugins-config-volume\") pod \"13017a08-dd33-4206-a3df-5f0bef5ea494\" (UID: \"13017a08-dd33-4206-a3df-5f0bef5ea494\") "
Sep 05 08:15:38.453445 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:38.453306    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-98db5\" (UniqueName: \"kubernetes.io/projected/13017a08-dd33-4206-a3df-5f0bef5ea494-kube-api-access-98db5\") pod \"13017a08-dd33-4206-a3df-5f0bef5ea494\" (UID: \"13017a08-dd33-4206-a3df-5f0bef5ea494\") "
Sep 05 08:15:38.453678 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:38.453658    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13017a08-dd33-4206-a3df-5f0bef5ea494-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "13017a08-dd33-4206-a3df-5f0bef5ea494" (UID: "13017a08-dd33-4206-a3df-5f0bef5ea494"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:15:38.455556 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:38.455531    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13017a08-dd33-4206-a3df-5f0bef5ea494-kube-api-access-98db5" (OuterVolumeSpecName: "kube-api-access-98db5") pod "13017a08-dd33-4206-a3df-5f0bef5ea494" (UID: "13017a08-dd33-4206-a3df-5f0bef5ea494"). InnerVolumeSpecName "kube-api-access-98db5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:15:38.554285 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:38.554252    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98db5\" (UniqueName: \"kubernetes.io/projected/13017a08-dd33-4206-a3df-5f0bef5ea494-kube-api-access-98db5\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:15:38.554285 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:38.554280    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13017a08-dd33-4206-a3df-5f0bef5ea494-plugins-config-volume\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:15:38.733035 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:38.732952    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:15:38.980231 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:38.980199    2563 generic.go:356] "Generic (PLEG): container finished" podID="13017a08-dd33-4206-a3df-5f0bef5ea494" containerID="5299400d3bfdd19cde80b7a163cbda69e897c95083f4aebfb4173100b385fec0" exitCode=0
Sep 05 08:15:38.980408 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:38.980245    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9d46f-7df8998-x7jzc"
Sep 05 08:15:38.980408 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:38.980270    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9d46f-7df8998-x7jzc" event={"ID":"13017a08-dd33-4206-a3df-5f0bef5ea494","Type":"ContainerDied","Data":"5299400d3bfdd19cde80b7a163cbda69e897c95083f4aebfb4173100b385fec0"}
Sep 05 08:15:38.980408 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:38.980315    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9d46f-7df8998-x7jzc" event={"ID":"13017a08-dd33-4206-a3df-5f0bef5ea494","Type":"ContainerDied","Data":"cd19ebe3bfc0926c7a8f51e5694993c64b7e65f8d2dae5ff5368ebcced3db8c1"}
Sep 05 08:15:38.980408 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:38.980338    2563 scope.go:117] "RemoveContainer" containerID="5299400d3bfdd19cde80b7a163cbda69e897c95083f4aebfb4173100b385fec0"
Sep 05 08:15:38.989906 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:38.989885    2563 scope.go:117] "RemoveContainer" containerID="5299400d3bfdd19cde80b7a163cbda69e897c95083f4aebfb4173100b385fec0"
Sep 05 08:15:38.990186 ip-10-0-133-14 kubenswrapper[2563]: E0905 08:15:38.990165    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5299400d3bfdd19cde80b7a163cbda69e897c95083f4aebfb4173100b385fec0\": container with ID starting with 5299400d3bfdd19cde80b7a163cbda69e897c95083f4aebfb4173100b385fec0 not found: ID does not exist" containerID="5299400d3bfdd19cde80b7a163cbda69e897c95083f4aebfb4173100b385fec0"
Sep 05 08:15:38.990228 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:38.990195    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5299400d3bfdd19cde80b7a163cbda69e897c95083f4aebfb4173100b385fec0"} err="failed to get container status \"5299400d3bfdd19cde80b7a163cbda69e897c95083f4aebfb4173100b385fec0\": rpc error: code = NotFound desc = could not find container \"5299400d3bfdd19cde80b7a163cbda69e897c95083f4aebfb4173100b385fec0\": container with ID starting with 5299400d3bfdd19cde80b7a163cbda69e897c95083f4aebfb4173100b385fec0 not found: ID does not exist"
Sep 05 08:15:39.002676 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:39.002648    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9d46f-7df8998-x7jzc"]
Sep 05 08:15:39.005896 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:39.005873    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9d46f-7df8998-x7jzc"]
Sep 05 08:15:39.730530 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:39.730494    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13017a08-dd33-4206-a3df-5f0bef5ea494" path="/var/lib/kubelet/pods/13017a08-dd33-4206-a3df-5f0bef5ea494/volumes"
Sep 05 08:15:39.730930 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:39.730892    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f14477b-08a3-4dcc-990a-4ee6456b7420" path="/var/lib/kubelet/pods/8f14477b-08a3-4dcc-990a-4ee6456b7420/volumes"
Sep 05 08:15:44.582509 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:44.582472    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:15:44.682821 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:44.682767    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:15:44.879642 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:44.879558    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:15:48.648472 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:48.648424    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3d4dc1-jqkxb"]
Sep 05 08:15:48.648961 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:48.648945    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13017a08-dd33-4206-a3df-5f0bef5ea494" containerName="payload-pre-processing"
Sep 05 08:15:48.649021 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:48.648963    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="13017a08-dd33-4206-a3df-5f0bef5ea494" containerName="payload-pre-processing"
Sep 05 08:15:48.649021 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:48.648971    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f14477b-08a3-4dcc-990a-4ee6456b7420" containerName="revoke-keys"
Sep 05 08:15:48.649021 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:48.648977    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f14477b-08a3-4dcc-990a-4ee6456b7420" containerName="revoke-keys"
Sep 05 08:15:48.649134 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:48.649042    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="13017a08-dd33-4206-a3df-5f0bef5ea494" containerName="payload-pre-processing"
Sep 05 08:15:48.649134 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:48.649052    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f14477b-08a3-4dcc-990a-4ee6456b7420" containerName="revoke-keys"
Sep 05 08:15:48.652079 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:48.652064    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3d4dc1-jqkxb"
Sep 05 08:15:48.654188 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:48.654166    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mpmmp\""
Sep 05 08:15:48.654312 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:48.654166    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 05 08:15:48.658000 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:48.657978    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3d4dc1-jqkxb"]
Sep 05 08:15:48.743630 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:48.743592    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:15:48.749800 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:48.749765    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcc26405-3169-47d8-8a96-490d7586cf0f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-3d4dc1-jqkxb\" (UID: \"fcc26405-3169-47d8-8a96-490d7586cf0f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3d4dc1-jqkxb"
Sep 05 08:15:48.850327 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:48.850293    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcc26405-3169-47d8-8a96-490d7586cf0f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-3d4dc1-jqkxb\" (UID: \"fcc26405-3169-47d8-8a96-490d7586cf0f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3d4dc1-jqkxb"
Sep 05 08:15:48.850961 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:48.850941    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcc26405-3169-47d8-8a96-490d7586cf0f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-3d4dc1-jqkxb\" (UID: \"fcc26405-3169-47d8-8a96-490d7586cf0f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3d4dc1-jqkxb"
Sep 05 08:15:48.963992 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:48.963901    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3d4dc1-jqkxb"
Sep 05 08:15:49.246736 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:49.246709    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3d4dc1-jqkxb"]
Sep 05 08:15:49.247983 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:15:49.247955    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcc26405_3169_47d8_8a96_490d7586cf0f.slice/crio-c9209c3dcc5c6e1b382f1de11c55a6e25da8cc9fdaa8829668c19538a564f609 WatchSource:0}: Error finding container c9209c3dcc5c6e1b382f1de11c55a6e25da8cc9fdaa8829668c19538a564f609: Status 404 returned error can't find the container with id c9209c3dcc5c6e1b382f1de11c55a6e25da8cc9fdaa8829668c19538a564f609
Sep 05 08:15:50.042375 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:50.042348    2563 generic.go:356] "Generic (PLEG): container finished" podID="fcc26405-3169-47d8-8a96-490d7586cf0f" containerID="ad7138028e43f6443e9afc2ef502ec21e16d8f6f65aa4777d5650a83bc728a2f" exitCode=0
Sep 05 08:15:50.042719 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:50.042434    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3d4dc1-jqkxb" event={"ID":"fcc26405-3169-47d8-8a96-490d7586cf0f","Type":"ContainerDied","Data":"ad7138028e43f6443e9afc2ef502ec21e16d8f6f65aa4777d5650a83bc728a2f"}
Sep 05 08:15:50.042719 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:50.042489    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3d4dc1-jqkxb" event={"ID":"fcc26405-3169-47d8-8a96-490d7586cf0f","Type":"ContainerStarted","Data":"c9209c3dcc5c6e1b382f1de11c55a6e25da8cc9fdaa8829668c19538a564f609"}
Sep 05 08:15:51.176271 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:51.176248    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3d4dc1-jqkxb"
Sep 05 08:15:51.270619 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:51.270589    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcc26405-3169-47d8-8a96-490d7586cf0f-maas-api-service-ca\") pod \"fcc26405-3169-47d8-8a96-490d7586cf0f\" (UID: \"fcc26405-3169-47d8-8a96-490d7586cf0f\") "
Sep 05 08:15:51.270973 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:51.270950    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcc26405-3169-47d8-8a96-490d7586cf0f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fcc26405-3169-47d8-8a96-490d7586cf0f" (UID: "fcc26405-3169-47d8-8a96-490d7586cf0f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:15:51.372259 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:51.372229    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcc26405-3169-47d8-8a96-490d7586cf0f-maas-api-service-ca\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:15:52.056385 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:52.056347    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3d4dc1-jqkxb" event={"ID":"fcc26405-3169-47d8-8a96-490d7586cf0f","Type":"ContainerDied","Data":"c9209c3dcc5c6e1b382f1de11c55a6e25da8cc9fdaa8829668c19538a564f609"}
Sep 05 08:15:52.056562 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:52.056388    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3d4dc1-jqkxb"
Sep 05 08:15:52.056562 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:15:52.056442    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c9209c3dcc5c6e1b382f1de11c55a6e25da8cc9fdaa8829668c19538a564f609"
Sep 05 08:16:04.157649 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:04.157613    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:16:04.221892 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:04.219399    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:16:04.281667 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:04.281634    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:16:04.324250 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:04.324212    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3d4dc1-jqkxb"]
Sep 05 08:16:04.328876 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:04.328831    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3d4dc1-jqkxb"]
Sep 05 08:16:04.381619 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:04.381583    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:16:04.707996 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:04.707969    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:16:05.730387 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:05.730353    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcc26405-3169-47d8-8a96-490d7586cf0f" path="/var/lib/kubelet/pods/fcc26405-3169-47d8-8a96-490d7586cf0f/volumes"
Sep 05 08:16:09.692742 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:09.692709    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-3d4dc1/ipp-route-3d4dc1-kserve-64b8465fff-6dwj6"]
Sep 05 08:16:09.787774 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:09.787740    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:16:24.714198 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:24.714136    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-3d4dc1/ipp-route-3d4dc1-kserve-64b8465fff-6dwj6" podUID="75ad22c4-6c8a-47ab-9d51-c457af6d60c1" containerName="main" containerID="cri-o://bb6553c4ee8983f21c1f6e9ef88530a03c99464ac5f281d4a3f61c2733cd4c08" gracePeriod=45
Sep 05 08:16:24.921231 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:24.921200    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:16:24.962924 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:24.962899    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-3d4dc1/ipp-route-3d4dc1-kserve-64b8465fff-6dwj6"
Sep 05 08:16:24.983226 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:24.983159    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:16:25.078170 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:25.078136    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75ad22c4-6c8a-47ab-9d51-c457af6d60c1-home\") pod \"75ad22c4-6c8a-47ab-9d51-c457af6d60c1\" (UID: \"75ad22c4-6c8a-47ab-9d51-c457af6d60c1\") "
Sep 05 08:16:25.078361 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:25.078180    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8lvjr\" (UniqueName: \"kubernetes.io/projected/75ad22c4-6c8a-47ab-9d51-c457af6d60c1-kube-api-access-8lvjr\") pod \"75ad22c4-6c8a-47ab-9d51-c457af6d60c1\" (UID: \"75ad22c4-6c8a-47ab-9d51-c457af6d60c1\") "
Sep 05 08:16:25.078361 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:25.078211    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75ad22c4-6c8a-47ab-9d51-c457af6d60c1-tmp-dir\") pod \"75ad22c4-6c8a-47ab-9d51-c457af6d60c1\" (UID: \"75ad22c4-6c8a-47ab-9d51-c457af6d60c1\") "
Sep 05 08:16:25.078361 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:25.078232    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75ad22c4-6c8a-47ab-9d51-c457af6d60c1-model-cache\") pod \"75ad22c4-6c8a-47ab-9d51-c457af6d60c1\" (UID: \"75ad22c4-6c8a-47ab-9d51-c457af6d60c1\") "
Sep 05 08:16:25.078361 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:25.078249    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75ad22c4-6c8a-47ab-9d51-c457af6d60c1-tls-certs\") pod \"75ad22c4-6c8a-47ab-9d51-c457af6d60c1\" (UID: \"75ad22c4-6c8a-47ab-9d51-c457af6d60c1\") "
Sep 05 08:16:25.078361 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:25.078278    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75ad22c4-6c8a-47ab-9d51-c457af6d60c1-dshm\") pod \"75ad22c4-6c8a-47ab-9d51-c457af6d60c1\" (UID: \"75ad22c4-6c8a-47ab-9d51-c457af6d60c1\") "
Sep 05 08:16:25.078627 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:25.078394    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75ad22c4-6c8a-47ab-9d51-c457af6d60c1-home" (OuterVolumeSpecName: "home") pod "75ad22c4-6c8a-47ab-9d51-c457af6d60c1" (UID: "75ad22c4-6c8a-47ab-9d51-c457af6d60c1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:16:25.078627 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:25.078493    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75ad22c4-6c8a-47ab-9d51-c457af6d60c1-model-cache" (OuterVolumeSpecName: "model-cache") pod "75ad22c4-6c8a-47ab-9d51-c457af6d60c1" (UID: "75ad22c4-6c8a-47ab-9d51-c457af6d60c1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:16:25.078627 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:25.078509    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75ad22c4-6c8a-47ab-9d51-c457af6d60c1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "75ad22c4-6c8a-47ab-9d51-c457af6d60c1" (UID: "75ad22c4-6c8a-47ab-9d51-c457af6d60c1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:16:25.078627 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:25.078602    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75ad22c4-6c8a-47ab-9d51-c457af6d60c1-home\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:16:25.078775 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:25.078642    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75ad22c4-6c8a-47ab-9d51-c457af6d60c1-tmp-dir\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:16:25.078775 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:25.078657    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75ad22c4-6c8a-47ab-9d51-c457af6d60c1-model-cache\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:16:25.080713 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:25.080678    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75ad22c4-6c8a-47ab-9d51-c457af6d60c1-dshm" (OuterVolumeSpecName: "dshm") pod "75ad22c4-6c8a-47ab-9d51-c457af6d60c1" (UID: "75ad22c4-6c8a-47ab-9d51-c457af6d60c1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:16:25.080713 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:25.080700    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75ad22c4-6c8a-47ab-9d51-c457af6d60c1-kube-api-access-8lvjr" (OuterVolumeSpecName: "kube-api-access-8lvjr") pod "75ad22c4-6c8a-47ab-9d51-c457af6d60c1" (UID: "75ad22c4-6c8a-47ab-9d51-c457af6d60c1"). InnerVolumeSpecName "kube-api-access-8lvjr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:16:25.080890 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:25.080704    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75ad22c4-6c8a-47ab-9d51-c457af6d60c1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "75ad22c4-6c8a-47ab-9d51-c457af6d60c1" (UID: "75ad22c4-6c8a-47ab-9d51-c457af6d60c1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:16:25.179826 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:25.179797    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8lvjr\" (UniqueName: \"kubernetes.io/projected/75ad22c4-6c8a-47ab-9d51-c457af6d60c1-kube-api-access-8lvjr\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:16:25.179826 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:25.179825    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75ad22c4-6c8a-47ab-9d51-c457af6d60c1-tls-certs\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:16:25.180033 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:25.179836    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75ad22c4-6c8a-47ab-9d51-c457af6d60c1-dshm\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:16:25.227709 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:25.227674    2563 generic.go:356] "Generic (PLEG): container finished" podID="75ad22c4-6c8a-47ab-9d51-c457af6d60c1" containerID="bb6553c4ee8983f21c1f6e9ef88530a03c99464ac5f281d4a3f61c2733cd4c08" exitCode=0
Sep 05 08:16:25.227912 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:25.227716    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-3d4dc1/ipp-route-3d4dc1-kserve-64b8465fff-6dwj6"
Sep 05 08:16:25.227912 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:25.227729    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-3d4dc1/ipp-route-3d4dc1-kserve-64b8465fff-6dwj6" event={"ID":"75ad22c4-6c8a-47ab-9d51-c457af6d60c1","Type":"ContainerDied","Data":"bb6553c4ee8983f21c1f6e9ef88530a03c99464ac5f281d4a3f61c2733cd4c08"}
Sep 05 08:16:25.227912 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:25.227771    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-3d4dc1/ipp-route-3d4dc1-kserve-64b8465fff-6dwj6" event={"ID":"75ad22c4-6c8a-47ab-9d51-c457af6d60c1","Type":"ContainerDied","Data":"83004793ec10247b16ea09577e97c053cda8113840d365da2ea3f7c0f0f5e3ae"}
Sep 05 08:16:25.227912 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:25.227793    2563 scope.go:117] "RemoveContainer" containerID="bb6553c4ee8983f21c1f6e9ef88530a03c99464ac5f281d4a3f61c2733cd4c08"
Sep 05 08:16:25.237022 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:25.237005    2563 scope.go:117] "RemoveContainer" containerID="bb6553c4ee8983f21c1f6e9ef88530a03c99464ac5f281d4a3f61c2733cd4c08"
Sep 05 08:16:25.237299 ip-10-0-133-14 kubenswrapper[2563]: E0905 08:16:25.237282    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb6553c4ee8983f21c1f6e9ef88530a03c99464ac5f281d4a3f61c2733cd4c08\": container with ID starting with bb6553c4ee8983f21c1f6e9ef88530a03c99464ac5f281d4a3f61c2733cd4c08 not found: ID does not exist" containerID="bb6553c4ee8983f21c1f6e9ef88530a03c99464ac5f281d4a3f61c2733cd4c08"
Sep 05 08:16:25.237361 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:25.237308    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb6553c4ee8983f21c1f6e9ef88530a03c99464ac5f281d4a3f61c2733cd4c08"} err="failed to get container status \"bb6553c4ee8983f21c1f6e9ef88530a03c99464ac5f281d4a3f61c2733cd4c08\": rpc error: code = NotFound desc = could not find container \"bb6553c4ee8983f21c1f6e9ef88530a03c99464ac5f281d4a3f61c2733cd4c08\": container with ID starting with bb6553c4ee8983f21c1f6e9ef88530a03c99464ac5f281d4a3f61c2733cd4c08 not found: ID does not exist"
Sep 05 08:16:25.248159 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:25.248135    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-3d4dc1/ipp-route-3d4dc1-kserve-64b8465fff-6dwj6"]
Sep 05 08:16:25.251407 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:25.251387    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-3d4dc1/ipp-route-3d4dc1-kserve-64b8465fff-6dwj6"]
Sep 05 08:16:25.730285 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:25.730252    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75ad22c4-6c8a-47ab-9d51-c457af6d60c1" path="/var/lib/kubelet/pods/75ad22c4-6c8a-47ab-9d51-c457af6d60c1/volumes"
Sep 05 08:16:44.609078 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:44.609034    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:16:45.832322 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:45.832288    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:16:46.680994 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:46.680962    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:16:52.130975 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:52.130937    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:16:57.692730 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:57.692691    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:16:57.748523 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:57.748495    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:16:57.989552 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:57.989465    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:16:58.371321 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:58.371285    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d14aea-d458m"]
Sep 05 08:16:58.371720 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:58.371702    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75ad22c4-6c8a-47ab-9d51-c457af6d60c1" containerName="main"
Sep 05 08:16:58.371808 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:58.371725    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="75ad22c4-6c8a-47ab-9d51-c457af6d60c1" containerName="main"
Sep 05 08:16:58.371808 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:58.371763    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcc26405-3169-47d8-8a96-490d7586cf0f" containerName="revoke-keys"
Sep 05 08:16:58.371808 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:58.371773    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcc26405-3169-47d8-8a96-490d7586cf0f" containerName="revoke-keys"
Sep 05 08:16:58.372011 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:58.371893    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcc26405-3169-47d8-8a96-490d7586cf0f" containerName="revoke-keys"
Sep 05 08:16:58.372011 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:58.371910    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="75ad22c4-6c8a-47ab-9d51-c457af6d60c1" containerName="main"
Sep 05 08:16:58.374976 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:58.374956    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d14aea-d458m"
Sep 05 08:16:58.377170 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:58.377142    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 05 08:16:58.377170 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:58.377156    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mpmmp\""
Sep 05 08:16:58.381209 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:58.381190    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d14aea-d458m"]
Sep 05 08:16:58.463031 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:58.462999    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:16:58.488936 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:58.488909    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6665ad57-fc42-486e-8133-69ff1da6712f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-d14aea-d458m\" (UID: \"6665ad57-fc42-486e-8133-69ff1da6712f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d14aea-d458m"
Sep 05 08:16:58.590056 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:58.590004    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6665ad57-fc42-486e-8133-69ff1da6712f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-d14aea-d458m\" (UID: \"6665ad57-fc42-486e-8133-69ff1da6712f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d14aea-d458m"
Sep 05 08:16:58.590647 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:58.590630    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6665ad57-fc42-486e-8133-69ff1da6712f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-d14aea-d458m\" (UID: \"6665ad57-fc42-486e-8133-69ff1da6712f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d14aea-d458m"
Sep 05 08:16:58.686149 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:58.686074    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d14aea-d458m"
Sep 05 08:16:58.964094 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:58.964070    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d14aea-d458m"]
Sep 05 08:16:58.966503 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:16:58.966472    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6665ad57_fc42_486e_8133_69ff1da6712f.slice/crio-5c6cfb8c77b243f013a09d918d7db26cdab53b26a919fa203a313a77c4dfc2c2 WatchSource:0}: Error finding container 5c6cfb8c77b243f013a09d918d7db26cdab53b26a919fa203a313a77c4dfc2c2: Status 404 returned error can't find the container with id 5c6cfb8c77b243f013a09d918d7db26cdab53b26a919fa203a313a77c4dfc2c2
Sep 05 08:16:59.404589 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:59.404550    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d14aea-d458m" event={"ID":"6665ad57-fc42-486e-8133-69ff1da6712f","Type":"ContainerStarted","Data":"35dec7b1691be0570057a243188ceeb45dccf10f14e63bdd322a1a4efd8a49ad"}
Sep 05 08:16:59.404783 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:59.404601    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d14aea-d458m" event={"ID":"6665ad57-fc42-486e-8133-69ff1da6712f","Type":"ContainerStarted","Data":"5c6cfb8c77b243f013a09d918d7db26cdab53b26a919fa203a313a77c4dfc2c2"}
Sep 05 08:16:59.419364 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:16:59.419322    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d14aea-d458m" podStartSLOduration=1.4193079960000001 podStartE2EDuration="1.419307996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:16:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:16:59.417644288 +0000 UTC m=+1722.187786638" watchObservedRunningTime="2026-09-05 08:16:59.419307996 +0000 UTC m=+1722.189450332"
Sep 05 08:17:00.411253 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:00.411214    2563 generic.go:356] "Generic (PLEG): container finished" podID="6665ad57-fc42-486e-8133-69ff1da6712f" containerID="35dec7b1691be0570057a243188ceeb45dccf10f14e63bdd322a1a4efd8a49ad" exitCode=0
Sep 05 08:17:00.411638 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:00.411296    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d14aea-d458m" event={"ID":"6665ad57-fc42-486e-8133-69ff1da6712f","Type":"ContainerDied","Data":"35dec7b1691be0570057a243188ceeb45dccf10f14e63bdd322a1a4efd8a49ad"}
Sep 05 08:17:01.546094 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:01.546069    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d14aea-d458m"
Sep 05 08:17:01.618057 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:01.618021    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6665ad57-fc42-486e-8133-69ff1da6712f-maas-api-service-ca\") pod \"6665ad57-fc42-486e-8133-69ff1da6712f\" (UID: \"6665ad57-fc42-486e-8133-69ff1da6712f\") "
Sep 05 08:17:01.618422 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:01.618398    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6665ad57-fc42-486e-8133-69ff1da6712f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6665ad57-fc42-486e-8133-69ff1da6712f" (UID: "6665ad57-fc42-486e-8133-69ff1da6712f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:17:01.719769 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:01.719662    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6665ad57-fc42-486e-8133-69ff1da6712f-maas-api-service-ca\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:17:02.423206 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:02.423179    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d14aea-d458m"
Sep 05 08:17:02.423206 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:02.423199    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d14aea-d458m" event={"ID":"6665ad57-fc42-486e-8133-69ff1da6712f","Type":"ContainerDied","Data":"5c6cfb8c77b243f013a09d918d7db26cdab53b26a919fa203a313a77c4dfc2c2"}
Sep 05 08:17:02.423462 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:02.423245    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5c6cfb8c77b243f013a09d918d7db26cdab53b26a919fa203a313a77c4dfc2c2"
Sep 05 08:17:08.821345 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:08.821307    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-d14aea-29809935-fdmx2"]
Sep 05 08:17:08.821815 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:08.821505    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:17:08.827504 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:08.827475    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-d14aea-29809935-fdmx2"]
Sep 05 08:17:08.850227 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:08.850198    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d14aea-79fb454499-4sb89"]
Sep 05 08:17:08.850660 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:08.850627    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-d14aea-79fb454499-4sb89" podUID="5cf0811d-1f0a-4452-a295-2d9c732ec0d5" containerName="payload-processing" containerID="cri-o://17fd1bb292067544e9a0e1a94355c1c8930bd2bab4ea1d601c4e1e33a17552de" gracePeriod=30
Sep 05 08:17:08.853515 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:08.853466    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d14aea-5c4975697d-h6q7p"]
Sep 05 08:17:08.854399 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:08.853711    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d14aea-5c4975697d-h6q7p" podUID="996b705a-1d70-4bf6-a84d-4e73d334c7f2" containerName="payload-pre-processing" containerID="cri-o://4870ad661d6f2da57e2693bf929697db4c7d4922f2cac79f3de96e7bd96ac40c" gracePeriod=30
Sep 05 08:17:08.893329 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:08.893275    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:17:08.961515 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:08.961456    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:17:09.001423 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:09.001350    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d14aea-d458m"]
Sep 05 08:17:09.003978 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:09.003954    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d14aea-d458m"]
Sep 05 08:17:09.019333 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:09.019305    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:17:09.126755 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:09.126731    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d14aea-79fb454499-4sb89"
Sep 05 08:17:09.129548 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:09.129527    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d14aea-5c4975697d-h6q7p"
Sep 05 08:17:09.192092 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:09.192052    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h45v9\" (UniqueName: \"kubernetes.io/projected/5cf0811d-1f0a-4452-a295-2d9c732ec0d5-kube-api-access-h45v9\") pod \"5cf0811d-1f0a-4452-a295-2d9c732ec0d5\" (UID: \"5cf0811d-1f0a-4452-a295-2d9c732ec0d5\") "
Sep 05 08:17:09.192283 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:09.192103    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9cfc2\" (UniqueName: \"kubernetes.io/projected/996b705a-1d70-4bf6-a84d-4e73d334c7f2-kube-api-access-9cfc2\") pod \"996b705a-1d70-4bf6-a84d-4e73d334c7f2\" (UID: \"996b705a-1d70-4bf6-a84d-4e73d334c7f2\") "
Sep 05 08:17:09.192283 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:09.192159    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5cf0811d-1f0a-4452-a295-2d9c732ec0d5-plugins-config-volume\") pod \"5cf0811d-1f0a-4452-a295-2d9c732ec0d5\" (UID: \"5cf0811d-1f0a-4452-a295-2d9c732ec0d5\") "
Sep 05 08:17:09.192283 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:09.192216    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/996b705a-1d70-4bf6-a84d-4e73d334c7f2-plugins-config-volume\") pod \"996b705a-1d70-4bf6-a84d-4e73d334c7f2\" (UID: \"996b705a-1d70-4bf6-a84d-4e73d334c7f2\") "
Sep 05 08:17:09.192590 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:09.192564    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5cf0811d-1f0a-4452-a295-2d9c732ec0d5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5cf0811d-1f0a-4452-a295-2d9c732ec0d5" (UID: "5cf0811d-1f0a-4452-a295-2d9c732ec0d5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:17:09.192670 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:09.192611    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/996b705a-1d70-4bf6-a84d-4e73d334c7f2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "996b705a-1d70-4bf6-a84d-4e73d334c7f2" (UID: "996b705a-1d70-4bf6-a84d-4e73d334c7f2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:17:09.194424 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:09.194394    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cf0811d-1f0a-4452-a295-2d9c732ec0d5-kube-api-access-h45v9" (OuterVolumeSpecName: "kube-api-access-h45v9") pod "5cf0811d-1f0a-4452-a295-2d9c732ec0d5" (UID: "5cf0811d-1f0a-4452-a295-2d9c732ec0d5"). InnerVolumeSpecName "kube-api-access-h45v9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:17:09.194762 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:09.194740    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/996b705a-1d70-4bf6-a84d-4e73d334c7f2-kube-api-access-9cfc2" (OuterVolumeSpecName: "kube-api-access-9cfc2") pod "996b705a-1d70-4bf6-a84d-4e73d334c7f2" (UID: "996b705a-1d70-4bf6-a84d-4e73d334c7f2"). InnerVolumeSpecName "kube-api-access-9cfc2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:17:09.293397 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:09.293361    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/996b705a-1d70-4bf6-a84d-4e73d334c7f2-plugins-config-volume\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:17:09.293397 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:09.293390    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h45v9\" (UniqueName: \"kubernetes.io/projected/5cf0811d-1f0a-4452-a295-2d9c732ec0d5-kube-api-access-h45v9\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:17:09.293397 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:09.293399    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9cfc2\" (UniqueName: \"kubernetes.io/projected/996b705a-1d70-4bf6-a84d-4e73d334c7f2-kube-api-access-9cfc2\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:17:09.293633 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:09.293411    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5cf0811d-1f0a-4452-a295-2d9c732ec0d5-plugins-config-volume\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:17:09.391240 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:09.391154    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:17:09.459366 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:09.459327    2563 generic.go:356] "Generic (PLEG): container finished" podID="5cf0811d-1f0a-4452-a295-2d9c732ec0d5" containerID="17fd1bb292067544e9a0e1a94355c1c8930bd2bab4ea1d601c4e1e33a17552de" exitCode=0
Sep 05 08:17:09.459561 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:09.459370    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d14aea-79fb454499-4sb89"
Sep 05 08:17:09.459561 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:09.459413    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d14aea-79fb454499-4sb89" event={"ID":"5cf0811d-1f0a-4452-a295-2d9c732ec0d5","Type":"ContainerDied","Data":"17fd1bb292067544e9a0e1a94355c1c8930bd2bab4ea1d601c4e1e33a17552de"}
Sep 05 08:17:09.459561 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:09.459470    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d14aea-79fb454499-4sb89" event={"ID":"5cf0811d-1f0a-4452-a295-2d9c732ec0d5","Type":"ContainerDied","Data":"4228eeb4b47dbeba9e7d07d7c8540716efd8a2060bc35e0e82a3c9cfb65fb803"}
Sep 05 08:17:09.459561 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:09.459500    2563 scope.go:117] "RemoveContainer" containerID="17fd1bb292067544e9a0e1a94355c1c8930bd2bab4ea1d601c4e1e33a17552de"
Sep 05 08:17:09.460895 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:09.460862    2563 generic.go:356] "Generic (PLEG): container finished" podID="996b705a-1d70-4bf6-a84d-4e73d334c7f2" containerID="4870ad661d6f2da57e2693bf929697db4c7d4922f2cac79f3de96e7bd96ac40c" exitCode=0
Sep 05 08:17:09.460995 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:09.460894    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d14aea-5c4975697d-h6q7p" event={"ID":"996b705a-1d70-4bf6-a84d-4e73d334c7f2","Type":"ContainerDied","Data":"4870ad661d6f2da57e2693bf929697db4c7d4922f2cac79f3de96e7bd96ac40c"}
Sep 05 08:17:09.460995 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:09.460921    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d14aea-5c4975697d-h6q7p"
Sep 05 08:17:09.460995 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:09.460946    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d14aea-5c4975697d-h6q7p" event={"ID":"996b705a-1d70-4bf6-a84d-4e73d334c7f2","Type":"ContainerDied","Data":"5bb0b49347797a2339228ee62c9d05ef499e28dcf2401349d28d8601332cd162"}
Sep 05 08:17:09.469713 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:09.469698    2563 scope.go:117] "RemoveContainer" containerID="17fd1bb292067544e9a0e1a94355c1c8930bd2bab4ea1d601c4e1e33a17552de"
Sep 05 08:17:09.470029 ip-10-0-133-14 kubenswrapper[2563]: E0905 08:17:09.469998    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17fd1bb292067544e9a0e1a94355c1c8930bd2bab4ea1d601c4e1e33a17552de\": container with ID starting with 17fd1bb292067544e9a0e1a94355c1c8930bd2bab4ea1d601c4e1e33a17552de not found: ID does not exist" containerID="17fd1bb292067544e9a0e1a94355c1c8930bd2bab4ea1d601c4e1e33a17552de"
Sep 05 08:17:09.470130 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:09.470040    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17fd1bb292067544e9a0e1a94355c1c8930bd2bab4ea1d601c4e1e33a17552de"} err="failed to get container status \"17fd1bb292067544e9a0e1a94355c1c8930bd2bab4ea1d601c4e1e33a17552de\": rpc error: code = NotFound desc = could not find container \"17fd1bb292067544e9a0e1a94355c1c8930bd2bab4ea1d601c4e1e33a17552de\": container with ID starting with 17fd1bb292067544e9a0e1a94355c1c8930bd2bab4ea1d601c4e1e33a17552de not found: ID does not exist"
Sep 05 08:17:09.470130 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:09.470065    2563 scope.go:117] "RemoveContainer" containerID="4870ad661d6f2da57e2693bf929697db4c7d4922f2cac79f3de96e7bd96ac40c"
Sep 05 08:17:09.478089 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:09.478074    2563 scope.go:117] "RemoveContainer" containerID="4870ad661d6f2da57e2693bf929697db4c7d4922f2cac79f3de96e7bd96ac40c"
Sep 05 08:17:09.478342 ip-10-0-133-14 kubenswrapper[2563]: E0905 08:17:09.478327    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4870ad661d6f2da57e2693bf929697db4c7d4922f2cac79f3de96e7bd96ac40c\": container with ID starting with 4870ad661d6f2da57e2693bf929697db4c7d4922f2cac79f3de96e7bd96ac40c not found: ID does not exist" containerID="4870ad661d6f2da57e2693bf929697db4c7d4922f2cac79f3de96e7bd96ac40c"
Sep 05 08:17:09.478380 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:09.478348    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4870ad661d6f2da57e2693bf929697db4c7d4922f2cac79f3de96e7bd96ac40c"} err="failed to get container status \"4870ad661d6f2da57e2693bf929697db4c7d4922f2cac79f3de96e7bd96ac40c\": rpc error: code = NotFound desc = could not find container \"4870ad661d6f2da57e2693bf929697db4c7d4922f2cac79f3de96e7bd96ac40c\": container with ID starting with 4870ad661d6f2da57e2693bf929697db4c7d4922f2cac79f3de96e7bd96ac40c not found: ID does not exist"
Sep 05 08:17:09.483458 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:09.483436    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d14aea-79fb454499-4sb89"]
Sep 05 08:17:09.485256 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:09.485239    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d14aea-79fb454499-4sb89"]
Sep 05 08:17:09.493835 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:09.493815    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d14aea-5c4975697d-h6q7p"]
Sep 05 08:17:09.497050 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:09.497030    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d14aea-5c4975697d-h6q7p"]
Sep 05 08:17:09.729725 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:09.729645    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5cf0811d-1f0a-4452-a295-2d9c732ec0d5" path="/var/lib/kubelet/pods/5cf0811d-1f0a-4452-a295-2d9c732ec0d5/volumes"
Sep 05 08:17:09.729989 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:09.729976    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6665ad57-fc42-486e-8133-69ff1da6712f" path="/var/lib/kubelet/pods/6665ad57-fc42-486e-8133-69ff1da6712f/volumes"
Sep 05 08:17:09.730282 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:09.730270    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="996b705a-1d70-4bf6-a84d-4e73d334c7f2" path="/var/lib/kubelet/pods/996b705a-1d70-4bf6-a84d-4e73d334c7f2/volumes"
Sep 05 08:17:09.730502 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:09.730492    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc954f0e-477e-44db-8f4c-c425905ceb19" path="/var/lib/kubelet/pods/fc954f0e-477e-44db-8f4c-c425905ceb19/volumes"
Sep 05 08:17:12.599341 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:12.599293    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:17:15.104815 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:15.104779    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:17:15.213943 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:15.213908    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:17:15.420363 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:15.420281    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:17:15.828965 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:15.828931    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-8c2e20-2bdwx"]
Sep 05 08:17:15.829267 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:15.829256    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6665ad57-fc42-486e-8133-69ff1da6712f" containerName="revoke-keys"
Sep 05 08:17:15.829312 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:15.829269    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6665ad57-fc42-486e-8133-69ff1da6712f" containerName="revoke-keys"
Sep 05 08:17:15.829312 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:15.829285    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cf0811d-1f0a-4452-a295-2d9c732ec0d5" containerName="payload-processing"
Sep 05 08:17:15.829312 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:15.829290    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cf0811d-1f0a-4452-a295-2d9c732ec0d5" containerName="payload-processing"
Sep 05 08:17:15.829312 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:15.829298    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="996b705a-1d70-4bf6-a84d-4e73d334c7f2" containerName="payload-pre-processing"
Sep 05 08:17:15.829312 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:15.829304    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="996b705a-1d70-4bf6-a84d-4e73d334c7f2" containerName="payload-pre-processing"
Sep 05 08:17:15.829468 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:15.829362    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="996b705a-1d70-4bf6-a84d-4e73d334c7f2" containerName="payload-pre-processing"
Sep 05 08:17:15.829468 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:15.829369    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cf0811d-1f0a-4452-a295-2d9c732ec0d5" containerName="payload-processing"
Sep 05 08:17:15.829468 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:15.829377    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6665ad57-fc42-486e-8133-69ff1da6712f" containerName="revoke-keys"
Sep 05 08:17:15.834174 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:15.834158    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-8c2e20-2bdwx"
Sep 05 08:17:15.836404 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:15.836373    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mpmmp\""
Sep 05 08:17:15.836894 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:15.836875    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 05 08:17:15.838717 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:15.838696    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-8c2e20-2bdwx"]
Sep 05 08:17:15.927181 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:15.927148    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:17:15.955234 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:15.955195    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f48cf9a-96c2-4c71-9e6a-3312d0df9926-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-8c2e20-2bdwx\" (UID: \"2f48cf9a-96c2-4c71-9e6a-3312d0df9926\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-8c2e20-2bdwx"
Sep 05 08:17:16.056410 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:16.056366    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f48cf9a-96c2-4c71-9e6a-3312d0df9926-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-8c2e20-2bdwx\" (UID: \"2f48cf9a-96c2-4c71-9e6a-3312d0df9926\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-8c2e20-2bdwx"
Sep 05 08:17:16.057076 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:16.057053    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f48cf9a-96c2-4c71-9e6a-3312d0df9926-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-8c2e20-2bdwx\" (UID: \"2f48cf9a-96c2-4c71-9e6a-3312d0df9926\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-8c2e20-2bdwx"
Sep 05 08:17:16.147080 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:16.147012    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-8c2e20-2bdwx"
Sep 05 08:17:16.416625 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:16.416540    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-8c2e20-2bdwx"]
Sep 05 08:17:16.419928 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:17:16.419898    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f48cf9a_96c2_4c71_9e6a_3312d0df9926.slice/crio-8b74f4aab31ccbfd830dfec13c6e30dc7eb86213c4ef85f5f937c0474256513a WatchSource:0}: Error finding container 8b74f4aab31ccbfd830dfec13c6e30dc7eb86213c4ef85f5f937c0474256513a: Status 404 returned error can't find the container with id 8b74f4aab31ccbfd830dfec13c6e30dc7eb86213c4ef85f5f937c0474256513a
Sep 05 08:17:16.500380 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:16.500353    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-8c2e20-2bdwx" event={"ID":"2f48cf9a-96c2-4c71-9e6a-3312d0df9926","Type":"ContainerStarted","Data":"8b74f4aab31ccbfd830dfec13c6e30dc7eb86213c4ef85f5f937c0474256513a"}
Sep 05 08:17:17.508365 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:17.508273    2563 generic.go:356] "Generic (PLEG): container finished" podID="2f48cf9a-96c2-4c71-9e6a-3312d0df9926" containerID="6405be57733429239aea5fc94ffab2c455346544c82421715833c7824c83af5c" exitCode=0
Sep 05 08:17:17.508365 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:17.508326    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-8c2e20-2bdwx" event={"ID":"2f48cf9a-96c2-4c71-9e6a-3312d0df9926","Type":"ContainerDied","Data":"6405be57733429239aea5fc94ffab2c455346544c82421715833c7824c83af5c"}
Sep 05 08:17:18.645227 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:18.645201    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-8c2e20-2bdwx"
Sep 05 08:17:18.784189 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:18.784099    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f48cf9a-96c2-4c71-9e6a-3312d0df9926-maas-api-service-ca\") pod \"2f48cf9a-96c2-4c71-9e6a-3312d0df9926\" (UID: \"2f48cf9a-96c2-4c71-9e6a-3312d0df9926\") "
Sep 05 08:17:18.784476 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:18.784452    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f48cf9a-96c2-4c71-9e6a-3312d0df9926-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2f48cf9a-96c2-4c71-9e6a-3312d0df9926" (UID: "2f48cf9a-96c2-4c71-9e6a-3312d0df9926"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:17:18.885036 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:18.884988    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f48cf9a-96c2-4c71-9e6a-3312d0df9926-maas-api-service-ca\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:17:19.520153 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:19.520120    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-8c2e20-2bdwx" event={"ID":"2f48cf9a-96c2-4c71-9e6a-3312d0df9926","Type":"ContainerDied","Data":"8b74f4aab31ccbfd830dfec13c6e30dc7eb86213c4ef85f5f937c0474256513a"}
Sep 05 08:17:19.520153 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:19.520149    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-8c2e20-2bdwx"
Sep 05 08:17:19.520363 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:19.520166    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8b74f4aab31ccbfd830dfec13c6e30dc7eb86213c4ef85f5f937c0474256513a"
Sep 05 08:17:26.230884 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:26.230806    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:17:26.300793 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:26.300753    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:17:26.369579 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:26.369537    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:17:26.409473 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:26.409437    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-8c2e20-2bdwx"]
Sep 05 08:17:26.414437 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:26.414410    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-8c2e20-2bdwx"]
Sep 05 08:17:26.438352 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:26.438327    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:17:26.832834 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:26.832806    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:17:27.730195 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:27.730165    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f48cf9a-96c2-4c71-9e6a-3312d0df9926" path="/var/lib/kubelet/pods/2f48cf9a-96c2-4c71-9e6a-3312d0df9926/volumes"
Sep 05 08:17:32.687441 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:32.687407    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:17:32.789614 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:32.789574    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:17:32.975464 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:32.975386    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:17:33.516170 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:33.516136    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:17:43.747720 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:43.747689    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:17:43.753948 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:43.753907    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-5ef325-29809935-xxrst"]
Sep 05 08:17:43.755960 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:43.755935    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-5ef325-29809935-xxrst"]
Sep 05 08:17:43.816372 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:43.816344    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:17:43.915862 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:43.915802    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:17:44.002478 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:44.002390    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:17:44.343680 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:44.343648    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:17:45.729864 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:45.729812    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4d8d2a7-3ab8-49ec-b21b-60edb0ecc8b4" path="/var/lib/kubelet/pods/f4d8d2a7-3ab8-49ec-b21b-60edb0ecc8b4/volumes"
Sep 05 08:17:50.141253 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:50.141216    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:17:50.198408 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:50.198375    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:17:50.397267 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:50.397170    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"]
Sep 05 08:17:50.397516 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:50.397483    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t" podUID="b72d3eee-5514-4bf7-ae01-a78e6c934bf3" containerName="istio-proxy" containerID="cri-o://1e89a5fbf8e89b406c875f4939b1bd3578e576b804bac1d38ab27492462eed41" gracePeriod=30
Sep 05 08:17:50.421467 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:50.421441    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:17:55.641903 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:55.641875    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"
Sep 05 08:17:55.713708 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:55.713614    2563 generic.go:356] "Generic (PLEG): container finished" podID="b72d3eee-5514-4bf7-ae01-a78e6c934bf3" containerID="1e89a5fbf8e89b406c875f4939b1bd3578e576b804bac1d38ab27492462eed41" exitCode=0
Sep 05 08:17:55.713708 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:55.713666    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"
Sep 05 08:17:55.713708 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:55.713695    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t" event={"ID":"b72d3eee-5514-4bf7-ae01-a78e6c934bf3","Type":"ContainerDied","Data":"1e89a5fbf8e89b406c875f4939b1bd3578e576b804bac1d38ab27492462eed41"}
Sep 05 08:17:55.713970 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:55.713748    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t" event={"ID":"b72d3eee-5514-4bf7-ae01-a78e6c934bf3","Type":"ContainerDied","Data":"9c6454b0f87711d4c214640aeb46b730ef8fad9cf684dcc86587fdddb9097a68"}
Sep 05 08:17:55.713970 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:55.713771    2563 scope.go:117] "RemoveContainer" containerID="1e89a5fbf8e89b406c875f4939b1bd3578e576b804bac1d38ab27492462eed41"
Sep 05 08:17:55.719437 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:55.719416    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-istio-token\") pod \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\" (UID: \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\") "
Sep 05 08:17:55.719563 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:55.719449    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-workload-socket\") pod \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\" (UID: \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\") "
Sep 05 08:17:55.719563 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:55.719484    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-credential-socket\") pod \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\" (UID: \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\") "
Sep 05 08:17:55.719684 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:55.719594    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-istio-envoy\") pod \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\" (UID: \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\") "
Sep 05 08:17:55.719684 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:55.719624    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-psnfd\" (UniqueName: \"kubernetes.io/projected/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-kube-api-access-psnfd\") pod \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\" (UID: \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\") "
Sep 05 08:17:55.719684 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:55.719662    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-istiod-ca-cert\") pod \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\" (UID: \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\") "
Sep 05 08:17:55.719831 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:55.719686    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-istio-data\") pod \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\" (UID: \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\") "
Sep 05 08:17:55.719831 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:55.719709    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-istio-podinfo\") pod \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\" (UID: \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\") "
Sep 05 08:17:55.719831 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:55.719751    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-workload-certs\") pod \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\" (UID: \"b72d3eee-5514-4bf7-ae01-a78e6c934bf3\") "
Sep 05 08:17:55.719831 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:55.719758    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b72d3eee-5514-4bf7-ae01-a78e6c934bf3" (UID: "b72d3eee-5514-4bf7-ae01-a78e6c934bf3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:17:55.720057 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:55.719823    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b72d3eee-5514-4bf7-ae01-a78e6c934bf3" (UID: "b72d3eee-5514-4bf7-ae01-a78e6c934bf3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:17:55.720057 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:55.719987    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-istio-data" (OuterVolumeSpecName: "istio-data") pod "b72d3eee-5514-4bf7-ae01-a78e6c934bf3" (UID: "b72d3eee-5514-4bf7-ae01-a78e6c934bf3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:17:55.720174 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:55.720053    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b72d3eee-5514-4bf7-ae01-a78e6c934bf3" (UID: "b72d3eee-5514-4bf7-ae01-a78e6c934bf3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:17:55.720174 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:55.720081    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-istio-data\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:17:55.720174 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:55.720122    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-workload-socket\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:17:55.720174 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:55.720135    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-credential-socket\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:17:55.720174 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:55.720167    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b72d3eee-5514-4bf7-ae01-a78e6c934bf3" (UID: "b72d3eee-5514-4bf7-ae01-a78e6c934bf3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:17:55.722103 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:55.722072    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b72d3eee-5514-4bf7-ae01-a78e6c934bf3" (UID: "b72d3eee-5514-4bf7-ae01-a78e6c934bf3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 05 08:17:55.722103 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:55.722082    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-kube-api-access-psnfd" (OuterVolumeSpecName: "kube-api-access-psnfd") pod "b72d3eee-5514-4bf7-ae01-a78e6c934bf3" (UID: "b72d3eee-5514-4bf7-ae01-a78e6c934bf3"). InnerVolumeSpecName "kube-api-access-psnfd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:17:55.722441 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:55.722417    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-istio-token" (OuterVolumeSpecName: "istio-token") pod "b72d3eee-5514-4bf7-ae01-a78e6c934bf3" (UID: "b72d3eee-5514-4bf7-ae01-a78e6c934bf3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:17:55.722441 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:55.722407    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b72d3eee-5514-4bf7-ae01-a78e6c934bf3" (UID: "b72d3eee-5514-4bf7-ae01-a78e6c934bf3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:17:55.723474 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:55.723459    2563 scope.go:117] "RemoveContainer" containerID="cf9f013bd65396c755ad6a3c35c761cc83b0a6dd1db95f1e0b51ca264100fb3b"
Sep 05 08:17:55.740632 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:55.740615    2563 scope.go:117] "RemoveContainer" containerID="1e89a5fbf8e89b406c875f4939b1bd3578e576b804bac1d38ab27492462eed41"
Sep 05 08:17:55.740931 ip-10-0-133-14 kubenswrapper[2563]: E0905 08:17:55.740912    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e89a5fbf8e89b406c875f4939b1bd3578e576b804bac1d38ab27492462eed41\": container with ID starting with 1e89a5fbf8e89b406c875f4939b1bd3578e576b804bac1d38ab27492462eed41 not found: ID does not exist" containerID="1e89a5fbf8e89b406c875f4939b1bd3578e576b804bac1d38ab27492462eed41"
Sep 05 08:17:55.740997 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:55.740938    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e89a5fbf8e89b406c875f4939b1bd3578e576b804bac1d38ab27492462eed41"} err="failed to get container status \"1e89a5fbf8e89b406c875f4939b1bd3578e576b804bac1d38ab27492462eed41\": rpc error: code = NotFound desc = could not find container \"1e89a5fbf8e89b406c875f4939b1bd3578e576b804bac1d38ab27492462eed41\": container with ID starting with 1e89a5fbf8e89b406c875f4939b1bd3578e576b804bac1d38ab27492462eed41 not found: ID does not exist"
Sep 05 08:17:55.740997 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:55.740955    2563 scope.go:117] "RemoveContainer" containerID="cf9f013bd65396c755ad6a3c35c761cc83b0a6dd1db95f1e0b51ca264100fb3b"
Sep 05 08:17:55.741170 ip-10-0-133-14 kubenswrapper[2563]: E0905 08:17:55.741150    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf9f013bd65396c755ad6a3c35c761cc83b0a6dd1db95f1e0b51ca264100fb3b\": container with ID starting with cf9f013bd65396c755ad6a3c35c761cc83b0a6dd1db95f1e0b51ca264100fb3b not found: ID does not exist" containerID="cf9f013bd65396c755ad6a3c35c761cc83b0a6dd1db95f1e0b51ca264100fb3b"
Sep 05 08:17:55.741230 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:55.741181    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf9f013bd65396c755ad6a3c35c761cc83b0a6dd1db95f1e0b51ca264100fb3b"} err="failed to get container status \"cf9f013bd65396c755ad6a3c35c761cc83b0a6dd1db95f1e0b51ca264100fb3b\": rpc error: code = NotFound desc = could not find container \"cf9f013bd65396c755ad6a3c35c761cc83b0a6dd1db95f1e0b51ca264100fb3b\": container with ID starting with cf9f013bd65396c755ad6a3c35c761cc83b0a6dd1db95f1e0b51ca264100fb3b not found: ID does not exist"
Sep 05 08:17:55.821346 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:55.821306    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-istiod-ca-cert\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:17:55.821346 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:55.821338    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-istio-podinfo\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:17:55.821346 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:55.821347    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-workload-certs\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:17:55.821346 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:55.821354    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-istio-token\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:17:55.821628 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:55.821363    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-istio-envoy\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:17:55.821628 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:55.821371    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-psnfd\" (UniqueName: \"kubernetes.io/projected/b72d3eee-5514-4bf7-ae01-a78e6c934bf3-kube-api-access-psnfd\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:17:56.031639 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:56.031567    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"]
Sep 05 08:17:56.037297 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:56.037275    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-5ef325-openshift-default-78d9c848db-cjq2t"]
Sep 05 08:17:57.735134 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:17:57.735098    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b72d3eee-5514-4bf7-ae01-a78e6c934bf3" path="/var/lib/kubelet/pods/b72d3eee-5514-4bf7-ae01-a78e6c934bf3/volumes"
Sep 05 08:18:17.803230 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:18:17.803203    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2bzp8_569cffae-1c5c-4b02-af57-9ca53c455e4c/ovn-acl-logging/0.log"
Sep 05 08:18:17.806815 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:18:17.806795    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2bzp8_569cffae-1c5c-4b02-af57-9ca53c455e4c/ovn-acl-logging/0.log"
Sep 05 08:18:18.103106 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:18:18.103076    2563 scope.go:117] "RemoveContainer" containerID="9f87c9e23371e68ef8213fc46ce35a99f29bd9a2f92be935a24b658a9977df36"
Sep 05 08:18:18.111283 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:18:18.111259    2563 scope.go:117] "RemoveContainer" containerID="a0c40aa01ad2dede63b5dd1a13d90d53edf30c0bf460e8ef22bb635df07f2d67"
Sep 05 08:18:18.119307 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:18:18.119283    2563 scope.go:117] "RemoveContainer" containerID="cae59ab90b0e9b9bc1e887a93a72be358576dda76360c50624e35f167ea972f3"
Sep 05 08:18:18.129407 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:18:18.129385    2563 scope.go:117] "RemoveContainer" containerID="1d3e3863f2327986517f00fa8b18c1308e111232c4e148ccd190a3d19426121d"
Sep 05 08:19:08.765201 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:19:08.765170    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:19:13.501208 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:19:13.501170    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:19:13.989615 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:19:13.989586    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:19:15.634125 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:19:15.634086    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:19:18.163808 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:19:18.163773    2563 scope.go:117] "RemoveContainer" containerID="d754a6269311700512b7987e20ba429a780bc7de98088b6c0959da8ffd8f962b"
Sep 05 08:19:18.172764 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:19:18.172744    2563 scope.go:117] "RemoveContainer" containerID="cf808fab0fdcfe306c1efe36e0e3acfdc5f2108cd1b50cbe9383fa882df6260f"
Sep 05 08:20:11.942200 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:20:11.942152    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:21:15.932452 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:21:15.932339    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:21:18.203951 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:21:18.203917    2563 scope.go:117] "RemoveContainer" containerID="1737ecc7f8a6e5a9c15122ca93cea30bfeb8c853da73d784c0ceb1fbe7c6747f"
Sep 05 08:21:18.212031 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:21:18.212007    2563 scope.go:117] "RemoveContainer" containerID="26308eeb67010b6bcc05b4511aadb496b7f6be7751e953b514b9acd75f22fd13"
Sep 05 08:22:10.509705 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:10.509630    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:22:18.232749 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:18.232717    2563 scope.go:117] "RemoveContainer" containerID="d22a0ab86207629ab8a927270e71e2363ffa245299214f037949409af9e16e87"
Sep 05 08:22:18.240914 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:18.240890    2563 scope.go:117] "RemoveContainer" containerID="ad7138028e43f6443e9afc2ef502ec21e16d8f6f65aa4777d5650a83bc728a2f"
Sep 05 08:22:20.995465 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:20.995429    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf"]
Sep 05 08:22:20.995833 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:20.995758    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b72d3eee-5514-4bf7-ae01-a78e6c934bf3" containerName="istio-proxy"
Sep 05 08:22:20.995833 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:20.995770    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b72d3eee-5514-4bf7-ae01-a78e6c934bf3" containerName="istio-proxy"
Sep 05 08:22:20.995833 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:20.995782    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b72d3eee-5514-4bf7-ae01-a78e6c934bf3" containerName="istio-proxy"
Sep 05 08:22:20.995833 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:20.995788    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b72d3eee-5514-4bf7-ae01-a78e6c934bf3" containerName="istio-proxy"
Sep 05 08:22:20.995833 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:20.995806    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f48cf9a-96c2-4c71-9e6a-3312d0df9926" containerName="revoke-keys"
Sep 05 08:22:20.995833 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:20.995811    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f48cf9a-96c2-4c71-9e6a-3312d0df9926" containerName="revoke-keys"
Sep 05 08:22:20.996099 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:20.995890    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f48cf9a-96c2-4c71-9e6a-3312d0df9926" containerName="revoke-keys"
Sep 05 08:22:20.996099 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:20.995900    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b72d3eee-5514-4bf7-ae01-a78e6c934bf3" containerName="istio-proxy"
Sep 05 08:22:20.998797 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:20.998779    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf"
Sep 05 08:22:21.000961 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:21.000940    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-cc5f87b2-kserve-self-signed-certs\""
Sep 05 08:22:21.008785 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:21.008763    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf"]
Sep 05 08:22:21.028113 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:21.028072    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:22:21.044403 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:21.044371    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11846242-bfa1-46b7-9710-7524453c86be-dshm\") pod \"e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf\" (UID: \"11846242-bfa1-46b7-9710-7524453c86be\") " pod="llm/e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf"
Sep 05 08:22:21.044485 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:21.044424    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11846242-bfa1-46b7-9710-7524453c86be-tmp-dir\") pod \"e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf\" (UID: \"11846242-bfa1-46b7-9710-7524453c86be\") " pod="llm/e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf"
Sep 05 08:22:21.044485 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:21.044445    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11846242-bfa1-46b7-9710-7524453c86be-tls-certs\") pod \"e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf\" (UID: \"11846242-bfa1-46b7-9710-7524453c86be\") " pod="llm/e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf"
Sep 05 08:22:21.044608 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:21.044557    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52tbg\" (UniqueName: \"kubernetes.io/projected/11846242-bfa1-46b7-9710-7524453c86be-kube-api-access-52tbg\") pod \"e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf\" (UID: \"11846242-bfa1-46b7-9710-7524453c86be\") " pod="llm/e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf"
Sep 05 08:22:21.044608 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:21.044591    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11846242-bfa1-46b7-9710-7524453c86be-model-cache\") pod \"e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf\" (UID: \"11846242-bfa1-46b7-9710-7524453c86be\") " pod="llm/e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf"
Sep 05 08:22:21.044704 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:21.044624    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11846242-bfa1-46b7-9710-7524453c86be-home\") pod \"e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf\" (UID: \"11846242-bfa1-46b7-9710-7524453c86be\") " pod="llm/e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf"
Sep 05 08:22:21.145192 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:21.145157    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11846242-bfa1-46b7-9710-7524453c86be-tls-certs\") pod \"e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf\" (UID: \"11846242-bfa1-46b7-9710-7524453c86be\") " pod="llm/e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf"
Sep 05 08:22:21.145392 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:21.145250    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52tbg\" (UniqueName: \"kubernetes.io/projected/11846242-bfa1-46b7-9710-7524453c86be-kube-api-access-52tbg\") pod \"e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf\" (UID: \"11846242-bfa1-46b7-9710-7524453c86be\") " pod="llm/e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf"
Sep 05 08:22:21.145392 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:21.145272    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11846242-bfa1-46b7-9710-7524453c86be-model-cache\") pod \"e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf\" (UID: \"11846242-bfa1-46b7-9710-7524453c86be\") " pod="llm/e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf"
Sep 05 08:22:21.145392 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:21.145294    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11846242-bfa1-46b7-9710-7524453c86be-home\") pod \"e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf\" (UID: \"11846242-bfa1-46b7-9710-7524453c86be\") " pod="llm/e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf"
Sep 05 08:22:21.145567 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:21.145445    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11846242-bfa1-46b7-9710-7524453c86be-dshm\") pod \"e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf\" (UID: \"11846242-bfa1-46b7-9710-7524453c86be\") " pod="llm/e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf"
Sep 05 08:22:21.145567 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:21.145536    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11846242-bfa1-46b7-9710-7524453c86be-tmp-dir\") pod \"e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf\" (UID: \"11846242-bfa1-46b7-9710-7524453c86be\") " pod="llm/e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf"
Sep 05 08:22:21.145704 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:21.145687    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11846242-bfa1-46b7-9710-7524453c86be-model-cache\") pod \"e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf\" (UID: \"11846242-bfa1-46b7-9710-7524453c86be\") " pod="llm/e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf"
Sep 05 08:22:21.145749 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:21.145725    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11846242-bfa1-46b7-9710-7524453c86be-home\") pod \"e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf\" (UID: \"11846242-bfa1-46b7-9710-7524453c86be\") " pod="llm/e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf"
Sep 05 08:22:21.145934 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:21.145910    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11846242-bfa1-46b7-9710-7524453c86be-tmp-dir\") pod \"e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf\" (UID: \"11846242-bfa1-46b7-9710-7524453c86be\") " pod="llm/e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf"
Sep 05 08:22:21.147698 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:21.147668    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11846242-bfa1-46b7-9710-7524453c86be-dshm\") pod \"e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf\" (UID: \"11846242-bfa1-46b7-9710-7524453c86be\") " pod="llm/e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf"
Sep 05 08:22:21.147871 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:21.147833    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11846242-bfa1-46b7-9710-7524453c86be-tls-certs\") pod \"e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf\" (UID: \"11846242-bfa1-46b7-9710-7524453c86be\") " pod="llm/e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf"
Sep 05 08:22:21.151806 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:21.151788    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52tbg\" (UniqueName: \"kubernetes.io/projected/11846242-bfa1-46b7-9710-7524453c86be-kube-api-access-52tbg\") pod \"e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf\" (UID: \"11846242-bfa1-46b7-9710-7524453c86be\") " pod="llm/e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf"
Sep 05 08:22:21.308359 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:21.308321    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf"
Sep 05 08:22:21.579883 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:21.579830    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf"]
Sep 05 08:22:21.582174 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:22:21.582129    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod11846242_bfa1_46b7_9710_7524453c86be.slice/crio-90c7896c99d219eb7c30a1505114d7a428d1237dc3f16a8f6ca49474b914e072 WatchSource:0}: Error finding container 90c7896c99d219eb7c30a1505114d7a428d1237dc3f16a8f6ca49474b914e072: Status 404 returned error can't find the container with id 90c7896c99d219eb7c30a1505114d7a428d1237dc3f16a8f6ca49474b914e072
Sep 05 08:22:22.003373 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:22.003280    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf" event={"ID":"11846242-bfa1-46b7-9710-7524453c86be","Type":"ContainerStarted","Data":"3242d0bdc95fea60f94d9ba2a8c4be8cd879bc73c1e94bb408dc13c6e03761b8"}
Sep 05 08:22:22.003373 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:22.003323    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf" event={"ID":"11846242-bfa1-46b7-9710-7524453c86be","Type":"ContainerStarted","Data":"90c7896c99d219eb7c30a1505114d7a428d1237dc3f16a8f6ca49474b914e072"}
Sep 05 08:22:22.020218 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:22.020170    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf" podStartSLOduration=2.020154701 podStartE2EDuration="2.020154701s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:22:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:22:22.017705903 +0000 UTC m=+2044.787848244" watchObservedRunningTime="2026-09-05 08:22:22.020154701 +0000 UTC m=+2044.790297038"
Sep 05 08:22:31.308864 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:31.308798    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf"
Sep 05 08:22:31.309297 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:31.308881    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf"
Sep 05 08:22:31.321158 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:31.321128    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf"
Sep 05 08:22:32.066563 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:32.066522    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf"
Sep 05 08:22:40.589734 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:40.589652    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:22:42.166543 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:42.166510    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf"]
Sep 05 08:22:42.188735 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:42.188702    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:22:51.895176 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:51.895139    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:22:51.997325 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:51.997288    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:22:57.187155 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:57.187091    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf" podUID="11846242-bfa1-46b7-9710-7524453c86be" containerName="main" containerID="cri-o://3242d0bdc95fea60f94d9ba2a8c4be8cd879bc73c1e94bb408dc13c6e03761b8" gracePeriod=45
Sep 05 08:22:57.438574 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:57.438517    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf"
Sep 05 08:22:57.561379 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:57.561342    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11846242-bfa1-46b7-9710-7524453c86be-home\") pod \"11846242-bfa1-46b7-9710-7524453c86be\" (UID: \"11846242-bfa1-46b7-9710-7524453c86be\") "
Sep 05 08:22:57.561562 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:57.561402    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-52tbg\" (UniqueName: \"kubernetes.io/projected/11846242-bfa1-46b7-9710-7524453c86be-kube-api-access-52tbg\") pod \"11846242-bfa1-46b7-9710-7524453c86be\" (UID: \"11846242-bfa1-46b7-9710-7524453c86be\") "
Sep 05 08:22:57.561562 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:57.561462    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11846242-bfa1-46b7-9710-7524453c86be-tmp-dir\") pod \"11846242-bfa1-46b7-9710-7524453c86be\" (UID: \"11846242-bfa1-46b7-9710-7524453c86be\") "
Sep 05 08:22:57.561562 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:57.561487    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11846242-bfa1-46b7-9710-7524453c86be-tls-certs\") pod \"11846242-bfa1-46b7-9710-7524453c86be\" (UID: \"11846242-bfa1-46b7-9710-7524453c86be\") "
Sep 05 08:22:57.561562 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:57.561533    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11846242-bfa1-46b7-9710-7524453c86be-dshm\") pod \"11846242-bfa1-46b7-9710-7524453c86be\" (UID: \"11846242-bfa1-46b7-9710-7524453c86be\") "
Sep 05 08:22:57.561562 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:57.561562    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11846242-bfa1-46b7-9710-7524453c86be-model-cache\") pod \"11846242-bfa1-46b7-9710-7524453c86be\" (UID: \"11846242-bfa1-46b7-9710-7524453c86be\") "
Sep 05 08:22:57.561833 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:57.561607    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11846242-bfa1-46b7-9710-7524453c86be-home" (OuterVolumeSpecName: "home") pod "11846242-bfa1-46b7-9710-7524453c86be" (UID: "11846242-bfa1-46b7-9710-7524453c86be"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:22:57.561833 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:57.561779    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11846242-bfa1-46b7-9710-7524453c86be-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "11846242-bfa1-46b7-9710-7524453c86be" (UID: "11846242-bfa1-46b7-9710-7524453c86be"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:22:57.561967 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:57.561814    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11846242-bfa1-46b7-9710-7524453c86be-home\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:22:57.561967 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:57.561871    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11846242-bfa1-46b7-9710-7524453c86be-model-cache" (OuterVolumeSpecName: "model-cache") pod "11846242-bfa1-46b7-9710-7524453c86be" (UID: "11846242-bfa1-46b7-9710-7524453c86be"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:22:57.563728 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:57.563700    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11846242-bfa1-46b7-9710-7524453c86be-kube-api-access-52tbg" (OuterVolumeSpecName: "kube-api-access-52tbg") pod "11846242-bfa1-46b7-9710-7524453c86be" (UID: "11846242-bfa1-46b7-9710-7524453c86be"). InnerVolumeSpecName "kube-api-access-52tbg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:22:57.563822 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:57.563773    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11846242-bfa1-46b7-9710-7524453c86be-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "11846242-bfa1-46b7-9710-7524453c86be" (UID: "11846242-bfa1-46b7-9710-7524453c86be"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:22:57.563822 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:57.563796    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11846242-bfa1-46b7-9710-7524453c86be-dshm" (OuterVolumeSpecName: "dshm") pod "11846242-bfa1-46b7-9710-7524453c86be" (UID: "11846242-bfa1-46b7-9710-7524453c86be"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:22:57.663340 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:57.663298    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11846242-bfa1-46b7-9710-7524453c86be-tmp-dir\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:22:57.663340 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:57.663331    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11846242-bfa1-46b7-9710-7524453c86be-tls-certs\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:22:57.663340 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:57.663344    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11846242-bfa1-46b7-9710-7524453c86be-dshm\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:22:57.663578 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:57.663355    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11846242-bfa1-46b7-9710-7524453c86be-model-cache\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:22:57.663578 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:57.663366    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52tbg\" (UniqueName: \"kubernetes.io/projected/11846242-bfa1-46b7-9710-7524453c86be-kube-api-access-52tbg\") on node \"ip-10-0-133-14.ec2.internal\" DevicePath \"\""
Sep 05 08:22:58.180243 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:58.180206    2563 generic.go:356] "Generic (PLEG): container finished" podID="11846242-bfa1-46b7-9710-7524453c86be" containerID="3242d0bdc95fea60f94d9ba2a8c4be8cd879bc73c1e94bb408dc13c6e03761b8" exitCode=0
Sep 05 08:22:58.180430 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:58.180251    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf"
Sep 05 08:22:58.180430 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:58.180250    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf" event={"ID":"11846242-bfa1-46b7-9710-7524453c86be","Type":"ContainerDied","Data":"3242d0bdc95fea60f94d9ba2a8c4be8cd879bc73c1e94bb408dc13c6e03761b8"}
Sep 05 08:22:58.180430 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:58.180326    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf" event={"ID":"11846242-bfa1-46b7-9710-7524453c86be","Type":"ContainerDied","Data":"90c7896c99d219eb7c30a1505114d7a428d1237dc3f16a8f6ca49474b914e072"}
Sep 05 08:22:58.180430 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:58.180348    2563 scope.go:117] "RemoveContainer" containerID="3242d0bdc95fea60f94d9ba2a8c4be8cd879bc73c1e94bb408dc13c6e03761b8"
Sep 05 08:22:58.188782 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:58.188606    2563 scope.go:117] "RemoveContainer" containerID="3242d0bdc95fea60f94d9ba2a8c4be8cd879bc73c1e94bb408dc13c6e03761b8"
Sep 05 08:22:58.189032 ip-10-0-133-14 kubenswrapper[2563]: E0905 08:22:58.188885    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3242d0bdc95fea60f94d9ba2a8c4be8cd879bc73c1e94bb408dc13c6e03761b8\": container with ID starting with 3242d0bdc95fea60f94d9ba2a8c4be8cd879bc73c1e94bb408dc13c6e03761b8 not found: ID does not exist" containerID="3242d0bdc95fea60f94d9ba2a8c4be8cd879bc73c1e94bb408dc13c6e03761b8"
Sep 05 08:22:58.189032 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:58.188912    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3242d0bdc95fea60f94d9ba2a8c4be8cd879bc73c1e94bb408dc13c6e03761b8"} err="failed to get container status \"3242d0bdc95fea60f94d9ba2a8c4be8cd879bc73c1e94bb408dc13c6e03761b8\": rpc error: code = NotFound desc = could not find container \"3242d0bdc95fea60f94d9ba2a8c4be8cd879bc73c1e94bb408dc13c6e03761b8\": container with ID starting with 3242d0bdc95fea60f94d9ba2a8c4be8cd879bc73c1e94bb408dc13c6e03761b8 not found: ID does not exist"
Sep 05 08:22:58.195914 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:58.195893    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf"]
Sep 05 08:22:58.199632 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:58.199610    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-cc5f87b2-kserve-ff5469fb7-xkkcf"]
Sep 05 08:22:59.729736 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:22:59.729699    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11846242-bfa1-46b7-9710-7524453c86be" path="/var/lib/kubelet/pods/11846242-bfa1-46b7-9710-7524453c86be/volumes"
Sep 05 08:23:17.830123 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:23:17.830096    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2bzp8_569cffae-1c5c-4b02-af57-9ca53c455e4c/ovn-acl-logging/0.log"
Sep 05 08:23:17.835013 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:23:17.834993    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2bzp8_569cffae-1c5c-4b02-af57-9ca53c455e4c/ovn-acl-logging/0.log"
Sep 05 08:23:18.263085 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:23:18.262989    2563 scope.go:117] "RemoveContainer" containerID="6405be57733429239aea5fc94ffab2c455346544c82421715833c7824c83af5c"
Sep 05 08:23:18.270994 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:23:18.270961    2563 scope.go:117] "RemoveContainer" containerID="35dec7b1691be0570057a243188ceeb45dccf10f14e63bdd322a1a4efd8a49ad"
Sep 05 08:23:39.080130 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:23:39.080101    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-54d58847c4-4sdn5_46f380f3-0a56-427b-85b1-c5463df61a92/authorino/0.log"
Sep 05 08:23:42.575303 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:23:42.575260    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:23:42.708306 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:23:42.708273    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:23:44.865543 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:23:44.865510    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-mq4pt_9b4d9b90-5a4e-491b-ab28-884452aac755/server/0.log"
Sep 05 08:23:44.984479 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:23:44.984452    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-bdp2h_ced931cd-0339-489d-b218-01ac7471b5f1/manager/0.log"
Sep 05 08:23:45.222130 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:23:45.222045    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9fc4f56d7-lqkkk_56de4a10-282d-48e5-b922-9e35da8fb5e2/copy-manifests/0.log"
Sep 05 08:23:45.227564 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:23:45.227538    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9fc4f56d7-lqkkk_56de4a10-282d-48e5-b922-9e35da8fb5e2/manager/0.log"
Sep 05 08:23:46.044044 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:23:46.044013    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29809935-qksv5_cdbfa942-081a-4296-898b-e3538a51051e/cleanup/0.log"
Sep 05 08:23:46.956641 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:23:46.956602    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-54d58847c4-4sdn5_46f380f3-0a56-427b-85b1-c5463df61a92/authorino/0.log"
Sep 05 08:23:48.025936 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:23:48.025906    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw_baa0f853-9766-4839-abc2-6da72b4ea0f0/istio-proxy/0.log"
Sep 05 08:23:48.366016 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:23:48.365989    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74c76cbcf8-f2jsh_f7d4a4c9-3912-4cff-b5e2-a06d853c81ca/kube-auth-proxy/0.log"
Sep 05 08:23:48.904789 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:23:48.904749    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-8557f77dd-lw4qb_9163aed9-c681-4b4d-a7f9-63438c02656f/router/0.log"
Sep 05 08:23:49.386134 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:23:49.386104    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-nxh4m_4712b98c-7254-4736-b545-55f5ca9eae13/main/0.log"
Sep 05 08:23:49.707586 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:23:49.707504    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fkpv_1d93ca88-04d7-45d1-ab86-7b1e7e20c9f3/main/0.log"
Sep 05 08:23:49.813954 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:23:49.813925    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-b628l_c5b025c7-bccc-417c-bf31-4e93ea8f8d95/main/0.log"
Sep 05 08:23:49.924525 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:23:49.924499    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-9k4qm_1b5b1ba7-aad0-4a90-9e8e-8159cd5ce603/main/0.log"
Sep 05 08:23:53.430014 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:23:53.429980    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:23:56.116480 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:23:56.116444    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:23:56.211637 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:23:56.211606    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log" limit=1024
Sep 05 08:23:59.091723 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:23:59.091687    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-lx5db_7e352b34-8143-4a94-86e7-11a9460d2946/global-pull-secret-syncer/0.log"
Sep 05 08:23:59.177783 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:23:59.177753    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8phbx_155f061b-e464-41b8-bf23-4183a33dd370/konnectivity-agent/0.log"
Sep 05 08:23:59.259736 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:23:59.259708    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-14.ec2.internal_5bcee7ae8d0b6018ec80e28e7bc76ca5/haproxy/0.log"
Sep 05 08:24:04.110696 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:04.110655    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-54d58847c4-4sdn5_46f380f3-0a56-427b-85b1-c5463df61a92/authorino/0.log"
Sep 05 08:24:06.178266 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:06.178235    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-94gwm_c91637a2-daa6-4523-8a9e-b2da9321546f/node-exporter/0.log"
Sep 05 08:24:06.199748 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:06.199717    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-94gwm_c91637a2-daa6-4523-8a9e-b2da9321546f/kube-rbac-proxy/0.log"
Sep 05 08:24:06.222658 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:06.222635    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-94gwm_c91637a2-daa6-4523-8a9e-b2da9321546f/init-textfile/0.log"
Sep 05 08:24:08.530527 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:08.530492    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-g27lz_cafd9630-0d63-4c63-9829-bb8f215e97ac/networking-console-plugin/0.log"
Sep 05 08:24:09.336117 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:09.336076    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-24lm5/perf-node-gather-daemonset-fcl72"]
Sep 05 08:24:09.336535 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:09.336509    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11846242-bfa1-46b7-9710-7524453c86be" containerName="main"
Sep 05 08:24:09.336535 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:09.336533    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="11846242-bfa1-46b7-9710-7524453c86be" containerName="main"
Sep 05 08:24:09.336739 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:09.336706    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b72d3eee-5514-4bf7-ae01-a78e6c934bf3" containerName="istio-proxy"
Sep 05 08:24:09.336739 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:09.336717    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="11846242-bfa1-46b7-9710-7524453c86be" containerName="main"
Sep 05 08:24:09.339726 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:09.339709    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fcl72"
Sep 05 08:24:09.341832 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:09.341814    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-24lm5\"/\"default-dockercfg-z7vr5\""
Sep 05 08:24:09.341971 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:09.341830    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-24lm5\"/\"kube-root-ca.crt\""
Sep 05 08:24:09.342441 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:09.342421    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-24lm5\"/\"openshift-service-ca.crt\""
Sep 05 08:24:09.348618 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:09.348597    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-24lm5/perf-node-gather-daemonset-fcl72"]
Sep 05 08:24:09.406181 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:09.406152    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0185b3bc-32a0-45f8-8b20-a22232697823-podres\") pod \"perf-node-gather-daemonset-fcl72\" (UID: \"0185b3bc-32a0-45f8-8b20-a22232697823\") " pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fcl72"
Sep 05 08:24:09.406315 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:09.406188    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0185b3bc-32a0-45f8-8b20-a22232697823-sys\") pod \"perf-node-gather-daemonset-fcl72\" (UID: \"0185b3bc-32a0-45f8-8b20-a22232697823\") " pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fcl72"
Sep 05 08:24:09.406315 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:09.406208    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0185b3bc-32a0-45f8-8b20-a22232697823-proc\") pod \"perf-node-gather-daemonset-fcl72\" (UID: \"0185b3bc-32a0-45f8-8b20-a22232697823\") " pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fcl72"
Sep 05 08:24:09.406315 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:09.406236    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0185b3bc-32a0-45f8-8b20-a22232697823-lib-modules\") pod \"perf-node-gather-daemonset-fcl72\" (UID: \"0185b3bc-32a0-45f8-8b20-a22232697823\") " pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fcl72"
Sep 05 08:24:09.406315 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:09.406288    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n45nk\" (UniqueName: \"kubernetes.io/projected/0185b3bc-32a0-45f8-8b20-a22232697823-kube-api-access-n45nk\") pod \"perf-node-gather-daemonset-fcl72\" (UID: \"0185b3bc-32a0-45f8-8b20-a22232697823\") " pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fcl72"
Sep 05 08:24:09.506669 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:09.506641    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0185b3bc-32a0-45f8-8b20-a22232697823-sys\") pod \"perf-node-gather-daemonset-fcl72\" (UID: \"0185b3bc-32a0-45f8-8b20-a22232697823\") " pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fcl72"
Sep 05 08:24:09.506669 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:09.506674    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0185b3bc-32a0-45f8-8b20-a22232697823-proc\") pod \"perf-node-gather-daemonset-fcl72\" (UID: \"0185b3bc-32a0-45f8-8b20-a22232697823\") " pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fcl72"
Sep 05 08:24:09.506889 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:09.506704    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0185b3bc-32a0-45f8-8b20-a22232697823-lib-modules\") pod \"perf-node-gather-daemonset-fcl72\" (UID: \"0185b3bc-32a0-45f8-8b20-a22232697823\") " pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fcl72"
Sep 05 08:24:09.506889 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:09.506737    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n45nk\" (UniqueName: \"kubernetes.io/projected/0185b3bc-32a0-45f8-8b20-a22232697823-kube-api-access-n45nk\") pod \"perf-node-gather-daemonset-fcl72\" (UID: \"0185b3bc-32a0-45f8-8b20-a22232697823\") " pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fcl72"
Sep 05 08:24:09.506889 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:09.506784    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0185b3bc-32a0-45f8-8b20-a22232697823-proc\") pod \"perf-node-gather-daemonset-fcl72\" (UID: \"0185b3bc-32a0-45f8-8b20-a22232697823\") " pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fcl72"
Sep 05 08:24:09.506889 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:09.506784    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0185b3bc-32a0-45f8-8b20-a22232697823-sys\") pod \"perf-node-gather-daemonset-fcl72\" (UID: \"0185b3bc-32a0-45f8-8b20-a22232697823\") " pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fcl72"
Sep 05 08:24:09.506889 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:09.506819    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0185b3bc-32a0-45f8-8b20-a22232697823-podres\") pod \"perf-node-gather-daemonset-fcl72\" (UID: \"0185b3bc-32a0-45f8-8b20-a22232697823\") " pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fcl72"
Sep 05 08:24:09.506889 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:09.506870    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0185b3bc-32a0-45f8-8b20-a22232697823-lib-modules\") pod \"perf-node-gather-daemonset-fcl72\" (UID: \"0185b3bc-32a0-45f8-8b20-a22232697823\") " pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fcl72"
Sep 05 08:24:09.507086 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:09.506946    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0185b3bc-32a0-45f8-8b20-a22232697823-podres\") pod \"perf-node-gather-daemonset-fcl72\" (UID: \"0185b3bc-32a0-45f8-8b20-a22232697823\") " pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fcl72"
Sep 05 08:24:09.513989 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:09.513967    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n45nk\" (UniqueName: \"kubernetes.io/projected/0185b3bc-32a0-45f8-8b20-a22232697823-kube-api-access-n45nk\") pod \"perf-node-gather-daemonset-fcl72\" (UID: \"0185b3bc-32a0-45f8-8b20-a22232697823\") " pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fcl72"
Sep 05 08:24:09.649889 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:09.649776    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fcl72"
Sep 05 08:24:09.734185 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:09.734153    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-8p5k6_c7f6c7f2-bb81-4da8-8ff6-2be8dae1554d/download-server/0.log"
Sep 05 08:24:09.919298 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:09.919167    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-24lm5/perf-node-gather-daemonset-fcl72"]
Sep 05 08:24:09.921436 ip-10-0-133-14 kubenswrapper[2563]: W0905 08:24:09.921410    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0185b3bc_32a0_45f8_8b20_a22232697823.slice/crio-7fec9c27b447c4b9deed72ff9c2de532f8ec4148ce075ad103a18b2c7562db78 WatchSource:0}: Error finding container 7fec9c27b447c4b9deed72ff9c2de532f8ec4148ce075ad103a18b2c7562db78: Status 404 returned error can't find the container with id 7fec9c27b447c4b9deed72ff9c2de532f8ec4148ce075ad103a18b2c7562db78
Sep 05 08:24:10.423900 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:10.423871    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-vtq4p_9da5c2fc-7ecf-41c4-b814-ae36d6909e16/volume-data-source-validator/0.log"
Sep 05 08:24:10.513218 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:10.513185    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fcl72" event={"ID":"0185b3bc-32a0-45f8-8b20-a22232697823","Type":"ContainerStarted","Data":"49652fcc425e4c4aff02b6c44d26e46b6adfa49f5b2c09ffa7bfc90746a23b73"}
Sep 05 08:24:10.513400 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:10.513246    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fcl72" event={"ID":"0185b3bc-32a0-45f8-8b20-a22232697823","Type":"ContainerStarted","Data":"7fec9c27b447c4b9deed72ff9c2de532f8ec4148ce075ad103a18b2c7562db78"}
Sep 05 08:24:10.513400 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:10.513285    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fcl72"
Sep 05 08:24:10.527939 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:10.527891    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fcl72" podStartSLOduration=1.527873639 podStartE2EDuration="1.527873639s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 08:24:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:24:10.527390677 +0000 UTC m=+2153.297533042" watchObservedRunningTime="2026-09-05 08:24:10.527873639 +0000 UTC m=+2153.298015973"
Sep 05 08:24:11.578130 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:11.578099    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9wmv2_2de9d5a3-384d-4f3e-866f-19f1b5d4126d/dns/0.log"
Sep 05 08:24:11.599094 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:11.599074    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9wmv2_2de9d5a3-384d-4f3e-866f-19f1b5d4126d/kube-rbac-proxy/0.log"
Sep 05 08:24:11.692828 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:11.692790    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fstjt_f0fd7e0b-f07e-43a2-a169-667bc753a36b/dns-node-resolver/0.log"
Sep 05 08:24:12.242833 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:12.242799    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-768c75965-mrkgv_5cfee253-a143-4b62-af4a-54a94e8d2c04/registry/0.log"
Sep 05 08:24:12.288873 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:12.288829    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-m7775_35c73ae3-8ad8-443c-ac44-c9cd3396e526/node-ca/0.log"
Sep 05 08:24:13.225314 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:13.225283    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw_baa0f853-9766-4839-abc2-6da72b4ea0f0/istio-proxy/0.log"
Sep 05 08:24:13.654516 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:13.654482    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74c76cbcf8-f2jsh_f7d4a4c9-3912-4cff-b5e2-a06d853c81ca/kube-auth-proxy/0.log"
Sep 05 08:24:13.785634 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:13.785590    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-8557f77dd-lw4qb_9163aed9-c681-4b4d-a7f9-63438c02656f/router/0.log"
Sep 05 08:24:14.654133 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:14.654104    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-q6zgs_f2e61eb0-2689-4dfd-8b47-420342ee2916/serve-healthcheck-canary/0.log"
Sep 05 08:24:15.135248 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:15.135218    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-dcbfv_999f5178-4c4b-4888-a348-c0d434b38967/insights-operator/0.log"
Sep 05 08:24:15.135622 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:15.135606    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-dcbfv_999f5178-4c4b-4888-a348-c0d434b38967/insights-operator/1.log"
Sep 05 08:24:15.233152 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:15.233126    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mtq5b_3387beaa-e6d7-4f0a-bc56-0c06ef410d80/kube-rbac-proxy/0.log"
Sep 05 08:24:15.254088 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:15.254064    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mtq5b_3387beaa-e6d7-4f0a-bc56-0c06ef410d80/exporter/0.log"
Sep 05 08:24:15.275167 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:15.275146    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mtq5b_3387beaa-e6d7-4f0a-bc56-0c06ef410d80/extractor/0.log"
Sep 05 08:24:16.528407 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:16.528371    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fcl72"
Sep 05 08:24:17.650948 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:17.650919    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-mq4pt_9b4d9b90-5a4e-491b-ab28-884452aac755/server/0.log"
Sep 05 08:24:17.733504 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:17.733469    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-bdp2h_ced931cd-0339-489d-b218-01ac7471b5f1/manager/0.log"
Sep 05 08:24:17.809156 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:17.809130    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9fc4f56d7-lqkkk_56de4a10-282d-48e5-b922-9e35da8fb5e2/manager/0.log"
Sep 05 08:24:17.835052 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:17.835030    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9fc4f56d7-lqkkk_56de4a10-282d-48e5-b922-9e35da8fb5e2/copy-manifests/0.log"
Sep 05 08:24:19.339296 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:19.339267    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-bd655598-g899n_d0690683-7305-4e76-ab7c-183a766d282e/manager/0.log"
Sep 05 08:24:19.398654 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:19.398622    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-c6dbv_509d9d1e-d7cf-4477-9eb1-02281b7fd194/openshift-lws-operator/0.log"
Sep 05 08:24:20.259073 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:20.259035    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw_baa0f853-9766-4839-abc2-6da72b4ea0f0/istio-proxy/0.log"
Sep 05 08:24:20.679656 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:20.679623    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74c76cbcf8-f2jsh_f7d4a4c9-3912-4cff-b5e2-a06d853c81ca/kube-auth-proxy/0.log"
Sep 05 08:24:20.777295 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:20.777270    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-8557f77dd-lw4qb_9163aed9-c681-4b4d-a7f9-63438c02656f/router/0.log"
Sep 05 08:24:22.271300 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:22.271273    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-n4gxw_baa0f853-9766-4839-abc2-6da72b4ea0f0/istio-proxy/0.log"
Sep 05 08:24:22.685285 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:22.685258    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74c76cbcf8-f2jsh_f7d4a4c9-3912-4cff-b5e2-a06d853c81ca/kube-auth-proxy/0.log"
Sep 05 08:24:22.790312 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:22.790271    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-8557f77dd-lw4qb_9163aed9-c681-4b4d-a7f9-63438c02656f/router/0.log"
Sep 05 08:24:25.016532 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:25.016499    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-rzj45_143aee12-5c44-440c-ac06-1a5146e2d401/kube-storage-version-migrator-operator/1.log"
Sep 05 08:24:25.017358 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:25.017338    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-rzj45_143aee12-5c44-440c-ac06-1a5146e2d401/kube-storage-version-migrator-operator/0.log"
Sep 05 08:24:26.008051 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:26.008023    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cd8wl_4d02d57b-ec85-4cd4-8da2-0ffee0574984/kube-multus-additional-cni-plugins/0.log"
Sep 05 08:24:26.030257 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:26.030234    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cd8wl_4d02d57b-ec85-4cd4-8da2-0ffee0574984/egress-router-binary-copy/0.log"
Sep 05 08:24:26.051597 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:26.051575    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cd8wl_4d02d57b-ec85-4cd4-8da2-0ffee0574984/cni-plugins/0.log"
Sep 05 08:24:26.072948 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:26.072927    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cd8wl_4d02d57b-ec85-4cd4-8da2-0ffee0574984/bond-cni-plugin/0.log"
Sep 05 08:24:26.098128 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:26.098103    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cd8wl_4d02d57b-ec85-4cd4-8da2-0ffee0574984/routeoverride-cni/0.log"
Sep 05 08:24:26.120009 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:26.119986    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cd8wl_4d02d57b-ec85-4cd4-8da2-0ffee0574984/whereabouts-cni-bincopy/0.log"
Sep 05 08:24:26.141266 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:26.141247    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cd8wl_4d02d57b-ec85-4cd4-8da2-0ffee0574984/whereabouts-cni/0.log"
Sep 05 08:24:26.513622 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:26.513589    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-cvxlh_f08f40f6-ff0d-419e-9268-fe051987663e/kube-multus/0.log"
Sep 05 08:24:26.627654 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:26.627617    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-l9lgq_404c81cf-16aa-49f0-9059-fb97a5791ed7/network-metrics-daemon/0.log"
Sep 05 08:24:26.646776 ip-10-0-133-14 kubenswrapper[2563]: I0905 08:24:26.646749    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-l9lgq_404c81cf-16aa-49f0-9059-fb97a5791ed7/kube-rbac-proxy/0.log"